The future of serverless computers and relational databases
Serverless computing is all about shifting focus to applications that require no infrastructure management and consume resources only for the times they are active. In the public cloud space, serverless usually where the cloud provider manages the allocation of server resources based on workload requirements. Amazon Web Service's Lambda was at the forefront, with Microsoft's Azure and others quickly catching up. Pricing for serverless frameworks is generally based on the actual computing resources consumed by the application, instead of prepurchased capacity. What is the impact of serverless computing for stateless applications on relational database? Relational databases are crucial components for many applications.
We have had several well-proven choices when it comes to deploying relational databases; from micro services to monolithic solutions and to platform-as-a-service solutions. We can deploy a single, large server, running a monolithic or consolidated database powering dozens of applications. We can also choose a micro services oriented platform with a suite of small, modular and independent services, each allowing a unique process that serves a specific goal. Adopting cloud services also provides us with the ability to deploy our databases via "infrastructure as a code" and leverage platform-as-a-service solutions, minimizing overhead and complexity for our databases.
However, they all still rely on the provisioning of database servers. Be it on premises, in the cloud, or using PaaS. We still provision our database capacity based on predicted workload that determines the size and configuration of our servers. We still have the ability to scale up, scale down, or scale out our database in response to workloads but it isn't meant to be done frequently. Instead, we are supposed to scale in response to periodic events, such as holiday seasons that will generate additional transactions for your e-commerce application or scale in response to a big, new client for our company's product. Having dedicated database servers makes sense when workloads are predictable and relatively constant. There could be peaks and dips, but they are usually predictable. We may need scale our database a few times in a year, but not a few times a day. Infrequent database scaling is a model best suited for traditional applications.
Future generation applications will introduce new challenges. Some of these workloads may be highly unpredictable. For example, bursts of database queries or transactions that might span just minutes or hours each day. Using the same e-commerce app example from earlier, how can our databases provide support for flash sales events without over-provisioning your database servers? Similar challenges exist for other workloads, ranging from online gaming, stock trading, and even analytics. Most database administrators will say that we should size our database according to our forecasted peak workload. That's the current and the correct paradigm when the process of scaling your database is a chore.
To leverage the serverless computing paradigm for the database, we need to decouple the storage and processing layers of our data architecture. Decoupling storage and compute isn't exactly new. The idea has already been implemented in the NoSQL and big data analytics space as well as by various relational database technologies such as Oracle RAC. Decoupling storage and compute isn't what we'd call serverless. To be fully serverless, it shouldn't exist during periods that it doesn't process data while also providing on-demand autoscaling. It should consist of a data architecture where the database layer will automatically start, shut down, and scale up or down based on the application workloads. We should only need to define a database endpoint and connect your applications. The underlying database technology should scale the storage and computing resources depending on the application needs at the time. In addition to the benefits in performance and flexibility, serverless database models should be cost effective.
Oracle RAC, Amazon Aurora and others offer read / write scalable relational databases but are not natively serverless. Google Cloud Spanner, Microsoft Cosmos DB and FaunaDB offer innovative solutions for serverless databases. But applications will have to be rewritten to go through extensive re-platforming to use these database technologies. For example, Google Spanner requires custom client libraries for connectivity and provides a version of SQL that is handled by a custom API. Combining serverless computing with on demand relational databases will require complete server scalability and maintain full ANSI SQL and ACID support. Amazon made an announcement at last year's Re:Invent conference about a serverless version of the Aurora MySQL database that will be released in 2018. According to Amazon, Aurora Serverless is designed for "workloads that are highly variable and subject to rapid change, this new configuration allows you to pay for the database resources you use, on a second-by-second basis." Amazon states that Aurora Serverless users will only pay for the processing when the database is active and for used storage. Amazon is building the database equivalent of an event-driven computing platform. Users can provision an endpoint, which acts as a proxy that will route queries to a rapidly scaled fleet of database resources. Aurora Serverless will allow our connections to remain active even as scaling operations take place. Amazon also claims that scaling will also come online within seconds.