AWS Lambda: Reusing database connections

Since I started to work with AWS Lambda, the complexity of my projects reduced dramatically, the idea to not get annoyed with hardware and scale immediately and near infinite is amazing. Nowadays I've been using a simple FastHttp Golang server running at Elastic Beanstalk as a proxy to connect my Lambdas with the world via AWS SDK (acting like API Gateway), and a mix of them written in Golang and Node.js.

What makes AWS Lambda amazing is that it runs in containers, the service takes care to create and remove them as needed automatically, what makes Lambda scalable. Each Lambda container is stateless, it means, no way to store stuffs for a long time, also, could be tricky to keep a persistent connection to a database.

At normal circumstances, using normal servers, it won’t happen, because the server keeps a persistent connection for each running node, which connects and disconnects just once when it starts and stops, and usually, server nodes runs for long times. But with Lambda (or any other serverless service or FaaS), each container needs to have your own connection, and containers starts and stops at any moment, according to workload, and many times, it means, 1 or 1000 containers in seconds, which represents thousands of connections and disconnections. Setting up a new database connection is expensive, takes time, and the database itself maybe won't like to have thousands of clients connecting and disconnecting along the time. Luckily, Lambda allow us Freeze some stuffs to reuse them along the container's life.

The AWS Lambda documentation says:

After a Lambda function is executed, AWS Lambda maintains the Execution Context for some time in anticipation of another Lambda function invocation. In effect, the service freezes the Execution Context after a Lambda function completes, and thaws the context for reuse, if AWS Lambda chooses to reuse the context when the Lambda function is invoked again. This Execution Context reuse approach has the following implications:

Any declarations in your Lambda function code (outside the handler code, see Programming Model) remains initialized, providing additional optimization when the function is invoked again. For example, if your Lambda function establishes a database connection, instead of reestablishing the connection, the original connection is used in subsequent invocations…

…do not assume that AWS Lambda automatically reuses the Execution Context for subsequent function invocations…

Yeah, any code outside the handler will be frozen between lambda invocations, and there, is the right place to start your database connection, because it will be possibly reused.

Ok, but…

Using Node.js environments, AWS Lambda never returns unless you finish all background processes, it includes, database connections. This is because Lambda waits for Node.js' event loop to be clear before returns anything. However, the context object has an option to tweak this behaviour, all we need to do is set context.callbackWaitsForEmptyEventLoop to false and Lambda will return as soon we execute callback, and persist the database connection to reuse, or disconnect after some minutes if no other request have been made. Knowing this information, just write the logic.

It just works! ;)