Performance optimization: latency and cold starts

Serverless applications are not slow. However, from the early days of serverless, one of the most dreadful terms in serverless is "cold start."

There are still servers in serverless, but they are hidden in layers of abstractions and fully managed by cloud vendors. When we write code, all we care about are functions. In serverless, functions run in some kind of containers (or micro VMs in case of AWS Lambda). The first time our function is triggered in a while, a cloud vendor needs to start the container, and execution can take a bit longer. That's a cold start. Subsequent invocations will hit the same container, and customers will get their results faster. We call these warm starts.

How slow is a cold start?

The only correct answer is: it depends. As you can see, our app is not slow. The function execution is a couple of hundreds of milliseconds slower if we hit a cold start. However, it can be slower. For example, some runtimes have slower cold start than the others (Node.js and Python are faster than Java). It also depends on the size of our code. Smaller code bundles will start faster, and we want to keep the number of dependencies as low as we can. Finally, running a function in a virtual private cloud (VPC) increases the cold start time. It's not as slow as it were (10-15s), but it still takes a second or more to run a cold function.

Most of the time, our customers will not even notice cold starts. However, in some cases, speed is mission-critical, and we want to speed up our functions as much as we can.

Some hacks help with reducing cold starts. For example, we can set up a cron job that will keep our functions warm, but that approach has many downsides. Fortunately, AWS introduced Provisioned Concurrency for AWS Lambda, a feature that keeps functions initialized and hyper-ready to respond in double-digit milliseconds.

Task

Your task is to add a Provisioned Concurrency to the Lambda function we use for saving receipts (one we created the first day). This will allow our customers to save expenses faster.

Note: Use Provisioned Concurrency for one function and one provisioned instance only, because of its cost. Provisioning one function for one day should not add more than a few cents to your monthly bill, but if we provision multiple instances cost will be higher.

As a bonus, try to find a way to cache HTTP connection for DynamoDB table, so we can additionally improve app performance.

Once you complete this exercise, take a minute and discuss this solution with your team. Is the app faster after we added provisioned concurrency?

Hints

Here are a few hints to help you with this task:

• AWS SAM supports Provisioned Concurrency, see the AWS::Serverless::Function resource docs for more details.
• Provisioned Concurrency requires an alias, we can't set it for the $LATEST version of our function.
• Provisioned Concurrency and Reserved Capacity are not the same. Make sure you read about main differences.
• By default, the default Node.js HTTP/HTTPS agent creates a new TCP connection for every new request. To avoid the cost of establishing a new connection, you can reuse an existing connection. For more info see this guide.