Implementing a serverless CI/CD using CodePipeline and CodeBuild

While we continued working on our Expenses Tracker serverless application, the "Sauf Pompiers" SARL decided it wants more engineers working on it, even though there (still!) arent too many engineers that know serverless applications very well. But, no problem, you will be their Lead and Mentor engineer.

Additionally, the "Sauf Pompiers" wants a streamlined development process, and if we take into account the bigger team coming to the project, it has become an imperative to setup a serverless CI/CD. We can't have a dozen engineers all deploying at the same time without any review process.

But where do we start with a serverless CI/CD?

Luckily, as we are using AWS, you are going to be happy to know it has quite a good continuous delivery system called AWS CodePipeline. With complete SAM support. It's main benefit being that authentication and tooling are easy to set up.

AWS CodePipeline, enables you to define a sequence of tasks that start from the source code and end up with a new version of the production system. Usually, the tasks will do something to the source code, such as run tests, produce application binaries or package artefacts, and each task can save the results so that another task can use them as input. AWS CodePipeline automatically moves artefacts between successful tasks. You can optionally set up manual approval steps requiring humans to verify release candidates. AWS CodePipeline runs build agents using AWS CodeBuild.

AWS CodeBuild is a service, almost identical to Jenkins, able to spin up Docker containers, where you can install development and testing tools, and then execute a defined sequence of system commands. You can either use one of the standard AWS containers, or pre-package your own. Standard containers usually come the AWS SDK installed, and with it also Python.

Note If you are using JavaScript for developing Lambda functions, it’s safe to choose the provided Node.js container, and it will still have Python and the AWS toolkit.

So the whole lifecycle is:

1. AWS CodePipeline automatically triggers on new commits into version control (GitHub, AWS CodeCommit)
2. AWS CodeBuild follows a buildspec.yml file that defined how your application should be built. Common defaults for most languages & dependency repositories. You are able to even have Custom Docker images.
3. CloudFormation does a transform of your SAM resources to known resources, executes a ChangeSet.
4. Run tests
5. Gate Keeper asks for Manual Approval (optional)

Task

Your task now is to setup the whole step-by-step guide from above.

1. Setup AWS CodePipeline and AWS CodeBuild. CodePipeline is to trigger on a version control update (its highly recommended to use GitHub).
2. Create a CodeBuild buildspec.yml file that will do a build of your codebase.
3. Setup the CloudFormation pipeline.yml stack for deployment.
4. Add a Manual approver step.
5. Run and enable it.

Hints

Here are a few hints to help you with this task, as this is a big one:

1. If you haven't done this before, version this project as a GitHub or AWS CodeCommit repository.

2. Create an full fledge CodePipeline + CodeBuild solution by utilizing the AWS CodePipeline CookieCutter solution and generate it. Note: Be sure to do it inside your project folder. The reason why we are doing it using this, is because it's already a best practice setup provided by AWS.

3. Follow up on the generated Pipeline Instructions.md. Get the GitHub Access token, and setup your SSM parameters for the GitHub repository (repo, token, user)

4. Change the CodeBuild setup in the buildspec.yml per your language setup (again as per the Pipeline Instructions.md) - See which images are available here

5. Follow up on the instructions and run the AWS CodePipeline by doing a test commit and push into your new GitHub repo.

Note: If you have any issues or want to remove it, you can always do a CloudFormation delete stack operation: aws cloudformation delete-stack --stack-name YOUR_STACK