Building and deploying AWS Lambda (.NET core 3.1) Web APIs using Gitlab CI/CD

GitLab includes utilities that allows us to create pipelines that can be used for CI (Continuous Integration) and CD (Continuous Delivery)

Today we will be building a simple .Net API and automatically deploying it as soon as any commit is pushed on to our source control (git)

First we will start by creating our function

Next up, I have created a sample ASP.net project which includes Web API functionalities and I have included the Lamda entry point. You can clone it here.

The project basically exposes “weatherforcast” endpoint which returns an array of weather forecasts

[
    {
        "date": "2019-07-16T19:04:05.7257911-06:00",
        "temperatureC": 52,
        "temperatureF": 125,
        "summary": "Mild"
    },
    {
        "date": "2019-07-17T19:04:05.7258461-06:00",
        "temperatureC": 36,
        "temperatureF": 96,
        "summary": "Warm"
    },
    {
        "date": "2019-07-18T19:04:05.7258467-06:00",
        "temperatureC": 39,
        "temperatureF": 102,
        "summary": "Cool"
    },
    {
        "date": "2019-07-19T19:04:05.7258471-06:00",
        "temperatureC": 10,
        "temperatureF": 49,
        "summary": "Bracing"
    },
    {
        "date": "2019-07-20T19:04:05.7258474-06:00",
        "temperatureC": -1,
        "temperatureF": 31,
        "summary": "Chilly"
    }
]

This is automatically generated using Visual Studio 2019 Web API project.

Next we will need to understand how manual deployment of .NET core projects work. To deploy lambda .Net projects, we will be using the AWS lambda .Net tools. To install the lamba tools, using the command line run the following

dotnet tool install -g Amazon.Lambda.Tools

Next, we just run the following command in our project directory

dotnet lambda package

As seen in the screenshot above, it created a zip file which we can upload to our lambda through the lambda dashboard.

To enable us to test our lambda deployment, we will need to create an API gateway to route the traffic to the lambda.I have created an API gateway with the following resources and configuration

Then i have deployed the API and tested it

Now it is time to automate it using GitLab CI/CD

First we will need to create an AWS user to deploy the project with. So we will go to the IAM section in the AWS dashboard and then create a user. Make sure to enable “Programmatic access” to get the key and secret.

Then click next, for the permissions. I’m going to give it full permission over Lambda but i would recommend tightening up the security and giving it only permissions to update the lambda.

Then create the user. After creating the user you should get this screen, copy the Access key ID and Secret access key and save them somewhere as we will need them later.

Next, you will need to have a repository set up on GitLab. you can fork the one i linked above. create a yaml file called “.gitlab-ci.yml” in the root directory of the repository.

First we will start by shaping the skeleton of our deployment process

stages:
  - build
  - deploy
build_job:
  stage: build
  script:
    - echo "build"

deploy_job:
  stage: deploy
  script:
    - echo "deploy"

This defines two main stages in our pipeline. The build stage and the deployment stage. In the build stage we are going to build our .Net project and in the deployment stage we are going to update our lambda code. Currently the script only prints “build” in the building stage and “deploy” in the deployment stage.

Next lets work on the build stage. As mentioned before, building and packaging the project is done using one command. so let’s add that command to our pipeline.

build_job:
  stage: build
  script:
    - dotnet lambda package

However, that will not work so easily as this will run on a linux machine with no .Net SDK or any dependencies which will throw an exception as it does not know that dotnet is. So lets fix this by telling the pipeline to use a docker container to run our build process. For the container we will use a container image that i have built that has the following:

1. .Net SDK 3.1
2. .Net Lambda Tools
3. Zip utility

So to add this to our pipeline we specify a key called image.

build_job:
  stage: build
  image: ahmedghanima/dotnet-lambda
  script:
    - export PATH="$PATH:/root/.dotnet/tools" # add lambda tools to path
    - dotnet lambda package

Also notice that i have added the export command as the dotnet lamda tools are not added to the linux path by default, this is done to be able to call it directly by just writing dotnet lambda

Next up, we will let gitlab store our build zip output so we can download it and use it in our deployment stages ( so that it can be shared across stages ). This is done by specifying artifacts.

build_job:
  stage: build
  image: ahmedghanima/dotnet-lambda
  script:
    - export PATH="$PATH:/root/.dotnet/tools" # add lambda tools to path
    - dotnet lambda package
  artifacts:
    expire_in: 1 week  
    paths:
      - '$CI_PROJECT_DIR/bin/Release/netcoreapp3.1/dotnet-deploy-example.zip'  # saving output package

This means that the output of our build process “dotnet-deploy-example.zip” can now be downloaded through the Gitlab interface and can be shared across stages.

Now the build process should be finished, lets move on to the deployment.

For the deployment we will do the same as well by using containers to run it so that we do not have to install the aws cli ourselves. We will be using an image provided by amazon.

amazon/lambda-build-node10.x

This image has the aws cli as of the time of this post, aws does not have an official aws cli image. To deploy and update our lambda code we can use the update-function-code command in the aws cli.

aws lambda update-function-code --function-name dotNetCore-Example --zip-file fileb://$CI_PROJECT_DIR/bin/Release/netcoreapp3.1/dotnet-deploy-example.zip

However, currently the deployment job does not have the zip file. so we need to tell it to download the artifacts from the build process. we can do that by specifying that the deployment process depends on the build process.

deploy_job:
  stage: deploy
  image: amazon/lambda-build-node10.x
  script:
    - aws lambda update-function-code --function-name  dotNetCore-Example --zip-file fileb://$OUTPUT_FILE_PATH
  dependencies:
    - build_job

Finally, we need to tell the aws cli to use our provided credentials that we created above to be able to update the function code otherwise we will receive an unauthorized error.

So lets go back to the Gitlab UI and go to our repository.Then lets head to Settings → CI/CD. Gitlab provides environment variables that can be passed to our pipeline. so we will use this feature to pass the aws api key and secret to our pipeline.

Now we will create 3 variables:

• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY
• AWS_DEFAULT_REGION

then add the correct value to each variable. In my case the default region is Frankfurt so that’s eu-central-1.

After adding the variables the aws cli container should automatically pick them up. So finally the gitlab-ci file should look like this

variables:
  OUTPUT_FILE_PATH: '$CI_PROJECT_DIR/bin/Release/netcoreapp3.1/dotnet-deploy-example.zip'
stages:
  - build
  - deploy
build_job:
  stage: build
  image: ahmedghanima/dotnet-lambda
  script:
    - export PATH="$PATH:/root/.dotnet/tools" # add lambda tools to path
    - dotnet lambda package
  artifacts:
    expire_in: 1 week  
    paths:
      - $OUTPUT_FILE_PATH  # saving output package

deploy_job:
  stage: deploy
  image: amazon/lambda-build-node10.x
  script:
    - aws lambda update-function-code --function-name  dotNetCore-Example --zip-file fileb://$OUTPUT_FILE_PATH
  dependencies:
    - build_job

Now any commit that will be pushed to our repository would fire the pipeline causing it to automatically build and deploy our project.

Resources

Gitlab CI/CD Documentation

Gitlab-ci file full references

Feel free to check out my other articles and my blog

If you found this article any useful, you know what to do now. Hit that clap button and follow me to get more articles on your feed.