Get the most out of Docker support in Lambda and host your models without separate servers.

Photo: Markus Spiske on Unsplash
  1. Lambda function: A non-server resource that executes action code based on incoming events / requests in the Docker image.
  2. API gateway endpoint: Used as a trigger for Lambda and as a starting point for customer requests. When prediction requests are received at the endpoint, the Lambda function is triggered when the request body is included in the event sent to the function. The value returned by the function is returned to the client in response.
  • MLPredictionFunction Resource: This is the Lambda feature we want to enable. This section contains most of the required configuration:
  • Properties: here we specify that the function is configured using the Docker image (PackageType: Image) and that the function is started via the API gateway (Type: API). The path name and type of the API path are also specified here.
  • The metadata contains the tag used in the constructed images, as well as the location / name of the Docker file used to construct the image.
  • The outputs list all the necessary resources that SAM creates. In this case, the SAM needs the API gateway endpoint, the Lambda function, and the associated IAM role.
!sam build
!sam local invoke -e events/event.json
!aws ecr create-repository --repository-name ml-deploy-sam
!aws ecr get-login-password --region <region> | docker login --username AWS  --password-stdin <account id>.dkr.ecr.<region>.amazonaws.com
!sam deploy -g

Comments

  • Image updates: To deploy an updated template or action code, simply build the image locally and run the deployment command again. SAM detects which aspects of the application have changed and updates the relevant resources accordingly.
  • Cold start: Every time Lambda spins a container with our function code, the model is loaded before processing begins. This results in a cold start scenario where the first request is significantly slower than the following. One way to combat this is to launch the feature from time to time with CloudWatch so that the container is always ready while the template is loaded.
  • Multiple functions: It is possible to enable multiple Lambda functions that serve a single API. This can be useful if you have multiple models you want to serve, or if you want a separate preprocessing / confirmation terminal. You can configure this simply by including Advanced features in the SAM template resources.

LEAVE A REPLY

Please enter your comment!
Please enter your name here