Rust Serverless Transformer

Overview

In this project we will deploy a simple rust fuction which uses an LLM from huggingface to continue the text given to it. The function will be deployed as a lambda function on AWS.

Prerequisites

Make sure you have the following installed:

• Docker
• Rust
• Cargo Lambda
• AWS CLI

App Development

1. Create a new rust project using cargo lambda

cargo lambda new <project-name>

2. Use the desired model from huggingface, in this case, we used a rustformers/pythia-ggml model. don't forget to add the correct path to the model in the file

3. Build the rust code as required, in this case we take in the query from the user and complete it using the model. In case there is no query, we use a default query to start the completion.

4. Test the code locally using the following command

cargo lambda watch

5. We can use services like postman to send and receive the requests from the lambda function.

6. Once the functionality is working as expected, make the production build

App deployment

Docker Image Creation

1. Do docker init in the project directory

2. modify the Dockerfile as required

3. Build the docker image
   

4. verify that the image is created either using the docker images command or viewing in the docker desktop

AWS ECR

We will use AWS ECR service to store our docker image so that we can use it to build the lambda function.

1. Create a new private repository in ECR
2. once the repository is created, click on the "view push commands" button to get the commands to push the image to the repository
3. Run the commands in the terminal to push the image to the repository
4. Verify that the image is pushed to the repository by checking the repository in the AWS console
5. copy the URI of the image to use it in the lambda function

AWS Lambda Function

1. Go to AWS lambda and create a new function
2. select the container image as the source
3. paste the URI of the image we copied earlier in the repository URI
4. select the other settings as required
5. Create the function
6. Once the function is created, goto configurations and enable function URL
7. The lambda function is now ready to be used

Sample Output

With no query, uses the default starter:

With a query:

GitHub Actions

Github Actions is used to automatically perform functions like Linting, formatting etc. whenever there is an update in the repository.