Automated Deployment of LibreChat on AWS EC2

This article presents a POC (Proof of Concept) project for the automated deployment of LibreChat on AWS EC2, using Terraform to orchestrate the infrastructure according to the Infrastructure as Code principle, a Bash User-Data script to install components on EC2, and AWS Systems Manager for centralized API key management and deployment tracking. The focus is on automation and cost optimization through the use of Spot instances.

Introduction

LibreChat is an advanced chatbot application that integrates multiple AI models, including Mistral AI, and offers features such as searching in conversations, creating custom presets, editing and continuing messages, as well as plugin integration. It provides a multilingual and multimodal user interface, multi-user management with secure authentication, and is fully open-source. This project explores its deployment on AWS EC2, using advanced tools for a fully automated implementation.

Architecture

The deployed architecture includes the following elements:

  1. An EC2 instance running Ubuntu Server.
  2. A User-Data bash script to automate the installation and configuration of components necessary for LibreChat.
  3. Terraform to define and provision the AWS infrastructure needed for deploying LibreChat.
  4. AWS Systems Manager (SSM) for storing and retrieving API keys necessary for LibreChat and for tracking the progress of the deployment.

Automation and Infrastructure as Code

Terraform

Terraform is a tool that allows defining and provisioning infrastructure as code (Infrastructure as Code). In this project, Terraform is used to create and configure the EC2 instance, as well as associated AWS resources, such as security groups and IAM roles.

User-Data

The User-Data bash script is executed upon the EC2 instance’s first start-up. It automates the installation and configuration of components necessary for LibreChat, such as Docker, Docker Compose, Git, Node.js, and NPM. The User-Data script also configures the API keys necessary for LibreChat, such as OpenAI, MistralAI, Anthropic, Google API, and Google CSE ID keys, by retrieving them from AWS Systems Manager (SSM).

An update_status function is defined in the User-Data script to update the deployment status via AWS SSM. This function allows monitoring the state of the deployment and quickly detecting any potential issues. The User-Data script also pushes the update_registration.sh function and sets it as a cron job to activate or deactivate registrations.

Example of the update_status function:

update_status() {
  STATUS_MESSAGE=$1
  aws ssm put-parameter --name "/librechat/deployment-status" --type "String" --value "$STATUS_MESSAGE" --overwrite --region $AWS_DEFAULT_REGION
}

Example of the update_registration.sh function:

#!/bin/bash
set -e

# Update registration status in SSM Parameter Store
aws ssm put-parameter --name "/librechat/registration_enabled" --type "String" --value "$1" --overwrite --region $AWS_DEFAULT_REGION

# Update LibreChat configuration file
if [ "$1" == "true" ]; then
  sed -i 's/enabled: false/enabled: true/g' /opt/librechat/config.yaml
else
  sed -i 's/enabled: true/enabled: false/g' /opt/librechat/config.yaml
fi

# Restart LibreChat service
systemctl restart librechat

The update_registration.sh function is used to update the registration activation status in the SSM Parameter Store and LibreChat’s configuration file. The LibreChat service is then restarted to apply the changes.

Deployment progress tracking with SSM

AWS Systems Manager (SSM) is a service that allows managing and configuring EC2 instances centrally. In this project, SSM is used to store and retrieve the API keys necessary for LibreChat, as well as to track the deployment progress.

A check_deployment_status function is also defined in the export.sh script to check the deployment status via AWS SSM. This function allows to follow in real time the progress of the deployment and to quickly detect potential problems.

Example of the check_deployment_status function:

check_deployment_status() {
  LAST_STATUS=""
  INSTANCE_ID=$(terraform output -raw instance_id)  # Récupère l'ID de l'instance Terraform.
  if [ -z "$INSTANCE_ID" ]; then
    echo "Aucune instance EC2 n'est actuellement déployée ou terraform output n'est pas configuré correctement."
    return 1
  fi
  IP_ADDRESS=$(aws ec2 describe-instances --instance-ids $INSTANCE_ID --query "Reservations[*].Instances[*].PublicIpAddress" --output text --region $AWS_DEFAULT_REGION 2>/dev/null)
  URL="https://$IP_ADDRESS/"
  echo "Vérification de l'état de déploiement..."
  ATTENTE_STATUS=true  # Utilisée pour contrôler l'affichage du message d'attente.
  while true; do
    STATUS=$(aws ssm get-parameter --name "/librechat/deployment-status" --query "Parameter.Value" --output text --region $AWS_DEFAULT_REGION 2>/dev/null)
    if [ $? -ne 0 ]; then
      if [ "$ATTENTE_STATUS" = true ]; then
        echo -ne "\rEn attente des informations de statut de déploiement.\n"
        ATTENTE_STATUS=false  # Empêche la répétition du message.
      fi
      sleep 1
      continue
    else
      ATTENTE_STATUS=true  # Réinitialise pour le prochain cycle.
    fi
    if [[ "$STATUS" != "$LAST_STATUS" ]]; then
      if [[ "$LAST_STATUS" != "" ]]; then
        echo -e " \e[32m✓\e[0m"  # Affiche une coche verte pour le statut précédent.
      fi
      echo -ne "\rÉtat actuel du déploiement : $STATUS"
      LAST_STATUS="$STATUS"
      if [[ "$STATUS" == "100% - Installation terminée" ]]; then
        echo -e "\n\e[32m✓ Installation terminée avec succès\e[0m"
        echo -e "Accédez à l'instance Librechat via : $URL"
        break
      elif [[ "$STATUS" == "Echec de l'installation" ]]; then
        echo -e "\n\e[31m✗ Échec de l'installation\e[0m"
        exit 1
      fi
    fi
    sleep 1
  done
}
}

The deployment status is stored in an SSM parameter, which allows you to check the deployment status at any time and from anywhere.

Error management with set -e and trap 'error_handler' ERR

In the User-Data script, a robust error handling has been set up using set -e and trap 'error_handler' ERR. This approach ensures that the script stops immediately in case of error and provides detailed information about the problem encountered.

Here is a piece of the User-Data script with integrated error handling:

#!/bin/bash
set -e
trap 'error_handler' ERR

error_handler() {
  local error_message=$1
  echo "Error occurred: ${error_message}"
  update_status "ERROR: ${error_message}"
  exit 1
}

update_status() {
  STATUS_MESSAGE=$1
  aws ssm put-parameter --name "/librechat/deployment-status" --type "String" --value "$STATUS_MESSAGE" --overwrite --region $AWS_DEFAULT_REGION
}

The error_handler function is called every time an error occurs in the script. It takes an error message as a parameter, displays it in the console, updates the deployment status via AWS SSM using the update_status function, and exits the script with an error code.

Thanks to set -e and trap 'error_handler' ERR, the deployment stops as soon as an error occurs, which facilitates debugging and problem resolution. Moreover, updating the deployment status in AWS SSM allows to follow the progress of the deployment and to quickly detect potential problems.

Cost reduction with Spot Instances

Spot Instances are EC2 instances that allow to use unused capacities at reduced prices compared to on-demand instances. In this project, Spot Instances are used to reduce the hosting costs of the application. The User-Data script supports the configuration of Spot Instances, which allows for significant cost savings without compromising the performance of the application.

Common use of export.sh in the automated deployment of LibreChat on AWS EC2

As part of the automated deployment of LibreChat on AWS EC2, a shell script named export.sh was created to facilitate the management of the various tasks related to the deployment and configuration of the infrastructure. This script is used from both the local workstation and within the GitLab CI pipelines, allowing for a common and consistent use of the functions it contains.

The export.sh script includes several useful functions for the deployment and management of the AWS infrastructure. Among these, we find:

  • terraform_plan: generates a Terraform plan to preview the modifications to be made to the infrastructure.
  • terraform_apply: applies the Terraform modifications on the AWS infrastructure.
  • terraform_destroy: deletes the Terraform resources created during the deployment.
  • check_deployment_status: checks the ongoing deployment status by querying AWS SSM.

Here is an example of using these functions in a GitLab CI pipeline:

stages:
  - Vérifications
  - Déploiements
  - Suppressions

Vérification Terraform:
  stage: Vérifications
  script:
    - /bin/bash -c "source export.sh && terraform_plan"

Déploiement Terraform:
  stage: Déploiements
  script:
    - /bin/bash -c "source export.sh && terraform_apply && check_deployment_status"

Suppression Terraform:
  stage: Suppressions
  script:
    - /bin/bash -c "source export.sh && terraform_destroy"

In this example, the functions terraform_plan, terraform_apply, and terraform_destroy are used in the different steps of the GitLab CI pipeline. By sourcing the export.sh script, the functions it contains become accessible in the pipeline’s execution environment.

The common use of export.sh between the local workstation and GitLab CI facilitates the management of the automated deployment of LibreChat on AWS EC2. The functions contained in this script simplify tasks and ensure consistency in the operations carried out on the infrastructure.

To learn more about the project and check the source code, visit the GitLab repository. You will find detailed information on the architecture, configuration, and best practices implemented in this automated deployment project.

Default Connection with SSM on the Instance

AWS Systems Manager (SSM) allows for default connection to the EC2 instance without having to use SSH. This feature simplifies access to the instance and enhances security by avoiding exposing the SSH port to the Internet. However, it is still possible to access the instance via SSH by opening the appropriate flow using the associated variable.

Self-Signed SSL Certificate and Security

By default, LibreChat makes port 80 accessible without enabling port 443. In this project, port 443 is enabled by default with a self-signed certificate, and port 80 redirects to port 443. Although an HTTPS warning is displayed in the browser, using the HTTPS protocol offers security against password theft on the network.

Conclusion

This project explores how to deploy and configure LibreChat on an AWS EC2 instance using Terraform for infrastructure, a User-Data bash script for component installation, and AWS Systems Manager for centralized configuration management and deployment progress tracking. The focus is also on cost reduction with Spot Instances and security by using a self-signed SSL certificate and configuring HTTP security headers.

By using this project, you will be able to deploy and configure LibreChat on an AWS EC2 instance in an efficient, secure, and cost-effective manner. This project can be extended and adapted to deploy other web applications on AWS using the same principles of automation, Infrastructure as Code, and centralized configuration management.

Continue Exploration with the GitLab Project

For an in-depth understanding and the technical details of deploying LibreChat on AWS EC2, including the architecture and configurations, I strongly encourage you to consult the project’s README on GitLab. This article introduces the project, its key concepts, and the benefits of this deployment, but it is in the project link that you will find all the details.

This document has been translated from the fr version to the en language using the gpt-4-1106-preview model. For more information about the translation process, visit https://gitlab.com/jls42/ai-powered-markdown-translator