Search
infrastructureblogia

Automated deployment of LibreChat on EC2 AWS

jls /
AWS Terraform SSM GenAI Mistral AI Anthropic OpenAI
Automated deployment of LibreChat on EC2 AWS

This article presents a project POC (Proof of Concept) for the automated deployment of LibreChat on AWS EC2, using Terraform to orchestrate the infrastructure following the principle of Infrastructure as Code, a Bash User-Data script to install components on EC2, and AWS Systems Manager for centralized management of API keys and deployment tracking. The emphasis is on automation and cost optimization through the use of Spot Instances.

Introduction

LibreChat is an advanced chatbot application that integrates multiple AI models, including Mistral AI, and offers features such as conversation search, creation of custom presets, message editing and continuation, as well as plugin integration. It provides a multilingual and multimodal user interface, multi-user management with secure authentication, and is fully open-source. This project explores its deployment on AWS EC2, using advanced tools for a fully automated implementation.

Architecture

The deployed architecture includes the following elements:

  1. An instance EC2 running Ubuntu Server.
  2. A User-Data bash script to automate the installation and configuration of the components required by LibreChat.
  3. Terraform to define and provision the AWS infrastructure required for the LibreChat deployment.
  4. AWS Systems Manager (SSM) to store and retrieve the API keys required by LibreChat and to track deployment progress.

Automation and Infrastructure as Code

Terraform

Terraform is a tool that allows you to define and provision infrastructure as code (Infrastructure as Code). In this project, Terraform is used to create and configure the EC2 instance, as well as the associated AWS resources, such as security groups and IAM roles.

User-Data

The User-Data Bash script is executed at the instance’s first boot. It automates the installation and configuration of the components required by LibreChat, such as Docker, Docker Compose, Git, Node.js and NPM. The User-Data script also configures the API keys required by LibreChat, such as OpenAI, MistralAI, Anthropic, Google API keys and Google CSE ID, by retrieving them from AWS Systems Manager (SSM).

A function update_status is defined in the User-Data script to update the deployment status via AWS SSM. This function allows monitoring the deployment state and quickly detecting any issues. The User-Data script also pushes the function update_registration.sh and sets it in cron to enable or disable registrations.

Example of the function update_status :

update_status() {
  STATUS_MESSAGE=$1
  aws ssm put-parameter --name "/librechat/deployment-status" --type "String" --value "$STATUS_MESSAGE" --overwrite --region $AWS_DEFAULT_REGION
}

Example of the function update_registration.sh :

#!/bin/bash
set -e

# Update registration status in SSM Parameter Store
aws ssm put-parameter --name "/librechat/registration_enabled" --type "String" --value "$1" --overwrite --region $AWS_DEFAULT_REGION

# Update LibreChat configuration file
if [ "$1" == "true" ]; then
  sed -i 's/enabled: false/enabled: true/g' /opt/librechat/config.yaml
else
  sed -i 's/enabled: true/enabled: false/g' /opt/librechat/config.yaml
fi

# Restart LibreChat service
systemctl restart librechat

The function update_registration.sh is used to update the activation status of registrations in the SSM Parameter Store and the LibreChat configuration file. The LibreChat service is then restarted to apply the changes.

Tracking deployment progress with SSM

AWS Systems Manager (SSM) is a service that enables centralized management and configuration of EC2 instances. In this project, SSM is used to store and retrieve the API keys required by LibreChat, as well as to track deployment progress.

A function check_deployment_status is also defined in the export.sh script to check the deployment status via AWS SSM. This function allows real-time monitoring of deployment progress and rapid detection of any issues.

Example of the function check_deployment_status :

check_deployment_status() {
  LAST_STATUS=""
  INSTANCE_ID=$(terraform output -raw instance_id)  # Récupère l'ID de l'instance Terraform.
  if [ -z "$INSTANCE_ID" ]; then
    echo "Aucune instance EC2 n'est actuellement déployée ou terraform output n'est pas configuré correctement."
    return 1
  fi
  IP_ADDRESS=$(aws ec2 describe-instances --instance-ids $INSTANCE_ID --query "Reservations[*].Instances[*].PublicIpAddress" --output text --region $AWS_DEFAULT_REGION 2>/dev/null)
  URL="https://$IP_ADDRESS/"
  echo "Vérification de l'état de déploiement..."
  ATTENTE_STATUS=true  # Utilisée pour contrôler l'affichage du message d'attente.
  while true; do
    STATUS=$(aws ssm get-parameter --name "/librechat/deployment-status" --query "Parameter.Value" --output text --region $AWS_DEFAULT_REGION 2>/dev/null)
    if [ $? -ne 0 ]; then
      if [ "$ATTENTE_STATUS" = true ]; then
        echo -ne "\rEn attente des informations de statut de déploiement.\n"
        ATTENTE_STATUS=false  # Empêche la répétition du message.
      fi
      sleep 1
      continue
    else
      ATTENTE_STATUS=true  # Réinitialise pour le prochain cycle.
    fi
    if [[ "$STATUS" != "$LAST_STATUS" ]]; then
      if [[ "$LAST_STATUS" != "" ]]; then
        echo -e " \e[32m✓\e[0m"  # Affiche une coche verte pour le statut précédent.
      fi
      echo -ne "\rÉtat actuel du déploiement : $STATUS"
      LAST_STATUS="$STATUS"
      if [[ "$STATUS" == "100% - Installation terminée" ]]; then
        echo -e "\n\e[32m✓ Installation terminée avec succès\e[0m"
        echo -e "Accédez à l'instance Librechat via : $URL"
        break
      elif [[ "$STATUS" == "Echec de l'installation" ]]; then
        echo -e "\n\e[31m✗ Échec de l'installation\e[0m"
        exit 1
      fi
    fi
    sleep 1
  done
}
}

The deployment status is stored in an SSM parameter, which allows checking the deployment state at any time and from anywhere.

Error handling with set -e and trap 'error_handler' ERR

In the User-Data script, robust error handling has been implemented using set -e and trap 'error_handler' ERR. This approach ensures that the script stops immediately on error and provides detailed information about the encountered issue.

Here is a snippet of the User-Data script with integrated error handling:

#!/bin/bash
set -e
trap 'error_handler' ERR

error_handler() {
  local error_message=$1
  echo "Error occurred: ${error_message}"
  update_status "ERROR: ${error_message}"
  exit 1
}

update_status() {
  STATUS_MESSAGE=$1
  aws ssm put-parameter --name "/librechat/deployment-status" --type "String" --value "$STATUS_MESSAGE" --overwrite --region $AWS_DEFAULT_REGION
}

The function error_handler is called whenever an error occurs in the script. It takes an error message as a parameter, prints it to the console, updates the deployment status via AWS SSM using the function update_status, and exits the script with an error code.

Thanks to set -e and trap 'error_handler' ERR, the deployment stops as soon as an error occurs, which simplifies debugging and issue resolution. Additionally, updating the deployment status in AWS SSM enables tracking deployment progress and quickly detecting potential problems.

Cost reduction with Spot Instances

Spot Instances are EC2 instances that allow using unused capacity at reduced prices compared to on-demand instances. In this project, Spot Instances are used to reduce hosting costs for the application. The User-Data script supports configuring Spot Instances, enabling significant cost savings without compromising application performance.

Common use of export.sh in the automated deployment of LibreChat on AWS EC2

As part of the automated deployment of LibreChat on AWS EC2, a shell script named export.sh was created to simplify the management of various tasks related to deployment and infrastructure configuration. This script is used both from the local workstation and in GitLab CI pipelines, enabling a common and consistent use of the functions it contains.

The script export.sh groups several useful functions for deploying and managing AWS infrastructure. Among them are:

  • terraform_plan : generates a Terraform plan to preview infrastructure changes.
  • terraform_apply : applies Terraform changes to the AWS infrastructure.
  • terraform_destroy : destroys the Terraform resources created during deployment.
  • check_deployment_status : checks the current deployment status by querying AWS SSM.

Here is an example of using these functions in a GitLab CI pipeline:

stages:
  - Vérifications
  - Déploiements
  - Suppressions

Vérification Terraform:
  stage: Vérifications
  script:
    - /bin/bash -c "source export.sh && terraform_plan"

Déploiement Terraform:
  stage: Déploiements
  script:
    - /bin/bash -c "source export.sh && terraform_apply && check_deployment_status"

Suppression Terraform:
  stage: Suppressions
  script:
    - /bin/bash -c "source export.sh && terraform_destroy"

In this example, the functions terraform_plan, terraform_apply, and terraform_destroy are used in the different stages of the GitLab CI pipeline. By sourcing the script export.sh, the functions it contains become available in the pipeline execution environment.

The shared use of export.sh between the local workstation and GitLab CI helps simplify management of the automated LibreChat deployment on AWS EC2. The functions in this script streamline tasks and ensure consistency in operations performed on the infrastructure.

For more information about the project and to view the source code, visit the GitLab repository. You will find detailed information about the architecture, configuration, and best practices implemented in this automated deployment project.

Default connection with SSM on the instance

AWS Systems Manager (SSM) allows connecting to the EC2 instance by default without using SSH. This feature simplifies instance access and enhances security by avoiding exposing the SSH port to the Internet. However, it is still possible to access the instance via SSH by opening the appropriate flow using the associated variable.

Self-signed SSL certificate and security

By default, LibreChat leaves port 80 accessible without enabling port 443. In this project, port 443 is enabled by default with a self-signed certificate, and port 80 redirects to port 443. Although an HTTPS warning appears in the browser, using HTTPS provides protection against password theft on the network.

Conclusion

This project explores how to deploy and configure LibreChat on an AWS EC2 instance using Terraform for infrastructure, a Bash User-Data script for component installation, and AWS Systems Manager for centralized configuration management and deployment tracking. The emphasis is also placed on cost reduction with Spot Instances and security by using a self-signed SSL certificate and configuring HTTP security headers.

By using this project, you will be able to deploy and configure LibreChat on an AWS EC2 instance efficiently, securely, and cost-effectively. This project can be extended and adapted to deploy other web applications on AWS using the same principles of automation, Infrastructure as Code, and centralized configuration management.

Continue exploring the GitLab project

For an in-depth understanding and technical details of the LibreChat deployment on AWS EC2, including architecture and configurations, I strongly encourage you to consult the project README on GitLab. This article introduces the project, its key concepts, and the benefits of this deployment, but the project link contains all the details.

This document was translated from the French version into the English language using the gpt-5-mini model. For more information on the translation process, see https://gitlab.com/jls42/ai-powered-markdown-translator