Skip to content

Changing the LLMs

The system's backend is built to be allows you to change the LLMs used for each task.

The models component of the system is built to be extremely flexible. For example, to change the used model no code changes are required. Moreover, the model component allows the user to select different models on each request.

This guide will tell you what models the system currently support and how they can be changed.

Supported Models

The system currently supports the following models:

  • Gemini: Any gemini model that is available through the Gemini API is supported by the system. This includes models like gemini-2.0-flash-lite, gemini-2.5-flash, and others.
  • Azure OpenAI: Any model that is available through Azure OpenAI is supported. This includes models like gpt-5, gpt-4o, gpt-4.1-mini, and others.
  • Ollama: Any model that is available through the Ollama API is supported. This includes models like llama3, llama2, and others.
  • OpenAI-compatible services: Any model that is available through an OpenAI-compatible service is supported.

The last one is particularly useful as it allows you to use any model that is compatible with the OpenAI API. Nearly all major LLM providers have an OpenAI-compatible API, so you can use models from them. In other words, if the model you want isn't one of the top three options, it will most likely be available through an OpenAI-compatible service.

Changing the Models

The main method to change the models used by the system is through environment variables. The backend reads the model names from environment variables, so you can change the models by updating these variables.

There are 4 environment variables that control the models used by the system:

  • SUMMARY_MODEL
  • PRESENTATION_OUTLINE_MODEL
  • SLIDES_CONTENT_GENERATION_MODEL
  • EMBEDDINGS_MODEL

The value for these keys should be made in the format MODEL_FAMILY/MODEL_NAME.

The MODEL_FAMILY can be one of the following:

  • gemini
  • azure-openai
  • ollama
  • openai

It is your responsibility to ensure that the model you are trying to use is available through the selected model family.

Updating Secrets and Keys

Each model family requires different secrets and keys to be used.

Gemini

Two environment variables are used to configure Gemini:

  • GEMINI_API_KEY: The API key for Gemini. This is currently not used as the system fetches the Gemini API key from the user. However, you can update the code to use this variable if you want to set a default API key for all users.
  • GEMINI_OPENAI_URL: The URL for the Gemini OpenAI API. Defaults to https://generativelanguage.googleapis.com/v1beta/openai/.

If you don't want users to provide their own Gemini API key, you can set the GEMINI_API_KEY environment variable and update fastapi.py to use this variable instead of fetching the key from the user. Note that other stylistic changes may be needed in the web app to reflect this change.

Azure OpenAI

Three environment variables are used to configure Azure OpenAI:

  • AZURE_OPENAI_ENDPOINT: The endpoint for Azure OpenAI.
  • AZURE_OPENAI_API_KEY: The API key for Azure OpenAI.
  • AZURE_OPENAI_API_VERSION: The API version for Azure OpenAI.

If you use an Azure OpenAI model (the MODEL_FAMILY is azure-openai), you must set these variables. Failure to do so will result in errors when trying to use the model.

Ollama

One environment variable is used to configure Ollama:

  • OLLAMA_URL: The URL for the Ollama API. Defaults to http://localhost:11434/v1.

On most scenarios, you won't need to change this variable unless you are running Ollama on a different host or port.

OpenAI-compatible services

Two environment variables are used to configure OpenAI-compatible services:

  • GENERIC_OPENAI_BASE_URL: The base URL for the OpenAI-compatible service.
  • GENERIC_OPENAI_API_KEY: The API key for the OpenAI-compatible service.

It is your responsibility to ensure that the service you are trying to use is compatible with the OpenAI API and supports the model you are trying to use.