How to Integrate LLM Mux into Visual Studio Code: Supercharge Your Coding with a Custom AI Assistant

Do you want an AI assistant right inside Visual Studio Code (VSC) to explain code, debug, or write unit tests, but also want the freedom to choose between GPT-4, Claude, or a Local LLM? This article will guide you on how to combine LLM Mux with VS Code to create the most powerful programming environment.

Prerequisites

1. Visual Studio Code installed.
2. Python (to run LLM Mux).
3. LLM Mux Source code: https://github.com/nghyane/llm-mux

Step 1: Install and Run LLM Mux

First, we need to set up the internal “AI server”.

1. Clone the project to your machine:Bashgit clone https://github.com/nghyane/llm-mux cd llm-mux
2. Install the necessary libraries (usually via requirements.txt):Bashpip install -r requirements.txt
3. Configure your API Keys in the .env or config file of the project (enter your OpenAI/Anthropic keys here).
4. Start the Server:Bashpython main.py # Assuming server runs at: http://localhost:8000

Step 2: Install the “CodeGPT” Extension in VS Code

By default, VS Code doesn’t support connecting directly to a Custom URL, so we need an intermediary Extension. CodeGPT is a great choice.

1. Open VS Code -> Select the Extensions tab (the square icon on the left).
2. Search for: CodeGPT.
3. Click Install.

Step 3: Configure VS Code Connection to LLM Mux

This is the most crucial step to “trick” the Extension into pointing to your LLM Mux.

1. Press Ctrl + Shift + P (or Cmd + Shift + P on Mac).
2. Type and select: CodeGPT: Set API KEY.
   • Enter any random character (e.g., sk-123456). Since LLM Mux manages the real keys, you can enter anything here, as long as it’s not empty.
3. Go to CodeGPT Settings (File -> Preferences -> Settings -> Type CodeGPT).
4. Find Provider: Select OpenAI (or OpenAI Compatible).
5. Find Base URL:
   • Change the default URL to your Local URL.
   • Example: http://localhost:8000/v1 (Note: add /v1 if the project requires it).

Step 4: Experience It

Now everything is ready!

• Open any code file.
• Highlight a code snippet, right-click, and select CodeGPT: Explain this Code.
• Or open the chat window on the left to command: “Write a Python function to connect to SQL”.

At this point, the request will go from VS Code -> LLM Mux -> Actual AI Model, helping you code faster, more securely, and with greater flexibility.