Using LM Studio With Kilo Code

Kilo Code supports running models locally using LM Studio. LM Studio provides a user-friendly interface for downloading, configuring, and running local language models. It also includes a built-in local inference server that emulates the OpenAI API, making it easy to integrate with Kilo Code.

Website: https://lmstudio.ai/

Setting Up LM Studio

  1. Download and Install LM Studio: Download LM Studio from the LM Studio website.

  2. Download a Model: Use the LM Studio interface to search for and download a model. Some recommended models include:

    • CodeLlama models (e.g., codellama:7b-code, codellama:13b-code, codellama:34b-code)
    • Mistral models (e.g., mistralai/Mistral-7B-Instruct-v0.1)
    • DeepSeek Coder models (e.g., deepseek-coder:6.7b-base)
    • Any other model that is supported by Kilo Code, or for which you can set the context window.

    Look for models in the GGUF format. LM Studio provides a search interface to find and download models.

  3. Start the Local Server:

    • Open LM Studio.
    • Click the "Local Server" tab (the icon looks like <->).
    • Select the model you downloaded.
    • Click "Start Server".

Configuration in Kilo Code

Open Settings (gear icon) and go to the Providers tab to add LM Studio. No API key is needed since LM Studio runs locally. You can configure the base URL if LM Studio is running on a different host or port.

The extension stores this in your kilo.json config file. You can also edit the config file directly — see the CLI tab for the file format.

Using Custom or Unlisted Models

If the model you loaded in LM Studio doesn't appear in the Kilo model picker, you can register it as a custom model in your config file:

{
  "model": "lmstudio/my-custom-model",
  "provider": {
    "lmstudio": {
      "models": {
        "my-custom-model": {
          "name": "My Custom Model",
        },
      },
    },
  },
}

The model key (my-custom-model) must match the model identifier that LM Studio serves. If the display name you want differs from the API identifier, use the id field to set the API-facing name separately:

{
  "provider": {
    "lmstudio": {
      "models": {
        "my-llama": {
          "id": "meta-llama-3.1-8b-instruct",
          "name": "Llama 3.1 8B (Local)",
        },
      },
    },
  },
}

See Custom Models for the full list of configuration fields and more examples.

Tips and Notes

  • Resource Requirements: Running large language models locally can be resource-intensive. Make sure your computer meets the minimum requirements for the model you choose.
  • Model Selection: LM Studio provides a wide range of models. Experiment to find the one that best suits your needs.
  • Local Server: The LM Studio local server must be running for Kilo Code to connect to it.
  • LM Studio Documentation: Refer to the LM Studio documentation for more information.
  • Troubleshooting: If you see a "Please check the LM Studio developer logs to debug what went wrong" error, you may need to adjust the context length settings in LM Studio.