Use LM Studio Local Inference Server with BrainSoup

Introduction

BrainSoup can connect to LM Studio through its local inference server, usually at http://localhost:1234 . This lets your BrainSoup agents use a local LLM running on your computer, which is useful for privacy, cost control, and offline-friendly workflows. If you are new to the product, the BrainSoup overview explains how local LLMs fit into its multi-agent workspace.

This article is for LM Studio setup and its local server tab, local inference tab, or Developer tab, depending on your LM Studio version. If you prefer Ollama's CLI and model library, use the Ollama local LLM setup guide for BrainSoup. If your main problem is Ollama context length or num_ctx , use the Ollama num_ctx  and context length guide.

What is LM Studio?

LM Studio is a desktop tool that allows you to download, manage, and run local LLMs on your machine. It provides a user-friendly interface for selecting, downloading, and loading models, then exposes them through a local API server that BrainSoup can call.

Step 1: Downloading and Installing LM Studio

1. Visit the LM Studio official website to download the latest version of the application suitable for your operating system.
2. Follow the installation instructions provided on the website to set up LM Studio on your machine.

Step 2: Downloading a Model with LM Studio

1. From the Search tab in LM Studio, select the desired model from the list of available models and click on the Download button.
2. Once the download is complete, the model will be listed in the Models dropdown menu, at the top of the screen.
3. Select the downloaded model from the Models dropdown menu to activate it.

Step 3: Enable the Local Inference Server in LM Studio

1. In LM Studio, navigate to the Local Server tab or the Developer tab, depending on your LM Studio version.
2. Choose a port number for the local server, or leave the default value (1234).
3. Click on the Start Server button to activate the server.
4. Make sure the server is serving the model you want BrainSoup to use. The default local inference server URL is http://localhost:1234 . Use localhost  when BrainSoup and LM Studio run on the same computer. Use a network address only if you intentionally expose LM Studio to another machine on your local network.

Step 4: Integrating LM Studio with BrainSoup

1. Navigate to the BrainSoup Settings.
2. Add "lm-studio" as a Provider in BrainSoup, and enter the following URL in the Server URL field: http://localhost:1234  (replace 1234  with the port number you selected in LM Studio). Leave the API Key field empty.
3. Add a model for the "lm-studio" provider. You can name it as "default", as LM Studio only supports one model at a time. Set a context window size suitable for your needs and supported by the model.

For document-heavy or tool-heavy BrainSoup agents, start with a context window of 8192 tokens when the loaded model supports it. Increase the value only if your model and hardware can handle the additional memory usage.

Step 5: Getting Started with Local LLMs in BrainSoup

The local model managed by LM Studio is now accessible within BrainSoup. You can select it for your agents in their respective settings. For this, follow these steps:

1. Open the agent settings by double-clicking on the agent's name in the left pane.
2. In the AI settings section, select the model named lm-studio/default  from the dropdown list.

Troubleshooting

BrainSoup cannot connect to LM Studio

Confirm that the LM Studio local inference server is started and that BrainSoup uses the same URL and port, for example http://localhost:1234 .

The agent answers slowly

Try a smaller model, a lower context window, or a more compact quantization. Local inference depends heavily on CPU, GPU, available memory, and model size.

The wrong model answers

LM Studio generally serves the model currently loaded in its local server. Load the intended model in LM Studio, then test the BrainSoup agent again.

Conclusion

Integrating local LLMs via LM Studio offers unparalleled control over your data privacy and computational resources. With this setup, you're equipped to harness the capabilities of advanced language models while maintaining full ownership of your data and infrastructure.