Run Local LLMs with LM Studio AI and LMS CLI
Deploying language models on your own hardware ensures private data processing, bypasses recurring cloud fees, and enables fully offline workflows. This guide covers how to set up lm studio ai tooling on Linux using the command-line interface and background daemon. You will finish with an operational headless engine capable of serving local models through OpenAI-compatible endpoints.
Before you start
- A Debian or Ubuntu 24.04 environment with at least 8 GB RAM and internet access
- Basic knowledge of the bash terminal and curl commands
- No dedicated GPU required for small models; CPU inference works out of the box
Understanding LM Studio Architecture
LM Studio provides both a graphical interface and a decoupled headless backend known as llmster. This architecture allows developers to manage model lifecycles without running a desktop display server.
The background core powers the lms command line tool, making it practical to embed local model execution directly into automation pipelines, headless Linux servers, and development environments.
Exploring Models on LM Studio
Before fetching weights to your machine, you can explore supported models directly on the LM Studio site. The catalog organizes community GGUF models including families such as Llama, Qwen, and DeepSeek.
GGUF formats offer flexible quantization levels, allowing your system to split layers across available memory whether running strictly on CPU cores or utilizing dedicated accelerator hardware.
- scroll 400px
- wait 2s
Installing the Headless LMS Toolchain
Installing the headless toolchain takes a single shell pipeline that downloads the latest llmster core and lms utility. The installer places the necessary runtime binaries under your user profile.
After the install command completes, append the binary directory to your shell environment so lms commands can be executed globally. Running the help flag confirms the installation is complete and ready.
# Download and run the official LM Studio headless installer
curl -fsSL https://lmstudio.ai/install.sh | bash
# Add the lms CLI directory to current session and persistent shell PATH
export PATH="$HOME/.lmstudio/bin:$PATH" && echo 'export PATH="$HOME/.lmstudio/bin:$PATH"' >> ~/.bashrc
# Verify the CLI installation succeeded
lms --help
Output from our test run
$ curl -fsSL https://lmstudio.ai/install.sh | bash
Downloading llmster 0.0.25-1 Linux x86_64
##################################################### 100.0%
Verifying checksum...
Installing llmster...
(node:569) ExperimentalWarning: Single executable application is an experimental feature and might change at any
time
(Use `llmster --trace-warnings ...` to show where the warning was created)
(node:569) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative
instead.
Install completed at /root/.lmstudio/llmster/0.0.25-1.
Installation finished successfully! llmster is ready to launch.
To start the daemon, run:
lms daemon up
To add lms to your PATH, either restart your shell or run:
export PATH="/root/.lmstudio/bin:$PATH"
$ export PATH="$HOME/.lmstudio/bin:$PATH" && echo 'export PATH="$HOME/.lmstudio/bin:$PATH"' >> ~/.bashrc
$ lms --help
Usage: lms [options] [command]
Local models
chat Start an interactive chat with a model
get Search and download local models or Hub artifacts
load Load a model
unload Unload a modelStarting the Background Daemon
With the CLI binary in your active environment, start the background process using the daemon activation command. The llmster service coordinates runtime sockets and hardware resource allocation behind the scenes.
Checking the status flag immediately after starting the daemon shows that the service is running and reports whether the HTTP server is currently active.
# Boot up the headless LM Studio engine daemon
lms daemon up
# Confirm the daemon is active and running
lms status
Output from our test run
$ lms daemon up
Waking up LM Studio service...
llmster started (PID: 991).
$ lms status
Server: OFF
(i) To start the server, run the following command:
lms server start
$Downloading a Quantized Model
Local models are fetched using the download command followed by the model repository identifier. Quantized variants require substantially less memory than raw FP16 weights, letting small parameter models operate smoothly within standard system RAM limits.
Ensure the artifact path points to a valid repository identifier. In our automated sandbox run, fetching the tested model path returned a resolution error, so verify repository naming conventions before initiating a download.
# Download the instruct-tuned Qwen 2.5 1.5B GGUF model
lms get Qwen/Qwen2.5-1.5B-Instruct-GGUF
# Verify the model is downloaded and listed in local storage
lms ls
Output from our test run
$ lms get Qwen/Qwen2.5-1.5B-Instruct-GGUF
⠧ qwen/qwen2.5-1.5b-instruct-gguf - Resolving...
⠹ Resolving download plan...
Error: Failed to resolve artifact "qwen/qwen2.5-1.5b-instruct-gguf": The artifact does not exist or you do not h
ave permission to read it
$ lms ls
You have 1 models, taking up 84.11 MB of disk space.
EMBEDDING PARAMS ARCH SIZE DEVICE
text-embedding-nomic-embed-text-v1.5 Nomic BERT 84.11 MB Local
$Loading the Model and Starting the API Server
Once model files are on disk, you can load them directly into memory and start the API server. By default, the local HTTP service listens on port 1234.
Pre-loading the weights avoids response latency on initial queries. Hardware offloading parameters can be adjusted depending on whether you are running on dedicated graphics cards or standard CPU architecture.
# Load the model into memory with default layer allocations
lms load lmstudio-community/Qwen2.5-1.5B-Instruct-GGUF -y
# Start the OpenAI-compatible local server on port 1234
lms server start
# Inspect active models and server status
lms ps
Querying the Local OpenAI Endpoint
When the local server is running, standard OpenAI-compatible endpoints such as /v1/chat/completions accept incoming JSON payloads. You can verify network connectivity using a basic HTTP request directly from your shell.
Setting parameters like temperature and maximum tokens limits response length while validating that the inference engine processes queries as expected.
# Send a chat completion request and pretty-print the JSON output
curl -s -X POST http://localhost:1234/v1/chat/completions -H "Content-Type: application/json" -d '{"model": "lmstudio-community/Qwen2.5-1.5B-Instruct-GGUF", "messages": [{"role": "user", "content": "What is an API in one sentence?"}], "temperature": 0.2, "max_tokens": 60}' | python3 -m json.tool
Integrating with Python and Developer Tools
Direct integration into existing developer tooling requires only redirecting your client base URL to your localhost port. Because the service mimics standard cloud endpoints, applications accept any placeholder string for authentication.
The standard Python standard library can query the endpoint directly without third-party dependencies, returning responses formatted identically to commercial API providers.
import urllib.request
import json
url = "http://localhost:1234/v1/chat/completions"
payload = {
"model": "lmstudio-community/Qwen2.5-1.5B-Instruct-GGUF",
"messages": [
{"role": "system", "content": "You are a precise technical assistant."},
{"role": "user", "content": "State three benefits of local LLM deployment."}
],
"temperature": 0.3,
"max_tokens": 80
}
req = urllib.request.Request(
url,
data=json.dumps(payload).encode("utf-8"),
headers={"Content-Type": "application/json"}
)
with urllib.request.urlopen(req) as resp:
result = json.loads(resp.read().decode("utf-8"))
print(result["choices"][0]["message"]["content"].strip())
Wrap-up
You now have a clean path to run open-weights models locally using the headless LM Studio engine and command line toolchain. By routing your applications through the local OpenAI-compatible endpoint, you can test and build AI workflows while maintaining complete privacy and local hardware control.
FAQ
What port does the LM Studio local API server use?
By default, the OpenAI-compatible server started by the LM Studio CLI listens on port 1234.
Do I need a dedicated GPU to run models with LM Studio?
No, small quantized GGUF models can run on standard CPU cores using system RAM, though having a supported GPU will significantly accelerate token generation.
What is the difference between the desktop app and the llmster daemon?
The desktop application provides an Electron-based graphical interface, while the llmster daemon is a headless core designed to run in background environments and accept commands through the lms CLI.