Lukas Tanaka

Posted on Sep 5

Run Local LLMs on Mac in 2026: LM Studio, Ollama and MLX Guide

#ai #llm #macos #tutorial

This guide is for Apple Silicon owners who want to run local LLMs on Mac for chat, development, and coding experiments.
You will download a model, start a conversation, expose a local API, and connect a coding client.
Choose one installation route below; the commands are documentation-based examples, not results from our own hardware tests.

Requirements: software versions and Apple Silicon hardware

Versions checked on September 5, 2026. Choose LM Studio for a graphical model browser, Ollama for a command-line workflow, or MLX with mlx-lm for Python control. These are workflow recommendations; their documented interfaces are linked below.

Component	Verified version or requirement	Primary source
LM Studio	0.4.23; Apple Silicon; macOS 14.0+; 16 GB+ RAM recommended	Download, requirements
Ollama	0.33.3, latest stable GitHub release; macOS 14 Sonoma+; Apple M-series CPU/GPU support	Release, download, Mac requirements
MLX	0.32.2 on PyPI; native Apple Silicon Python 3.10+ and macOS 14.0+	PyPI, installation
mlx-lm	0.31.3 on PyPI; use the stricter MLX Python requirement above; large-model memory wiring requires macOS 15.0+	PyPI and requirements
Model storage	Download size depends on the exact quantization; example packages below range from 1.81 GB to 67.8 GB, before caches	Linked Hugging Face cards in Step 1

Memory capacity and memory bandwidth are separate specifications. Apple's published MacBook Pro figures are:

Chip tier	M4 generation	M5 generation
Base	120 GB/s	153 GB/s
Pro	273 GB/s	307 GB/s
Max, 32-core GPU / 40-core GPU	410 / 546 GB/s	460 / 614 GB/s

Sources: Apple's M4 specifications, M4 Pro/Max specifications, and M5 MacBook Pro specifications. These are bandwidth figures, not measured LLM generation speeds. Do not translate the percentage difference into a promised tokens-per-second improvement.

For a broader hardware comparison, see the full Local LLMs guide. If you are also considering a separate machine, compare mini PCs for local LLMs.

1. Choose a 4-bit model for your unified memory

Start with the downloaded package size, then leave room for macOS, applications, runtime buffers, and the model's context cache. LM Studio's memory estimator explicitly accounts for context and vision settings; mlx-lm also documents memory costs from model weights and caches. A download that nearly fills physical memory is a poor starting point. Memory estimation, MLX LM memory guidance.

The table uses specific MLX 4-bit packages, with sizes displayed on their Hugging Face model pages. “Candidate” is our conservative capacity assessment for a single model and modest context, not a verified load test. “Avoid” means choose a smaller package for this guide. Larger contexts require another memory check.

Model and source package	Published package size	16 GB	32 GB	64 GB	128 GB
Llama 3.2 3B Instruct, 4-bit	1.81 GB	Candidate	Candidate	Candidate	Candidate
Qwen3.5-9B, 4-bit	5.95 GB	Candidate	Candidate	Candidate	Candidate
Qwen3.6-27B, 4-bit	16.1 GB	Avoid	Candidate	Candidate	Candidate
Gemma 4 31B IT, 4-bit	18.4 GB	Avoid	Candidate	Candidate	Candidate
DeepSeek-R1-Distill-Qwen-32B, 4-bit	18.4 GB	Avoid	Candidate	Candidate	Candidate
Llama 3.3 70B Instruct, 4-bit	39.7 GB	Avoid	Avoid	Candidate	Candidate
Mistral Small 4 119B, 4-bit	67.8 GB	Avoid	Avoid	Avoid	Candidate

This is a selection for use in 2026, including earlier Llama releases and the DeepSeek distillation. Package sizes are decimal GB and are not peak unified-memory measurements. GGUF builds of the same model can have different sizes. The linked Qwen, Gemma, and Mistral multimodal packages document mlx-vlm; follow those cards for image input. The standalone mlx-lm walkthrough below uses text-only Llama.

Two larger releases need an explicit exclusion. GLM-5.3-Flash has 320B total / 18B active parameters. DeepSeek-V4-Flash has 284B total / 13B active. Calculating total parameters × 4 bits ÷ 8 gives roughly 160 GB and 142 GB of weights alone. Neither is a fully resident 4-bit recommendation for these memory tiers; active parameters do not represent the entire checkpoint.

Tokens/sec: not published for the exact Mac, model, and quantization combinations in this table by the cited model cards. We therefore give no generation-speed estimates. Record your own chip, runtime version, context, and quantization before comparing results.

Use the LLM GPU calculator for additional planning, and check the AI model releases timeline when revisiting your shortlist.

2. Install LM Studio and start a chat

Follow one of Steps 2–4, then continue to Step 5. Keeping the first experiment to one runtime makes its memory use easier to diagnose.

Download the macOS build from the official download page and open LM Studio once. Its bundled lms command should then be available in a new terminal. The app's Discover tab accepts model names or Hugging Face URLs; the CLI offers the same download workflow. CLI setup, model discovery.

lms --help
lms get --mlx mlx-community/Llama-3.2-3B-Instruct-4bit
lms ls
lms load --context-length 4096 --identifier local-chat
lms chat local-chat

Choose the downloaded Llama package when lms load prompts you. This creates a loaded model named local-chat and starts an interactive conversation. Ask it to explain a short function, then inspect whether the response answers your question. You can also select the downloaded model in the app's chat interface. Download flags, loading options, chat command.

For another package, inspect the exact repository and quantization before downloading. LM Studio recommends choosing a 4-bit or higher option when the machine can accommodate it. Its support for both GGUF and MLX is documented in the app overview.

3. Install Ollama, pull Qwen, and chat

Download Ollama for macOS, drag the application into Applications, and launch it. The app checks that its CLI is available and offers to create the command link if necessary. Mac installation.

ollama --version
ollama pull qwen3.5:9b
ollama run qwen3.5:9b

The first model command downloads the specified tag; the second opens a chat. This example uses Ollama's published Qwen3.5 9B tag, whose package is separate from the MLX package in the memory table. Check its download details rather than assuming identical storage requirements.

In another terminal, inspect the loaded model:

ollama ps

This shows the running model, processor allocation, and allocated context. The app supplies the server; use ollama serve if running the CLI without an existing server. Avoid starting a second server on the same port. CLI reference, context inspection.

4. Install MLX and mlx-lm in a Python environment

Use a native Apple Silicon Python meeting the requirements above. Create an isolated environment so that this experiment has its own packages. Python virtual environments.

python3 -m venv .venv-local-llm
source .venv-local-llm/bin/activate
python -m pip install --upgrade pip
python -m pip install mlx==0.32.2 mlx-lm==0.31.3
mlx_lm.chat --model mlx-community/Llama-3.2-3B-Instruct-4bit

This installs the verified package versions and opens a chat with the downloaded model. The upstream installation commands are pip install mlx and pip install mlx-lm; the pins above make the guide's package choices explicit. MLX repository, MLX LM repository.

For a single response instead of an interactive session:

mlx_lm.generate \
  --model mlx-community/Llama-3.2-3B-Instruct-4bit \
  --prompt "Explain a Python virtual environment in three sentences."

The model identifier selects an existing quantized package; you do not need to download full-precision weights and convert them first. Keep this small text example working before substituting a larger or multimodal checkpoint.

5. Expose and check an OpenAI-compatible endpoint

For LM Studio, start its server after loading local-chat:

lms server start --port 1234

Ollama already serves its API while the app or ollama serve is running. For standalone MLX, exit the chat first, keep the virtual environment activated, and run:

mlx_lm.server \
  --model mlx-community/Llama-3.2-3B-Instruct-4bit \
  --host 127.0.0.1 --port 8080

These commands produce local HTTP services. Keep the chosen service running while using its client. The base URLs and model identifiers for the examples are:

Runtime	OpenAI-compatible base URL	Example model identifier
LM Studio	`http://127.0.0.1:1234/v1`	`local-chat`
Ollama	`http://127.0.0.1:11434/v1`	`qwen3.5:9b`
mlx-lm	`http://127.0.0.1:8080/v1`	`mlx-community/Llama-3.2-3B-Instruct-4bit`

Sources: LM Studio API, Ollama API, MLX LM server.

Test Ollama with a model listing followed by a chat request:

curl -sS http://127.0.0.1:11434/v1/models
curl -sS http://127.0.0.1:11434/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"qwen3.5:9b","messages":[{"role":"user","content":"Say hello."}],"stream":false}'

For another runtime, substitute its port and model identifier from the table. A successful request should return a JSON response containing generated text. Resolve connection or model-name errors here before adding an editor. OpenAI compatibility describes supported API operations; it does not promise every client feature. In particular, the cited mlx-lm server implements Chat Completions, while Codex's LM Studio integration uses Responses.

6. Connect Claude Code, Codex, or Continue

The following assumes your chosen coding client is installed. Start with one small file and a simple explanation request before asking the model to edit a repository.

Claude Code through Ollama

Claude Code needs Ollama's Anthropic-compatible API, so its base URL has no /v1 suffix. The documented manual configuration can be scoped to one command:

ANTHROPIC_AUTH_TOKEN=ollama \
ANTHROPIC_API_KEY="" \
ANTHROPIC_BASE_URL=http://127.0.0.1:11434 \
claude --model qwen3.5:9b

This connects the Claude Code client to a local Qwen model; it does not download Claude weights. Ollama also provides ollama launch claude --model qwen3.5:9b. Its integration guidance recommends 64k+ context for local coding work, which can require more memory than the initial chat configuration. Claude Code integration.

Codex through Ollama or LM Studio

OpenAI documents local providers through --oss and --local-provider. To use the Ollama model already downloaded:

codex --oss --local-provider ollama -m qwen3.5:9b

For LM Studio, use --local-provider lmstudio and the identifier of a loaded model suitable for coding and tool use. LM Studio documents Codex's /v1/responses connection and recommends more than approximately 25k context. Do not assume the small Llama chat example is an adequate coding agent just because its API responds. OpenAI configuration, LM Studio Codex integration.

Continue with a local provider

Merge this model entry into your Continue YAML configuration, preserving existing entries:

name: Local Mac
version: 0.0.1
schema: v1
models:
  - name: Local Qwen
    provider: ollama
    model: qwen3.5:9b
    apiBase: http://127.0.0.1:11434
    defaultCompletionOptions:
      contextLength: 4096

Select Local Qwen and try a chat. This deliberately starts with modest context. For LM Studio, use provider: lmstudio, model: local-chat, and apiBase: http://127.0.0.1:1234/v1. A standalone MLX Chat Completions server can use Continue's openai provider with its matching base URL and model identifier. Ollama configuration, LM Studio configuration, compatible providers.

Troubleshooting

“Model requires more system memory” or severe memory pressure

Unload unused models with ollama stop qwen3.5:9b or lms unload --all, close other memory-heavy applications, and retry with a smaller model or context. Continue documents that its context settings can trigger this error even when the same model works elsewhere. In LM Studio, run lms load --estimate-only and select the package before loading it. Continue fix, LM Studio estimator.

GGUF loads in one tool but the MLX package fails

Match the runtime to the package. LM Studio supports llama.cpp/GGUF and MLX; the linked multimodal MLX cards prescribe mlx-vlm. Download the intended format and follow that package's instructions rather than renaming a file. Ollama's current releases also include an MLX engine, so treating it as permanently “GGUF only” is outdated. LM Studio runtimes, Ollama release.

Chat works, but a longer prompt or coding agent fails

Check both the model's supported context and the runtime's allocation. Increasing context consumes additional memory; a model card's maximum is not a promise that your Mac can allocate it. Inspect ollama ps, change the app's context setting, and shorten the input while diagnosing. For a CLI-managed server, quit the app first and use OLLAMA_CONTEXT_LENGTH=64000 ollama serve only after checking capacity. Context settings.

MLX cannot install, or Metal reports a build error

For “no matching distribution,” verify native Python with python -c "import platform; print(platform.processor())"; MLX's guide expects arm, rather than an Intel interpreter. If building from source and seeing “unable to find utility metal,” check the documented Xcode installation and active developer directory. Prefer the published Python packages for this walkthrough. For an Ollama GPU failure, inspect ~/.ollama/logs/server.log before changing settings. MLX troubleshooting, Ollama logs.

Connection refused, unknown model, or unsupported endpoint

Confirm the server is running, then repeat Step 5's model-list and chat requests with its exact port. Copy the returned model identifier into the client. If chat succeeds but a coding client fails, check its required API: Claude Code uses Anthropic compatibility, and Codex requires the documented Responses integration. A working Chat Completions endpoint alone does not establish either connection.

FAQ

Can I run local LLMs on a 16 GB Mac?

Yes, start with a small quantized model and modest context. The cited Llama 3.2 3B and Qwen3.5-9B MLX packages occupy 1.81 GB and 5.95 GB respectively, leaving more planning headroom than the larger packages. Check actual runtime memory before expanding context.

Is LM Studio, Ollama, or MLX best for Apple Silicon?

Choose LM Studio if you want a graphical model browser, Ollama if you prefer pulling named models from a terminal, and mlx-lm if you want direct Python control. These recommendations follow their documented workflows. This guide has no matched benchmark establishing a universal speed winner.

Does local inference work without internet?

LM Studio explicitly documents offline chat and local serving after downloading models. Complete model and runtime downloads first. When adding coding clients, review their tools and provider configuration separately; local inference alone does not establish that every connected feature is offline. Offline operation.

Does 4-bit mean the entire model uses exactly half a byte per parameter?

Treat that calculation as a rough starting point. The linked quantized packages include integer weights alongside floating-point tensors, and their published sizes differ from a simple parameter-count calculation. Use the actual package size, then account for runtime memory and context.

Can a 128 GB Mac run GLM-5.3-Flash or full DeepSeek-V4-Flash at 4-bit?

Neither is a fully resident recommendation here. Their total parameter counts imply approximately 160 GB and 142 GB of weights alone under an idealized 4-bit calculation. The smaller active-parameter figures describe computation per token, not the storage requirement for the complete model.

PromptZone - AI Prompts, Guides and Tools for Builders