The Best Local AI for Each Job at 8, 16, 32, 64 and 128GB
As of November 2026: at 8GB run Qwen3.5 4B; at 16GB, Qwen3.5 9B or Gemma 4 12B; at 32GB, Qwen3.8 27B at 4-bit; at 64GB, the same 27B at 8-bit; at 128GB, gpt-oss 120B or Qwen3.5 122B. Add a small embedding model for RAG and a speech model at any size.
The arithmetic that decides every pick
A model's weights take roughly parameters times bits per weight, divided by eight, in bytes. Qwen3.8 27B has about 27.8 billion parameters on its Hugging Face card. At 16-bit that is about 56GB, which is exactly the size of the bf16 build in the Ollama library. At 8-bit (q8_0, which carries a little overhead per block) it is 30GB. At the common 4-bit format, q4_K_M, which averages a bit under five bits per weight, it is 18GB including the vision encoder.
Then add what the weights do not cover. The KV cache grows with context length, so a model that loads with 2GB to spare still fails on a long document. Your operating system and open apps need their share. On a Mac, macOS by default lets the GPU wire only part of total memory, roughly two thirds to three quarters depending on the machine, so a 16GB Mac does not have 16GB for a model. A practical rule: budget the download size plus 20 to 30 percent, and keep the result under about 70 percent of total memory.
Mixture-of-experts models change one half of the equation. Memory follows total parameters, but speed follows active parameters. Qwen3.6 35B-A3B needs about 23GB at 4-bit, yet only about 3 billion parameters work on each token, so it generates far faster than a dense 27B. The full derivation for squeezing large models into small machines is in the guide to running a 30B model on 16GB.
The picks by memory size
Every size below comes from the Ollama library as of November 2026. Downloads in LM Studio's GGUF and MLX builds are close to these numbers. All of the chat models listed accept images as well as text unless noted, which covers reading photos, receipts and scans.
8GB: small and specific
Qwen3.5 4B at 4-bit (3.3GB) or Gemma 4 E4B in its QAT build (6.1GB, tighter) for chat, rewriting and short questions. Expect a capable assistant for drafts and summaries, not for multi-step reasoning. For embeddings, EmbeddingGemma (622MB) or Qwen3 Embedding 0.6B (639MB) fit alongside. Skip local coding agents at this size.
16GB: the first genuinely useful tier
Qwen3.5 9B at 4-bit (6.6GB) or Gemma 4 12B in QAT (7.2GB) for chat, email and reading documents, with room left for context. Qwen3.5 9B at 8-bit (10GB) is the quality step if nothing else is open. The 27B class at 18GB does not fit here.
32GB: the 27B class
Qwen3.8 27B at 4-bit (18GB) is the default pick for writing, document questions, vision and coding help, with enough headroom for a useful context window. Qwen3.6 35B-A3B (about 23GB) is the faster alternative when speed matters more than the last bit of quality. Devstral Small 2 24B (15GB) is a focused option for tool-using coding agents.
64GB: the same models, better
The best use of 64GB is usually not a bigger model but a better build of the same one: Qwen3.8 27B at 8-bit (30GB) with its long context window intact, plus an embedding model and a speech model loaded at once. The 35B-A3B coding builds at 8-bit (about 38GB) also fit.
128GB: large mixture-of-experts models
gpt-oss 120B (65GB), Qwen3.5 122B-A10B at 4-bit (81GB) and Qwen3-Coder-Next (52GB at 4-bit, 85GB at 8-bit) become practical. Qwen3.8-Flash-Next, with 125 billion parameters plus a 51 billion parameter n-gram embedding table on its model card, is the heaviest realistic pick at around 4-bit. Bigger flagships such as GLM-5.3 appear in Ollama only as cloud models.
The jobs that are not chat
The source video, Alex Grankin's tour of twelve everyday jobs on a 128GB MacBook Pro, spends half its time on work that is not a chatbot, and those jobs need far less memory. Speech to text runs on NVIDIA's Parakeet TDT 0.6B, Whisper or Qwen3-ASR (0.6B and 1.7B), all small enough for any machine; the video uses the Handy app for dictation. Image generation uses FLUX.3 klein in Draw Things; NVIDIA's own blog lists that model at 5.6 billion parameters running in as little as 8GB of graphics memory. The video's agent demo uses Hermes Agent driving a local model.
The practical consequence: on 32GB or more, run a speech model, an embedding model and a 27B chat model together rather than chasing one huge model. Each job gets a specialist, and none of them fight for memory.
Coding and agents need more than the model fits
Coding is the job where memory size matters most, because agents read files, run tools and accumulate long contexts. A 27B model at 4-bit answers code questions well on 32GB, but a long agent session fills the KV cache quickly, and quality on large multi-file changes still trails hosted frontier models. The guide to using a local LLM for coding covers which coding tasks suit a local model and how to set up the editor side.
For agents, the model is one cost and the context is the other. Every document pasted into the prompt costs memory and time on every turn. That is the case for retrieval in front of the model: an embedding model indexes your files, and each question pulls back a few relevant passages instead of whole documents.
Retrieval stretches every memory tier
Small local models fail more often for lack of facts than for lack of reasoning. Giving a 9B model the right three paragraphs usually does more than moving to a model twice its size, and it costs a fraction of the memory. Embedding models are the cheapest component in the stack: Qwen3 Embedding 4B is 2.5GB at 4-bit and 8B is 4.7GB, and both fit beside a chat model from 16GB up.
RDK is built for that layer. It indexes local folders and Obsidian vaults as encrypted private chunks, and agents query it through MCP, the CLI or the Partner API, getting a few hundred tokens of context instead of whole documents. Chunk content stays on your node. On repeated or reference-heavy work, retrieval instead of regeneration cuts token spend 80 to 90 percent, and for a local model the same saving shows up as a shorter context and a faster answer.
Frequently asked questions
- What is the best local AI model for 16GB of RAM?
- As of November 2026, Qwen3.5 9B at 4-bit, a 6.6GB download, or Gemma 4 12B in its QAT build at 7.2GB. Both handle chat, documents and images. Qwen3.5 9B at 8-bit, 10GB, is higher quality if little else is open. The 27B class needs about 18GB for weights alone, so it does not fit in 16GB.
- How do you calculate how much memory a local model needs?
- Multiply parameters by bits per weight and divide by eight to get bytes. A 27.8B model is about 56GB at 16-bit, 30GB at 8-bit and 18GB at 4-bit. Add the KV cache, which grows with context, plus the operating system. Keep the total under about 70 percent of your machine's memory.
- Does 4-bit quantization make a model much worse?
- Modern 4-bit formats such as q4_K_M keep most of a model's quality for everyday chat, writing and document work, which is why they are the default downloads. Losses show up first in precise tasks: long reasoning chains, exact code and arithmetic. If a 27B model at 4-bit and a 9B model at 8-bit both fit, the larger model at 4-bit usually wins.
- Which local model should you use for embeddings and RAG?
- EmbeddingGemma at 622MB or Qwen3 Embedding 0.6B at 639MB on small machines, and Qwen3 Embedding 4B or 8B, 2.5GB and 4.7GB at 4-bit, when you have room. Pick one and keep it: changing the embedding model means re-embedding the whole corpus, because vectors from different models are not comparable.
- Is 128GB worth it for local AI?
- Only if you need the large mixture-of-experts models such as gpt-oss 120B or Qwen3.5 122B-A10B, or several models loaded at once. For most single-user work, 32GB to 64GB running a 27B model plus retrieval covers chat, documents and coding help. Bandwidth, not capacity, then decides how fast the answers arrive.