Opening

This week's most-downloaded model is Ternary-Bonsai-2-27B-gguf, which claims to run a 27B model on a laptop with a file size under 8GB, racking up 405,000 downloads in a week. Also in this episode: the phone-class MiniCPM5-2B, and Edge0-35B-A3B-preview, which lets a 24GB Mac mini run a 35B LoRA. We'll walk through each model's hardware requirements and hidden caveats one by one.

Featured This Week

1. Ternary-Bonsai-2-27B-gguf: squeezing 27B into a ternary-quantized build under 8GB

• What it is: This is a ternary-quantized GGUF build of the Qwen3.8-27B series from prism-ml, 27B parameters, Apache-2.0 licensed, aimed at drastically shrinking the original 53.8GB F16 version. The official model card describes it as "_{5.9 GB language model (down from} 54 GB FP16)."
• Where it's useful:
  • Freelance engineers handling sensitive client contracts or internal documents that can't be sent to cloud APIs. Here, a laptop with a discrete GPU can run a 27B-class model offline for summarization and Q&A. The official model card explicitly lists this use case as "privacy-sensitive and offline settings" and "laptop-local 27B agents."
  • Teams with only one consumer-grade GPU that still want to self-host an internal Q&A service. The PQ2_0 file is just 7.21GB, and the official positioning is "single-GPU and commodity-GPU serving," so there's no need to rent an A100 to run a 27B model.
  • Mac-based developers who want a local coding assistant with thinking/reasoning mode support. The official throughput table lists Apple M5 Pro decoding at 28.1 tok/s and M5 Max at 47.0 tok/s (both PQ2_0), which is still fast enough for interactive use.
• Will it actually run: The official model card lists three files: PTQ1_0 (dense trits) at 5.95GB, PQ2_0 (2-bit slots) at 7.21GB, versus the F16 reference version at 53.8GB, plus an optional vision tower (Q8_0) at 0.63GB. On throughput, the card also lists RTX 5090 at 129.9 tok/s and H100 SXM at 113.9 tok/s (also PQ2_0 decoding). Note that the official card does not specify minimum RAM/VRAM requirements, so we can't offer a guess where there are no numbers to go on.
• This week's traction: HF trending score 881 (405,600 downloads, 926 likes), with 2 other quantized/repackaged variants currently in the same family.
• How it compares: The author's own comparison table places it within the same Qwen3.8-27B quantization family: an average of 84.78 across 14 benchmarks, beating the traditional IQ2_XXS's 72.59, and only slightly behind the three-times-larger UD-Q4_K_XL (85.18). The card describes this as "far above the conventional IQ2_XXS build at less than two-thirds of its footprint." That said, a third-party hands-on review from MindStudio notes it performs well on vision and creative tasks but is less stable on bug fixes and basic UI code generation, so results vary by task.
• How to get it: The README's instructions are hf download prism-ml/Ternary-Bonsai-2-27B-gguf Ternary-Bonsai-2-27B-PQ2_0.gguf --local-dir ., then launch with llama-cli -m ... -ngl 99 -fa on -c 32768, or use ollama run hf.co/prism-ml/Ternary-Bonsai-2-27B-gguf:F16. Importantly, the README explicitly states "Stock llama.cpp will not run these files," so you'll need to switch to PrismML's own llama.cpp fork to get it running, a caveat worth knowing before you start.
• Link: huggingface.co/prism-ml/Ternary-Bonsai-2-27B-gguf

2. MiniCPM5-2B: small enough to fit on a phone, a foundation for on-device agents

• What it is: A 2B-parameter model from OpenBMB, Apache-2.0 licensed, shipped simultaneously in GGUF/MLX/GPTQ/LiteRT formats, clearly aimed at on-device and resource-constrained scenarios.
• Where it's useful:
  • App developers who want to build an offline assistant into Android/iOS apps without racking up API costs per user. The official release directly provides a .litertlm version, and the model card states LiteRT-LM supports "Android / iOS / desktop / IoT, CPU + GPU."
  • Anyone wanting to self-host a persistent agent for tasks like log organization or internal API calls. The official card describes the use case as "local assistants, coding agents, tool-use workflows," and lists a BFCL v4 tool-calling score of 66.6.
  • Handling long documents (regulations, meeting transcripts) on a single machine without a GPU budget. The official card lists a native 131,072-token context window and can run on low-end hardware using the 4-bit GPTQ version.
• Will it actually run: The official card doesn't specify concrete RAM/VRAM thresholds, nor any tokens/s figures. What it does specify are format options: the official quantized repos include MiniCPM5-2B-GGUF (llama.cpp/Ollama/LM Studio), MiniCPM5-2B-MLX (Apple Silicon), and MiniCPM5-2B-GPTQ (4-bit). The card's description of hardware is limited to positioning language like "on-device, local deployment, and resource-constrained scenarios." We can only infer from the 2B parameter count and four available formats that it should run fine on lightweight devices, but the actual speed isn't given, so this is an estimate, not a guarantee.
• This week's traction: HF trending score 338 (357,200 downloads, 1.6K likes), with 2 other quantized/repackaged variants currently in the same family.
• How it compares: OpenBMB's own comparison table claims an average of 53.9 across 34 benchmarks, beating LFM2.5-2.6B (33.2), Qwen3.5-2B (28.0), and Gemma-4-E2B-it (24.6), even edging out the larger 4B-class Qwen3.5-4B (51.1). The card's phrasing is "remains competitive with 4B-class models overall." However, third-party review sites eesel AI and buildfastwithai note that these results come from OpenBMB's own curated benchmark set, and leading on benchmarks doesn't guarantee production reliability, so it's worth testing against your own tasks first.
• How to get it: The README's transformers example is AutoModelForCausalLM.from_pretrained("openbmb/MiniCPM5-2B", torch_dtype="auto", device_map="auto"); to start a service, the official instructions are vllm serve openbmb/MiniCPM5-2B --port 8000; via llama.cpp it's llama-server -m MiniCPM5-2B-F16.gguf -a MiniCPM5-2B --port 8080.
• Link: huggingface.co/openbmb/MiniCPM5-2B

3. Edge0-35B-A3B-preview: running 35B on a Mac mini via LoRA adapter

• What it is: A LoRA/adapter release from the Edge0 team, built on the 35B MoE model Qwen3.6-35B-A3B, using int4 quantization combined with LoRA and a prerouter adapter, Apache-2.0 licensed. The official card explicitly states the backend is MLX, currently targeting Apple Silicon only.
• Where it's useful:
  • Apple Silicon developers who want to run a 35B-class model locally for multilingual Q&A. The official card lists peak active memory of just 2.9 GiB, along with a benchmark table showing the Mac mini M4 Pro (24GB) decoding at 14.9 to 17.7 tok/s.
  • Anyone who wants to serve multiple tasks simultaneously without re-quantizing each time. The card describes the use case as "batch serving via LoRA adapters without re-quantization," suitable for self-hosted services running multiple adapters.
  • Edge deployment scenarios where VRAM is tight but disk space is plentiful. The card's positioning statement is "edge / on-device inference where GPU VRAM is scarce and storage is fast," with the official card listing a total model footprint of 19.6GB.
• Will it actually run: The official card is quite thorough here: peak active memory of 2.9 GiB (short context), total storage of 19.6GB, using 4-bit (int4) with LoRA and a prerouter adapter, backend MLX. The card states directly, "currently targets Apple Silicon," and warns, "Long contexts grow the KV cache; use shorter contexts to keep peak memory at 3 GiB." On speed, the official numbers show Mac mini M4 Pro 24GB decoding at 14.9 to 17.7 tok/s, with cold/warm prefill at 113/140 tok/s.
• This week's traction: HF trending score 250 (52,500 downloads, 3.4K likes).
• How it compares: Worth pointing out: the author only compares against their own fp16 base model, Qwen3.6-35B-A3B, with no comparison against other edge inference solutions. The official table shows the int4 version averaging 79.2 versus fp16's 83.2, an average degradation of 3.9 points by the author's own account. The card also states outright that this is an "early preview release," and openly admits agent capability is "currently weak," unsuitable for long-horizon autonomous tasks, a limitation the author states themselves, worth mentioning when introducing this model.
• How to get it: The README requires first installing the official framework pip install -e 'git+https://github.com/Edge0-AI/edge0.git#egg=edge0[fetch]', then huggingface-cli download Edge0/Edge0-35b-a3b-preview --local-dir ./Edge0-35b-a3b-preview, followed by edge0 chat --name edge0-35b for chat, or edge0 serve --port 8085 to start an OpenAI-compatible API. The card also provides an mlx_lm route via load("Edge0/Edge0-35B-A3B-preview").
• Link: huggingface.co/Edge0/Edge0-35B-A3B-preview

Closing

These three picks share a common thread: they're all finding ways to squeeze large models into small memory footprints. Ternary-Bonsai uses ternary quantization to compress 27B down under 8GB, MiniCPM5-2B is built directly for phone formats, and Edge0 uses LoRA to push a 35B model onto a Mac mini. If you happen to have a laptop with a discrete GPU and documents you can't send to the cloud, Ternary-Bonsai-2-27B-gguf is probably this week's top pick worth clicking into the model card to check the requirements carefully. See you next Wednesday for more open-source models worth watching.