On-Premise AI Lab

🧪 M4 Pro Blazes Past 125 Tokens! Offline Coding Agent Model flint Debuts: Local AI Lab

Mark Ku

October 26, 2026•17 min read

Hesitant to upload proprietary code to the cloud, but still want a blazing-fast coding agent to help write tests? In this issue, we introduce RecursiveSI/flint, a 39.8B offline code model that clocks 125 tokens per second on an M4 Pro. We also cover ccore/Qwen3.5-4B-3D-Coder, a lightweight model specializing in Three.js, and Carbon-X, a dedicated biomedical model. Which one is the best fit for your local setup? Let's find out!

Featured This Issue

1. RecursiveSI/flint: A 39.8B MoE Offline Coding Agent Model Designed Specifically for Local Deployments

What is it: According to the official README, this is a 39.8B Mixture of Experts (MoE) code generation model optimized for local hardware and licensed under Apache 2.0. It offers extremely fast inference speeds with no commercial restrictions, making it ideal for development teams that require strict data privacy where "data never leaves the local environment."
Where can it be used:
- Offline Coding Agent: Software engineers working on planes, in basements, or in closed offline environments without internet access who need an AI assistant to quickly analyze local projects, auto-fix bugs, and write unit tests.
- Sensitive Proprietary Code Development: Enterprise development teams handling core proprietary code who need a fully local AI assistant to protect intellectual property and security, preventing code from leaking to third-party APIs.
- Low-Latency Development for Individual Developers: Individual engineers working on a MacBook or consumer-grade GPU who want a highly responsive smart assistant that quickly generates code, supports tool calling, and has minimal response times.
Can you run it: The official README notes that the full BF16 weight size is around 80 GB, which can run on a single B300 or H200 GPU. However, for local hardware, official benchmarks show it can reach over 125 tokens per second on an M4 Pro MacBook Pro, and over 150 tokens per second on a 64GB NVIDIA RTX Spark. Additionally, the official release includes an NVFP4 quantized version, which is better suited for consumer-grade hardware deployments.
Current Traction: As of now, this model has a trending score of 886 on Hugging Face, accumulating 8.5k downloads and 926 likes. Furthermore, two quantized or repackaged versions from the same family have already appeared on Hugging Face this week, showing that community interest is rapidly heating up.
How it compares: The creators point out that compared to other 30B to 40B class models, Flint is specifically designed for local hardware, offline operation, and strict data privacy as a coding agent. It achieves impressive scores of 54.9% on Terminal-Bench 2.1 and 89.7% on SWE-bench Pro v2.
How to get it: According to the official README, developers can use huggingface_hub to download the RecursiveSI/flint weights and deploy them using the official Docker and SGLang service packages, or load them via vLLM and enable the glm47 parser.
Link: RecursiveSI/flint

2. ccore/Qwen3.5-4B-3D-Coder: An Ultra-Lightweight 4B Model Specializing in Three.js Interactive 3D Scene Generation

What is it: This is an ultra-lightweight, 4-billion-parameter (4B) model. It is an experimental version independently fine-tuned by CCORE based on Qwen3.5-4B. Licensed under Apache 2.0, it is specifically designed to generate interactive 3D code that can run directly in a web browser.
Where can it be used:
- Rapid Development of 3D Web Components: Front-end web engineers who want to quickly generate Three.js HTML/JS code containing interactive 3D scenes (such as a rotating radar control tower or a 3D cherry blossom garden) without writing it from scratch.
- 3D Voxel-Style Scene Iteration: Indie game developers or web designers who want to quickly iterate and preview 3D voxel-style scene models on a standard laptop using simple text prompts.
- Local Lightweight Agent Experiments: Developers looking to use lightweight agent frameworks (such as Hermes Agent or OpenClaw) locally to automate the generation of 3D web components, completely eliminating the need to call expensive, rate-limited cloud APIs.
Can you run it: Since it is a 4B parameter model, the hardware barrier to entry is very low. The official README provides llama.cpp deployment instructions for the Q4_K_M quantized version. Standard consumer PCs, laptops, and Macs can run it smoothly, making it perfect for individual developers to deploy on the fly.
Current Traction: It currently has a trending score of 113 on Hugging Face, with 1.3k downloads and 115 likes, which is quite popular for a niche, specialized model.
How it compares: The creators note that compared to the general-purpose Qwen3.5-4B model, this fine-tuned version is deeply optimized for generating interactive Three.js 3D scenes. It can directly output a single HTML file containing cdnjs references that runs immediately in a browser, saving developers from tedious package bundling steps.
How to get it: The official README explains that you can use llama.cpp to load ccore/Qwen3.5-4B-3D-Coder:Q4_K_M, and start a local OpenAI-compatible server using the command llama serve -hf ccore/Qwen3.5-4B-3D-Coder:Q4_K_M.
Link: ccore/Qwen3.5-4B-3D-Coder

3. HuggingFaceBio/Carbon-X: A 14.8B Bioinformatics Model Designed for DNA Sequence Understanding and Generation

What is it: This is a 14.8B parameter model specifically designed for DNA sequence understanding and generation, licensed under Apache 2.0. It can comprehend both DNA sequences and natural language, serving as a powerful open-source tool at the intersection of biomedicine and computer science.
Where can it be used:
- Local Analysis of Sensitive Genomic Sequences: Bioinformatics engineers or researchers analyzing sensitive patient genomic sequences who need to predict whether specific DNA mutations (variants) are benign or pathogenic, requiring strict local data processing to comply with medical regulations.
- Bidirectional Dialogue Between DNA and Natural Language: Biomedical developers who need an AI that understands both DNA sequences and human language, allowing users to input raw DNA sequences directly in a chat and have the AI explain their biological functions in English or predict the next base pair.
Can you run it: The official model card indicates that its BF16 weights are approximately 25 GB, which can be loaded and run on a single 80 GB GPU (such as an H100 or H200). According to LLM Explorer, the required VRAM is around 29.7 GB. To lower the barrier to local deployment, the creators also provide a GGUF quantized version, making it easier for developers to run on consumer-grade hardware.
Current Traction: It currently has a trending score of 201 on Hugging Face, with 1.1k downloads and 210 likes.
How it compares: The creators state that compared to the baseline Qwen3-14B model, Carbon-X improved average scores by 27 percentage points across five DNA understanding and generation benchmarks. They also point out that its average performance is close to GPT-6.1 Sol, but at only 1/1800th of the inference cost.
How to get it: According to the official README, developers can load HuggingFaceBio/Carbon-X via Transformers. Note that because it uses a custom hybrid DNA tokenizer, you must specify trust_remote_code=True when loading. Alternatively, you can use vLLM for service deployment.
Link: HuggingFaceBio/Carbon-X

Summary

Today's selection clearly shows that local models are evolving in two main directions: extreme specialization and hardware-specific optimization. Whether it is Flint, optimized specifically for Mac M4, or the smaller models dedicated to 3D front-end development and DNA sequencing, they all demonstrate the immense potential of local deployment. If you happen to have an M4 chip or a consumer-grade GPU on hand, why not start playing with the lightweight 3D-Coder first to experience the magic of local AI? See you in the next issue of the Local AI Lab!