---
title: "Modifying Only 131,072 Weights: Qwen3.5-0.8B, Whose README Claims GSM8K Jumped from 13.8% to 67.1% | On-Premise AI Lab"
description: "A 0.8B small model where the author says no training was done at all, only 131,072 weights were touched, and the README table claims GSM8K jumped from 13.8% to 67.1%: that's this issue's RikiyaT/Qwen3.5-0.8B-weight-space-programming."
canonical_url: "https://blog.markkulab.net/en/local-ai-lab/local-ai-lab-2026-11-04"
author: "Mark Ku"
author_url: "https://blog.markkulab.net/en/author/mark-ku"
site: "Mark Ku's Tech Notes"
date_published: "2026-11-04T02:00:00.000Z"
category: "On-Premises AI Lab"
tags: ["local-ai", "podcast", "地端模型", "lora", "開源模型", "Ripple-65B-Preview", "Apriel-Voice-27B-preview", "Qwen3.5-0.8B-weight-space-programming", "agentic-coding", "語音模型", "vLLM"]
language: "en"
license: "CC BY 4.0"
license_url: "https://creativecommons.org/licenses/by/4.0/"
attribution: "when reusing or quoting, credit the author and link back to the original"
---

# Modifying Only 131,072 Weights: Qwen3.5-0.8B, Whose README Claims GSM8K Jumped from 13.8% to 67.1% | On-Premise AI Lab

## Opening

A 0.8B small model where the author says there was no training at all, just 131,072 weights touched, and the README table claims GSM8K jumped from 13.8% to 67.1%. That's this episode's RikiyaT/Qwen3.5-0.8B-weight-space-programming. The other two are the Apache-2.0 licensed ricursiveAI/Ripple-65B-Preview and the MIT licensed ServiceNow-AI/Apriel-Voice-27B-preview, one of which has an official serving example that goes straight to four GPUs. I'll tell you which one in a moment. The official release dates for all three are unsourced, so every number below comes from the HF pages as captured for this episode.

## This Episode's Picks

### 1. ricursiveAI/Ripple-65B-Preview: an agentic coding model that the vendor says fits on consumer hardware at 65B dense

- **What it is**: A 65B parameter dense model, Apache-2.0 licensed, with safetensors weights available for direct download. This repo is the BF16 instruct version. The official model card positions it as an agentic coding model that "runs on local devices," claims 80.52 on TerminalBench 2.1, and states support for up to 1M context length. The team shipped four repos at once: BF16 instruct, an NVFP4 quantized version, a pretrain-only Base Preview, and DFlash2.

- **Where you'd use it**:
  - **Backend engineers who want to bring a coding agent inside the corporate network**: You have an internal service and want an agent that can open a terminal, run tests, and fix bugs by following stack traces, but the code can't leave the data center. The official model card emphasizes agentic coding strength, so you can wire it into terminal/SWE-style agent harnesses.
  - **Teams feeding in whole codebases or long spec documents**: The vendor claims support for up to 1M context length, which suits dropping an entire repo or a long, tedious system spec in for cross-file Q&A and refactoring plans, the kind of tasks that require seeing many files at once to say anything useful.
  - **People doing post-training experiments on a local box**: The vendor says Base Preview is a "pretrain-only checkpoint" with no midtraining and no synthetic QA data. For anyone wanting to do their own SFT or LoRA experiments, that clean starting point is arguably more valuable than the instruct version.

- **Can you actually run it?**: The official model card's exact words are "At only 65B total parameters, the model can fit on consumer hardware such as the NVIDIA DGX Spark," and it says NVFP4 checkpoints and a colab are provided to encourage local development. The vendor also mentions that on data center GPUs, "2xB200 at batch=8 reaches 250 tok/s (before quantization and custom kernels)." **The single consumer GPU threshold is not stated by the vendor.** If you just want a rough sense of scale: at 65B × BF16, the weights alone land in the tens-to-hundreds-of-GB range. That's our own arithmetic, not an official figure, and not a measured result. If you genuinely want to go local, the path the vendor points to is NVFP4. Also note the repo is tagged custom_code, so you'll need to allow remote code at load time.

- **Buzz this episode**: HF trending score 107, 0 downloads, 109 likes. Likes are clearly running ahead of downloads, suggesting people are liking and watching rather than actually pulling the weights down.

- **Versus comparable models**: The official model card claims that by going dense rather than MoE, it matches or beats Reflection Beam 501B, Meta Muse Spark Open 816B, and Nemotron 3 Ultra 550B, MoE models ten times its size, at just 65B. But in the daily.dev community discussion, most commenters see "U.S. open-weight SOTA" as a narrowly scoped marketing claim. On X, a thread from @kimmonismus questions whether a 65B dense model can run agent loops at acceptable speed on a device with roughly 273GB/s memory bandwidth. All of the above is other people's commentary; this show has not run the model.

- **How to get it**: The HF page publishes safetensors weights, the license page says Apache 2.0, and the model card links out to all four repos (BF16/NVFP4/Base/DFlash2). The vendor says that to run the local quantized version, start from the NVFP4 checkpoint and the provided colab.

- **Link**: https://huggingface.co/ricursiveAI/Ripple-65B-Preview

### 2. ServiceNow-AI/Apriel-Voice-27B-preview: an MIT licensed, self-hostable speech understanding model covering 49 languages

- **What it is**: A 27B parameter speech model, MIT licensed, with downloadable safetensors weights and a companion arXiv technical report. The official table lists its capabilities as "transcription, translation, and spoken question answering," with support for 49 languages. One important caveat the vendor states outright: this checkpoint "produces text and tool calls, not native full-duplex speech," so you have to bolt on an external TTS for speech synthesis.

- **Where you'd use it**:
  - **Teams self-hosting a voice support frontline**: The model card says that via a shared Gemma 4 backbone it "understands spoken requests, reasons over conversational context, and produces text responses and tool calls," and the vendor recommends pairing it with an external speech synthesizer to build an interactive voice assistant. In other words, you use it as the layer that understands what the human said and decides which API to call, and you pick your own TTS.
  - **Transcribing and summarizing internal meetings and interview recordings**: The long audio section of the model card notes that the processor automatically chunks 16 kHz mono long-form recordings into 30-second segments, and the vendor says it performed well on continuous recordings up to about 30 minutes in testing, with the serving example even including a configuration for one-hour audio. For companies where meeting recordings can't go to the cloud, this is one of the few controllable options.
  - **Cross-language voice pipelines**: The vendor lists support for 49 languages, and you can pass an audio_url through an OpenAI-compatible API to get transcription back directly, which keeps integration cost into an existing pipeline relatively low.

- **Can you actually run it?**: Set your expectations first. The official model card's serving example is vLLM, dtype bfloat16, `--tensor-parallel-size 4`, meaning four GPUs. The optimized serving section states "approximately 2× the output-token throughput versus the baseline configuration on four H100 80 GB." The vendor **does not provide GGUF or consumer GPU quantization options, and no single-card threshold is stated**. Based on the examples given, this is a server-class self-hosting project, not a laptop one.

- **Buzz this episode**: HF trending score 75, 571 downloads, 75 likes. Compared to the previous model, this one's download count has actually started moving.

- **Versus comparable models**: The official model card's comparison table places it in the "2-stage" category and claims EVA-X 74.13, ahead of open-source peers UltraVox v0.6, Gemma 4-12B-IT, and Audex-30B-A3B. But the same table also shows τ-Voice task success rate still trailing closed-source S2S systems like Gemini-3.8 Live and GPT Realtime 2.1. These are all ServiceNow's own evaluation numbers.

- **How to get it**: The model card offers two paths. One is `vllm serve --model ServiceNow-AI/Apriel-Voice-27B-preview --trust-remote-code`, with an official v0.32.0 vLLM Docker image and a pointer to upstream vLLM PR #64683. The other is Transformers directly via `AutoModel.from_pretrained(..., trust_remote_code=True)` paired with AutoProcessor on 16 kHz audio.

- **Link**: https://huggingface.co/ServiceNow-AI/Apriel-Voice-27B-preview

### 3. RikiyaT/Qwen3.5-0.8B-weight-space-programming: no gradient descent, just asking a frontier LLM to edit the weights

- **What it is**: The model name says 0.8B while the HF parameter field shows 0.9B params; both refer to the same model, not two. Apache-2.0 licensed, safetensors, page tagged BF16/F32. Its origin story is unusual: the README says the author asked a frontier LLM (codename Astra) to trace the base model's thinking loops within a budget of 50 forward passes, then subtract from just 64 output columns of the layer 15 and layer 19 down_proj matrices, touching 131,072 weights total, or 0.016% of the model. The author stresses the entire process involved "no gradient descent and no training."

- **Where you'd use it**:
  - **Engineers who want to reproduce "modify a model without training it" on their own machine**: The README includes provenance-linked logs and apply_edit.py, so the whole pipeline is laid out for you to re-run. A topic with this low a download cost that can still eat a whole weekend is rare.
  - **People teaching small-model inference or giving internal talks**: The README table says the base version loops until the token limit on nearly every math problem (93.0%), while after the edit MATH-500 rises from 7.9% to 48.1% and GSM8K from 13.8% to 67.1%. There's also an official Space demo where you can watch the base version spin in circles while the edited version solves the problem, which makes for a very strong live contrast. All of these numbers come from the README table, not third-party verification.
  - **Researchers working on model verifiability and supply-chain trust**: The README says the edit is published as a bit-exact diff, the rebuild script sha256-verifies every tensor, and it lists exactly which "only two tensors differ" from Qwen/Qwen3.5-0.8B along with their hashes. If you've been wondering how to prove exactly which parts of a weight file were modified, this is a ready-made reference implementation.

- **Can you actually run it?**: The official model card **does not state minimum hardware requirements**. The page shows 0.9B params and BF16/F32, and the README usage example loads it in vLLM with `dtype="bfloat16"` and `max_model_len=34816`. Roughly speaking, 0.9B scale weights are a light load for a typical dev machine, though the page tags it BF16/F32 mixed, so actual file size may exceed a pure BF16 calculation. That's our estimate, not an official figure. Separately, the README mentions an independent reproduction that took 51 minutes on a single NVIDIA H200 to run 2,000 generations, which is an evaluation-scale number, not a single-inference threshold.

- **Buzz this episode**: HF trending score 91, 1.3k downloads, 92 likes. There are 2 quantized/repackaged variants from the same family on Hugging Face this episode, meaning someone has already ported it to other runtimes.

- **Versus comparable models**: Its control group is literally the original Qwen/Qwen3.5-0.8B. The README's numbers show the two differ in only two tensors, yet IFEval rises from 21.0 to 62.3 and the rate of hitting the token limit drops from 93.0% to 33.1%. Unlike typical small models tuned via LoRA or SFT, the author stresses this is pure weight editing. Note that the README says the blog post's evaluation was on a text-only export, while this checkpoint is the full multimodal version, with the language model tensors hash-verified as identical.

- **How to get it**: The README usage section goes straight to vLLM: `LLM(model="RikiyaT/Qwen3.5-0.8B-weight-space-programming", dtype="bfloat16", max_model_len=34816)`, paired with the `enable_thinking=True` chat template and temperature 0.6 / top_p 0.95 / top_k 20 / presence_penalty 1.5. The HF page also lists one-click commands for transformers, vLLM, SGLang, and Docker Model Runner, plus community quantizations that work with llama.cpp/Ollama/LM Studio.

- **Link**: https://huggingface.co/RikiyaT/Qwen3.5-0.8B-weight-space-programming

## Closing

This episode's three models happen to line up as a spectrum from heavy to light. Ripple-65B-Preview claims its dense architecture can run on consumer hardware like the DGX Spark, but the vendor never states a single-card threshold. Apriel-Voice-27B-preview's official serving example goes straight to four GPUs, with the optimized section using four H100 80GBs, clearly positioning it as data-center-grade self-hosting. And Qwen3.5-0.8B-weight-space-programming is the only one of the three you could pull down and play with tonight: 1.3k downloads plus two community repackages, so if you want to get hands-on, starting there gives you the best return. One more reminder: official release dates for all three are unsourced, and the trending scores and download counts above come from HF pages captured for this episode, so they don't reflect live status right now. Same time tomorrow, see you at the on-prem AI lab.

---

## About this article and its author

Originally published on [Mark Ku's Tech Notes](https://blog.markkulab.net/en/local-ai-lab/local-ai-lab-2026-11-04)

License: [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) — when reusing or quoting, credit the author and link back to the original

### About the author

**[Mark Ku](https://blog.markkulab.net/en/author/mark-ku)** — Software engineer

- 10+ years as a software engineer
- Built North-American e-commerce and AI SaaS subscription billing

### Free tools built by the author

All of these are free to use:

- [Free PDF Sign Tool](https://blog.markkulab.net/en/tools/pdf-sign): Online PDF sign tool — draw, type, or upload a signature, then drag, resize, and download. Everything runs in your browser; nothing is uploaded.
- [VS Code Refactory](https://blog.markkulab.net/en/tools/refactory): Refactory is a VS Code refactoring extension: 34 actions plus a 37-rule code-smell inspection layer with a Code Health dashboard, across 18 languages, backed by 534 tests. It learns your repo's conventions: where interfaces live, where DI is registered, whether 'use client' belongs. It ranks files by git churn × complexity so you know what to fix first, and hands any smell to the Claude Code already on your machine. Free to use, and your source never leaves your computer.
- [DB-Kit Database Manager](https://blog.markkulab.net/en/tools/db-kit): DB-Kit is a lightweight, cross-platform database manager built with Tauri + Rust + React. Manage MySQL, MariaDB, PostgreSQL, SQL Server, Oracle, SQLite, MongoDB, Redis, Kafka, Elasticsearch and RabbitMQ from one consistent interface: passwords encrypted in the OS keychain, SSH tunnels, full CRUD, a visual query builder, stacked multi-statement result sets, cross-connection data transfer, schema & data compare with sync SQL and schema snapshots, Excel / CSV import & export, visualized execution plans, ER diagrams, scheduled backups, SQL stress testing with p50–p99 latency percentiles, a 15-rule SQL review engine, Review & Run (AI review plus per-statement backups and an auto-generated rollback script), Kafka message browsing with monitoring & alerts, an SSH terminal with SFTP / FTP, Docker / Kubernetes, remote desktop (RDP / VNC / RustDesk), file / folder compare, stored procedure integration tests, six UI languages, a built-in AI assistant (a local CLI or any Anthropic / OpenAI-compatible API; natural-language SQL, AI review and tuning advice) and the dbk CLI. Free and open source (MIT), with installers for Windows, macOS and Linux.
- [VS Code Super Mermaid](https://blog.markkulab.net/en/tools/super-mermaid): Super Mermaid is a VS Code extension for beautiful Mermaid diagrams out of the box: auto-colored live preview, mouse pan & zoom, high-res PNG / SVG export, 21 templates and multiple themes. Free and open source (MIT).
- [React Super Mermaid](https://blog.markkulab.net/en/tools/react-super-mermaid): react-super-mermaid is an open-source React component library: render beautiful Mermaid diagrams with a single <MermaidViewer>, with built-in colorful / sketch themes, pan & zoom, in-diagram search, and high-res SVG / PNG export. Lightweight, SSR-safe, fully typed. Free and open source (MIT).
- [Jira / Confluence Super Mermaid](https://blog.markkulab.net/en/tools/jira-super-mermaid): An Atlassian Forge app: write Mermaid syntax directly inside a Jira issue or a Confluence page and get flowcharts, sequence diagrams, state machines and Gantt charts. 11 diagram types, SVG / PNG export, light and dark themes, full CJK support. Runs on Atlassian: your diagrams live in your own site and the app calls no third-party service. Free, coming soon to the Atlassian Marketplace.
- [Mermaid Live Preview](https://blog.markkulab.net/en/tools/mermaid-preview): Write Mermaid in your browser, see it render instantly, and share the whole diagram as a single link. No sign-up, nothing uploaded to a server, and mermaid.live share links work as-is.
- [React Intl Phone Number](https://blog.markkulab.net/en/tools/react-intl-phone-number): react-intl-phone-number is an open-source React component: framework-agnostic and antd-free, with E.164 in/out, a searchable flag / country-code dropdown, configurable validation levels (strict / mobile-strict / loose), themeable CSS, and i18n — phone logic powered by google-libphonenumber. Lightweight and fully typed. Free and open source (MIT).
- [Uptime Kuma Cluster](https://blog.markkulab.net/en/tools/uptime-kuma-cluster): Turn single-node Uptime Kuma into a highly available cluster: OpenResty + Lua smart load balancing, shared MariaDB state, health checks and automatic failover, plus cluster-management REST APIs. One Docker Compose command to start. Free and open source (MIT).
- [AI Podcast Cut](https://blog.markkulab.net/en/tools/ai-podcast-cut): Drop in a recording and it removes fillers and stutters, levels loudness segment by segment, and sends a second agent to review every cut. Cut points snap to word boundaries and zero crossings, every splice gets a fade, and sentence-end breaths are preserved. Desktop app for Windows, macOS and Linux. MIT licensed; the Windows installer bundles ffmpeg.
- [AI Video Cut](https://blog.markkulab.net/en/tools/ai-video-cut): Pick something in your video: type it (face, license plate, phone screen, logo), click points or drag a box, or let Claude Code / Codex pick it. AI Video Cut tracks it frame by frame, then mosaics or blurs it, recolors it, pins stickers and text to it, swaps a screen or poster for your own image or video, or removes it using background that other frames actually captured. Plus sequence editing, local captions and an AI assistant. Free, open-source (MIT) Windows desktop app (macOS / Linux experimental) that runs locally, nothing uploaded.
- [open-pos restaurant POS](https://blog.markkulab.net/en/tools/open-pos): One computer and one receipt printer is enough to open the shop. Your data lives on your own disk, no subscription, no lock-in, MIT licensed. Money is integer New Taiwan dollars with tax split by the statutory formula, so sales + tax always equals the total. Printing goes straight over ESC/POS on TCP 9100, with no vendor driver. Tauri + Rust + SQLite desktop app, v1.0 in development.
- [.NET e-commerce API template](https://blog.markkulab.net/en/tools/dotnet-ecommerce-api): An e-commerce backend API template built on .NET 10: Controller → Service → Repository layering, Autofac convention-based registration, SqlSugar for MySQL / MariaDB and SQL Server, Redis cache and message queue, Hangfire jobs, JWT auth, and an extensible payment provider module (ECPay, Newebpay). Apache 2.0.
- [Open Token Monitor team AI usage](https://blog.markkulab.net/en/tools/open-token-monitor): Collect token usage and equivalent cost from every employee's AI coding tools into a hub you host yourself: PostgreSQL, org roster, dashboards compared by company and department, and a reports API. Electron and Rust/Tauri clients upload every 30 minutes. MIT licensed, one Docker command.
- [ai-security-monitor defensive security](https://blog.markkulab.net/en/tools/ai-security-monitor): Your website is live. Who checks its certificates, security settings and known vulnerabilities? A self-hosted Security Hub brings website scans and host monitoring together, deduplicates findings and compares scans after fixes.

### Daily podcasts

- [Mark's Tech Insights — Daily AI News](https://blog.markkulab.net/en/category/tech-news): Daily curated AI and tech trends. Catch the latest developments via audio summaries — covering AI applications, software architecture, DevOps, and engineering practice. — RSS: https://blog.markkulab.net/feed.xml
- [AI股市蝦聊](https://blog.markkulab.net/en/category/ai-stock-chat): Every trading day, an AI-analyzed take on the Taiwan stock market, delivered as a two-host conversation covering the session and the next-day outlook. — RSS: https://blog.markkulab.net/ai-stock-chat/feed.xml
- [開源好物日報](https://blog.markkulab.net/en/category/open-source-weekly): A daily two-host pick of free open-source tools surfaced from real Hacker News, GitHub, and Reddit buzz — what pain they solve and the fastest way to get started. — RSS: https://blog.markkulab.net/open-source-weekly/feed.xml
- [AI 運動日報](https://blog.markkulab.net/en/category/sports-weekly): Two hosts talk NBA, MLB and world sport every day — scores, records and the stories behind them, from a Taiwanese fan perspective. — RSS: https://blog.markkulab.net/sports-weekly/feed.xml
- [AI 國際新聞快報](https://blog.markkulab.net/en/category/world-news): A daily two-host briefing that makes sense of the past 24 hours in world news — geopolitics, the global economy, conflict and security, disasters and climate — from a Taiwanese perspective, neutral and fully sourced. — RSS: https://blog.markkulab.net/world-news/feed.xml
- [地端 AI 實驗室](https://blog.markkulab.net/en/category/local-ai-lab): Every day, a two-host look at open-weight models and LoRAs you can actually run on your own machine: what they are for, whether your GPU can handle them, and how they compare — sourced from official model cards. — RSS: https://blog.markkulab.net/local-ai-lab/feed.xml
- [AI 行銷日報](https://blog.markkulab.net/en/category/ai-marketing): Every day, two hosts break down the latest in AI for marketing, advertising, SEO/GEO and content marketing: practical, never sponsored, every story sourced. — RSS: https://blog.markkulab.net/ai-marketing/feed.xml
- [AI 資安日報](https://blog.markkulab.net/en/category/ai-security): Every day, two hosts cover the latest where AI meets cybersecurity: LLM and AI agent vulnerabilities, prompt injection, model supply-chain risks and AI-powered attacks, from a defense-first angle with every story sourced. — RSS: https://blog.markkulab.net/ai-security/feed.xml

### Newsletter

[Subscribe to the newsletter](https://blog.markkulab.net/en/subscribe) — Be the first to know about new posts. No spam, unsubscribe anytime.