Opening

The most eye-popping number this week is Strata running a 125B-parameter model on a single 12GB gaming GPU, hitting 5.9k stars under an MIT license in under a month. Two other local-first approaches also surfaced the same week: Ledge.sh turns Markdown notes into a persistent terminal that can actually run commands, while Magnitude compiles kernels on the fly, right on your own machine. I'll tell you exactly how much of a speedup that gives over llama.cpp in a bit.

This Week's Open Source Picks

1. Ledge.sh: Turning Markdown Notes into a Persistent, Runnable Terminal

• What it is: Ledge.sh lets you take the Markdown notes you already write and turn code blocks directly into one-click executable commands. The whole note shares a single persistent shell, so variables, cd, and virtual environments all persist across multiple runs, unlike Jupyter where each cell runs in isolation. It's a great fit for turning the "copy-paste into terminal" playbooks you already have into one-click, rerunnable runbooks.
• Buzz this week: It landed 202 points and 88 comments on Hacker News, with discussion centered on "this is exactly the runbook tool I wanted." That said, the GitHub repo currently sits at just 278 stars, so the buzz is mostly driven by this HN thread.
• Standout features:
  • Built-in support for running Shell, Python, Node, Ruby, PHP, and TypeScript directly; SQL and Redis blocks can connect to your existing databases, and adding other interpreters is just one line in the settings
  • Adding one line, host:, to the frontmatter routes the entire note's execution over SSH to that machine. Once connected to a Ledge Server, closing your laptop won't interrupt a running block, and the mobile app lets you check back on progress
  • Comes with an MCP server; running claude mcp add ledge -- ledge mcp lets Claude Code read, write, and search your notes, and even the prompt blocks themselves are executable
• Quick start: On macOS, download .dmg; on Windows, run Ledge-Setup.exe (requires WSL); on Linux, extract and run ./installer. To keep it running persistently on a remote machine, run curl -fsSL https://ledge.sh/server.sh | sh on that machine. Once installed, open any note and hit execute on a code block, if the next block can still read a variable set by the previous one, that confirms the persistent shell is actually working. If you've connected a Ledge Server, the mobile app should show the block currently executing, confirming the server side is wired up too.
• How it compares: The biggest difference from Jupyter is that it's not a cell-based kernel but one persistent shell per note, and the file itself is just plain Markdown, no .ipynb-style diff hell. Compared to Runme (a similar tool that turns Markdown code blocks into executable runbooks, conceptually similar but without persistent SSH execution, a mobile client, or a built-in MCP server), Ledge adds remote persistence, mobile check-ins, and AI agent integration.
• Who it's for: Engineers sitting on a pile of "copy-paste into terminal" operational notes who want to turn their SOPs into one-click, rerunnable runbooks.
• Link: ledge.sh

2. Strata: Running a 125B Model on a 12GB Gaming GPU

• What it is: Strata tackles the age-old problem of "I want to run a big model but my GPU isn't big enough." It dynamically distributes model weights across VRAM, system RAM, and SSD (dynamic expert routing), which is different from traditional layer-by-layer offloading, so even a 12GB VRAM card can fit a 125B-parameter model. It also exposes both OpenAI- and Anthropic-compatible endpoints, so your existing tooling can plug into a local model with barely any changes.
• Buzz this week: It shot up to 5.9k GitHub stars (5,931 exactly) within 30 days, under an MIT license, with C++ as the primary language.
• Standout features:
  • 94 tokens/s output, 2,650 tokens/s input, and a 70GB download: these three numbers all describe the same measured result, running the 125B-parameter Qwen3.8-Flash-Next on just 12GB VRAM with Q2_0 quantization. The 70GB figure is the size of the weights themselves, not something that needs to fully fit in the GPU; that's made possible by the VRAM/RAM/SSD distribution mentioned above. Switching to IQ3_S quantization drops throughput to 53 tokens/s, higher quantization levels consume more resources and cost you speed
  • Speculative decoding, "guess then verify," delivers a 1.6x to 1.8x speedup
  • The install script auto-detects your hardware, picks an appropriately sized model, downloads the weights, and spins up the service directly, complete with a browser UI for chatting, monitoring dashboards, and tuning settings. It supports NVIDIA RTX 20/30/40/50 series and AMD RX 7900 XT/XTX, 7800 XT, and 9070+ (12GB VRAM or more), with optional image input support
• Quick start: On Windows, just double-click START-HERE.bat; on Linux, run ./setup.sh, and the installer handles the rest automatically. Make sure you have 32GB of RAM ready (64GB recommended) and roughly 80GB of free SSD space. Once installed, open http://127.0.0.1:8080 in your browser, or just hit the OpenAI-compatible endpoint directly with curl http://127.0.0.1:8080/v1/models; seeing the model list respond confirms the service is actually up.
• How it compares: Ollama and LM Studio work by picking a model that fits within your VRAM; Strata flips that around, betting on "spread the whole machine's resources thin to force a big model to run," at the cost of a 70GB download and Q2-level quantization. The README is upfront that it's built on top of llama.cpp/ggml, and that AMD throughput runs roughly 35% to 65% lower than an equivalent NVIDIA setup.
• Who it's for: Anyone with just a single 12GB consumer GPU who wants to get hands-on with a 125B-class model locally, and is willing to trade some quality from Q2 quantization for the simple fact that it actually runs.
• Link: niko1221/Strata

3. Magnitude: Compiling Kernels On the Fly, One Click to Wire Up Your Coding Agent Locally

• What it is: Magnitude takes a different approach to local inference. Instead of shipping a pre-built kernel tuned for a broad category of hardware like llama.cpp does, it compiles and tunes on the spot, right on your own device, then lets you point coding agents like Claude Code, Codex, and Cline at your local model with a single click.
• Buzz this week: This was a Launch HN post, pulling in 194 points and 97 comments, with 6,263 GitHub stars accumulated under an Apache-2.0 license, primarily written in Rust.
• Standout features:
  • Official benchmarks against llama.cpp: 92% faster decoding on Metal, 19% faster on CUDA; prefill is 9% faster on Metal and 23% faster on CUDA. That said, the official numbers don't disclose the test hardware or the llama.cpp version used, so take them as a rough guide and benchmark on your own machine for the real answer
  • Memory management tuned specifically for agent workloads: each agent uses 27% less memory, memory is released once the agent stops, and a shared prefix cache across sessions prevents them from slowing each other down
  • The Connections page lets you one-click connect Pi, OpenCode, Hermes, Codex, Claude Code, and Cline, across Apple Silicon, NVIDIA, AMD, and CPU-only setups. Once the model is downloaded, the entire workflow requires no network connection
• Quick start: Install the desktop app (CLI included) from magnitude.dev/download, pick a model to download on the Discover page, then go to the Connections page and one-click wire up your coding agent. Once connected, go back to Claude Code or Cline and ask it something, if it responds normally even with your network unplugged, that confirms it's actually running on the local model instead of the cloud.
• How it compares: llama.cpp, Ollama, and LM Studio all ship pre-compiled kernels tuned for a wide range of hardware out of the box. Magnitude instead tunes per device on the spot, adding per-agent memory reclamation and a shared prefix cache across sessions. Compared to Strata from the same week: Strata is a Windows one-click installer that forces a single large model to run on limited hardware, while Magnitude leans toward Mac and multi-platform support, with a focus on directly plugging into coding agents. The README is also upfront that it's "hand-optimized for popular open-weight model families," not a universal solution for every model.
• Who it's for: Engineers already using coding agents like Claude Code and Cline who want to connect them to a local model, and who care about the performance of on-device tuning itself.
• Link: magnitudedev/magnitude

Closing

These three tools are really three different answers to the same question: can local setups actually run something "genuinely big"? Ledge.sh tackles the daily grind of repetitive operations, while Strata and Magnitude represent two distinct approaches to local inference, one betting on distributing hardware resources, the other on on-the-fly compilation. If you can only try one first, I'd start with Strata's 12GB threshold, since that's a GPU most people already have sitting in their machine. I'll dig up more great finds next week, see you then.