Mark Ku's Blog
Open in ChatGPTOpen in Claude

Mini-AGI enables continual learning on 8GB VRAM by using disk-based paging for MoE experts, while laya classifies data locally in 33ms without text generation using 421M parameters, and Drop sandboxes coding agents via Linux namespaces without requiring Docker or OS changes.

Podcast ConversationAI dialogue version of this article · Mandarin audio

This show is written by AI and voiced with synthetic speech. The hosts are virtual characters.

Opening

Mini-AGI scored 276 points on Hacker News this week, a solo project that trains a continually-learning language model that doesn't forget, using just an 8GB VRAM laptop. Also featured: laya, which shot up to 19k stars in 30 days and makes decisions in 33 milliseconds without generating text, plus Drop, which lets you run coding agents on autopilot without losing sleep. How exactly does Drop lock down permissions at the OS level? We'll get to that.

This Week's Open Source Picks

1. laya: A Local Decision Model That Doesn't Generate Text, 33ms per Query

  • What it is: laya is a small model that runs locally, and instead of chatting, it makes decisions for your programs, things like classification, scoring, yes/no questions, spitting out structured answers directly via a single forward pass, without the LLM-style approach of generating a wall of text you then have to parse yourself. It's built for engineers who want routing, classification, or gating logic in their pipeline without calling out to a cloud API.
  • This week's buzz: 19.0k ★ on GitHub, a project that shot up in the last 30 days, licensed under Apache-2.0.
  • Highlights: Three typed decision modes (choice classification with confidence scores, score on a 1-5 scale, and calibrated yes/no probabilities); 32.8-39.5ms per query on a Tesla T4, averaging 7.2ms per query in batch mode; the English version is 421M parameters and the multilingual version is 322M, supporting 100+ languages, with 45 out of 51 tested languages hitting usable quality.
  • Quick start: pip install laya, use Router(preload=True) to auto-select a checkpoint, then feed state and questions into router.predict() to get your answer. Mac users can instead install the sister project laya-mlx, which runs on Apple Silicon's native MLX runtime without needing PyTorch.
  • How it compares: Its rival is TypeSafe's paid Jev API. After fine-tuning, laya hits 0.766 accuracy versus Jev's 0.727, and its 32.8ms latency blows past Jev's 236-276ms. But Jev's selling point is zero-shot use with no fine-tuning required, while laya's base checkpoint only scores 0.362 (barely above the random baseline of 0.318), meaning you need to prepare your own data and fine-tune before it's actually useful, which is also the most-questioned point in the HN discussion thread.
  • Who it's for: Backend engineers who want local classification, routing, and gating without being locked into a cloud API.
  • Link: https://github.com/NandhaKishorM/laya

2. Drop: Let Your Coding Agent Go Full Autopilot Without the Guilt

  • What it is: Drop is a sandboxing tool built to solve the anxiety of letting a coding agent run --dangerously-skip-permissions tasks unattended. It shifts permission control from agent self-restraint to OS-level enforcement, rootless, with no changes needed to any of your existing tools.
  • This week's buzz: 175 points on Hacker News, 59 comments, licensed under Apache-2.0.
  • Highlights: Isolation via Linux namespaces (user, mount, PID, IPC, cgroup, network), dropping all capabilities before startup, fully rootless throughout; optionally stack gVisor on top so processes never touch the host kernel; the current working directory is writable by default while .git is read-only, networking runs through pasta and blocks connections to services on localhost by default; unlike Docker/Podman, there's no need to build an image first, it works directly with your existing distro, and anything you've already installed is still there inside the sandbox.
  • Quick start: sudo apt-get install passt (⟦dnf⟧ on Fedora, ⟦pacman⟧ on Arch), download the drop binary and install it to ~/.local/bin, then from your project directory run drop init followed by drop run to enter the sandbox.
  • How it compares: The README itself uses Docker/Podman as the comparison point: containers require building an image first, while Drop borrows your local distro directly, skipping the whole container setup process, and you can further harden isolation with gVisor. Compared to "asking the agent to behave itself" permission prompts, the difference is that this is OS-level enforcement: the sandboxed process doesn't even need to know Drop exists.
  • Who it's for: Engineers who run agents on autopilot every day and worry about them messing with the system.
  • Link: https://droprun.sh/

3. Mini-AGI: Training a Model That Doesn't Forget on an 8GB Laptop

  • What it is: Mini-AGI is a solo project focused on byte-level continual learning language models. The point isn't to recreate a Transformer textbook example, it's to prove that a model can keep reading new material without forgetting the old, while treating MoE experts as ordinary files on disk that get paged into VRAM as needed, small enough to fit on an 8GB VRAM RTX 3070 laptop GPU.
  • This week's buzz: 276 points and 75 comments on Hacker News; 665 stars on GitHub, licensed under MIT.
  • Highlights: The vocabulary is just 256 byte values plus 9 structural tokens, no tokenizer configuration needed, the file itself is the training data; a halting mechanism scores each character as it goes, letting simple characters exit early to save compute; roughly 170 experts normally sit on disk with only 32 resident in VRAM at a time, with a RAM cache layer in between that predictively moves data as needed. The most counterintuitive finding: setting the learning rate of the shared trunk (embedding, attention, router, halting head) to 0.1x that of the experts cut forgetting from +2.23 nats down to +0.0067 nats, preserving 99.84% of progress.
  • Quick start: After git clone, run pip install torch numpy pyyaml matplotlib flask, pull test corpus with python3 -m corpora all --limit 5000, kick off training with python3 train.py read data/train --save --weights-dir weights, and launch the web interactive UI with python3 serve.py --port 8080. Requirements: an 8GB VRAM GPU plus Python 3.10+.
  • How it compares: Unlike nanoGPT-style tutorial projects that recreate GPT at home, Mini-AGI's focus is continual learning without forgetting plus MoE disk paging, so there's no fixed checkpoint to download; it's designed for you to keep feeding it new data continuously.
  • Who it's for: Engineers who want to experiment with continual learning on their own GPU and are interested in MoE memory engineering.
  • Link: https://github.com/volotat/mini-AGI/

Closing

All three picks this week are really about the same thing: keeping compute on your own machine, whether that's laya's millisecond-level decisions, Drop's OS-level isolation, or Mini-AGI's disk-paged MoE. If I had to pick just one to try right now, I'd go with Drop, it's the fastest to set up and hits closest to the pain point most of us face running agents on autopilot every day. See you next week for another round of open source picks.

Author

Mark Ku

10 年以上的軟體工程師,做過北美電商與 AI SaaS 訂閱收費系統。現在經營貳陸資訊有限公司(www.226network.com),幫小公司做系統、網站、LINE BOT 與 AI 自動化,也在這裡分享開發筆記與開源工具。Read More

Found this useful?

The author's free tools, daily podcasts and newsletter are all here.

Mark Ku · This article is licensed under CC BY 4.0. Credit the author and link back to the original when reusing it.

Comments

Subscribe to Newsletter

Subscribe to get new posts delivered instantly — never miss a tech share.

By submitting, you agree to receive emails. You can anytime.

Popular Posts

View all
Mark Ku
··643

Oracle Cloud Always Free Tier: Linux Host and Static IP for a $0 Cloud Solution

Oracle Cloud Always Free Tier: Linux Host and Static IP for a $0 Cloud Solution
Mark Ku
··581

Say Goodbye to Postman's Fee Trap! A Hands-on Guide to Bruno, the Open-Source Git-Native API Testing Powerhouse.

Say Goodbye to Postman's Fee Trap! A Hands-on Guide to Bruno, the Open-Source Git-Native API Testing Powerhouse.
Mark Ku
··314

A Free, Open-Source, Notion-like Knowledge Base — A Complete Guide to Deploying and Backing Up Outline Wiki

A Free, Open-Source, Notion-like Knowledge Base — A Complete Guide to Deploying and Backing Up Outline Wiki
Mark Ku
··234

Training Your Own AI Voice: Hardware Requirements, Open-Source Model Comparison, and LoRA Fine-Tuning

Training Your Own AI Voice: Hardware Requirements, Open-Source Model Comparison, and LoRA Fine-Tuning
Mark Ku
··230

Setting Up Samba on Ubuntu to Share Folders with Windows 11

Setting Up Samba on Ubuntu to Share Folders with Windows 11
Mark Ku
··228

Building an Efficient API Management Platform: Deploying Kong Gateway from Scratch - Part 1

Building an Efficient API Management Platform: Deploying Kong Gateway from Scratch - Part 1
🛠️ 8GB筆電就能訓,還不會忘記?Mini-AGI靠硬碟分頁做到了|開源好物週報 - Mark Ku's Tech Notes