---
title: "🛠️ Save 80% Memory on LLM Training! Unsloth Lets You Say Goodbye to VRAM Explosion Nightmares: Open Source Daily"
description: "Always running out of VRAM when training LLMs? Unsloth can save you 80% memory and boost training speed by 5x. In this issue of Open Source Daily, besides this fine-tuning powerhouse, we also feature LiteLLM, which seamlessly connects hundreds of LLMs with a single standard API, and a god-tier AI assistant that edits code directly in your terminal and automatically commits to Git, we'll tell you who it is in just a moment."
canonical_url: "https://blog.markkulab.net/en/open-source-weekly/open-source-weekly-2026-10-26"
author: "Mark Ku"
author_url: "https://blog.markkulab.net/en/author/mark-ku"
site: "Mark Ku's Tech Notes"
date_published: "2026-10-26T02:00:00.000Z"
category: "Open Source Daily Finds"
tags: ["open-source", "podcast", "開源工具", "開源", "unsloth", "litellm", "aider", "AI工具", "開發效率", "開源好物日報"]
language: "en"
license: "CC BY 4.0"
license_url: "https://creativecommons.org/licenses/by/4.0/"
attribution: "when reusing or quoting, credit the author and link back to the original"
---

# 🛠️ Save 80% Memory on LLM Training! Unsloth Lets You Say Goodbye to VRAM Explosion Nightmares: Open Source Daily

## Introduction
Always running out of VRAM when training Large Language Models? Unsloth can save you 80% of your memory and boost training speeds by up to 5x. In this issue of Open Source Daily, besides this fine-tuning powerhouse, we will also introduce LiteLLM, which seamlessly connects hundreds of LLMs using a single standard API, and a god-tier AI assistant that edits your code directly in the terminal and automatically commits to Git. We will reveal who it is in just a moment.

## Today's Open Source Picks

### 1. Unsloth: An Ultra-Fast, Low-Memory Optimization Framework Designed for LLM Fine-Tuning
- **What it is**: It solves the painful issue of running out of VRAM (OOM) when fine-tuning large models like Llama 3 and Mistral locally. Designed specifically for AI engineers, it allows you to easily run fine-tuning even on consumer-grade GPUs, such as the RTX 3090 or 4090.
- **Recent Buzz**: With over 22,000 stars on GitHub, it is widely recognized as a performance dark horse in the open-source AI fine-tuning community.
- **Key Features**:
  - **Extreme Memory Optimization**: By replacing PyTorch's default Autograd operations with custom hand-written Triton kernels, it reduces memory usage by 80% without any loss in precision.
  - **Multiplied Training Speed**: Benchmarks show that fine-tuning Llama 3 on a single GPU is 2 to 5 times faster than the traditional Hugging Face + PEFT setup.
  - **Seamless Ecosystem Integration**: It natively supports Hugging Face, TRL, and PyTorch. Once fine-tuning is complete, you can directly export to GGUF, 16-bit, or LoRA weights.
- **Quick Start**:
  In a Linux environment equipped with CUDA 12.1 and PyTorch 2.4, run the following command to install:
  ```bash
  pip install "unsloth[cu121-torch240] @ git+https://github.com/unslothai/unsloth.git"
  ```
  Then, load the optimized model in Python with just a few lines of code:
  ```python
  from unsloth import FastLanguageModel
  model, tokenizer = FastLanguageModel.from_pretrained(
      model_name = "unsloth/llama-3-8b-Instruct",
      max_seq_length = 2048,
      load_in_4bit = True,
  )
  ```
- **Comparison**: Compared to traditional fine-tuning with Hugging Face `peft` or native PyTorch, which generate a massive amount of redundant temporary data during backpropagation, `unsloth` rewrites the underlying kernels. This not only delivers superior speed but also solves the hardware bottleneck of frequent OOM errors.
- **The Catch**: Currently, it only supports NVIDIA GPUs (with Ampere, Ada, or Hopper architectures yielding the best results). Additionally, for highly obscure custom model architectures, manual adaptation is required, meaning you cannot enjoy out-of-the-box acceleration.
- **Who it's for**: AI developers who want to quickly fine-tune open-source LLMs on a limited budget or a single consumer-grade GPU.
- **Link**: [https://github.com/unslothai/unsloth](https://github.com/unslothai/unsloth)

---

### 2. LiteLLM: A Lightweight Middleware That Unifies Hundreds of LLM APIs into the OpenAI Standard Format
- **What it is**: It solves the pain point of having to integrate OpenAI, Anthropic, Gemini, and even local Ollama instances simultaneously when developing AI applications. Built for backend and AI engineers, it eliminates the need to write tedious integration logic for every single API provider.
- **Recent Buzz**: With over 18,000 stars on GitHub, it is an essential tool for multi-model deployment in enterprise-grade AI projects.
- **Key Features**:
  - **Unified API Format**: Whether the underlying call is to Claude 3.5 or GPT-4o, you can consistently use the OpenAI standard format with `completion(model="...", messages=...)` in your code.
  - **Enterprise-Grade Proxy**: Featuring built-in load balancing, automatic retries (fallbacks), and budget controls, it can be deployed directly as a standalone proxy server.
  - **Real-Time Cost Tracking**: It automatically calculates token consumption and costs for each API call, and supports multi-tenant key management.
- **Quick Start**:
  After installing via pip, start the proxy service directly in your terminal:
  ```bash
  pip install litellm[proxy]
  export ANTHROPIC_API_KEY="your-api-key-here"
  litellm --model anthropic/claude-3-5-sonnet-20241022
  ```
  Now, you can send standard OpenAI-formatted requests directly to `http://0.0.0.0:4000/v1/chat/completions`.
- **Comparison**: Compared to all-in-one orchestration frameworks like LangChain, which have complex concepts, `litellm` focuses solely on "API translation and routing." It is extremely lightweight, has zero learning curve, and won't hijack your codebase architecture.
- **The Catch**: Although it supports hundreds of models, translation can occasionally fail for certain non-mainstream models with specialized parameters, such as specific alignment arguments, requiring you to fall back on the original API documentation.
- **Who it's for**: Development teams that need to quickly switch between multiple model sources or build a unified internal AI gateway.
- **Link**: [https://github.com/BerriAI/litellm](https://github.com/BerriAI/litellm)

---

### 3. Aider: A God-Tier AI Pair Programmer in Your Terminal
- **What it is**: It solves the inefficiency of constantly copying and pasting code between your browser and IDE when coding with ChatGPT. It allows engineers to command the AI to modify local files directly in the terminal using voice or text.
- **Recent Buzz**: Boasting over 25,000 stars on GitHub, it frequently ranks at the top of developer tool discussions on Hacker News.
- **Key Features**:
  - **Directly Modify Local Code**: Instead of just giving suggestions, Aider directly reads your Git repository, analyzes the code structure, and precisely modifies multiple related files.
  - **Automatic Git Commits**: Once modifications are complete and pass tests, Aider automatically writes a standard-compliant commit message and commits the code for you.
  - **Powerful Git Repository Awareness**: Utilizing "Repository Map" technology, it automatically sends a map of your project's architecture to the LLM, ensuring the AI never loses context even when working on large projects.
- **Quick Start**:
  Installation is incredibly simple. Just navigate to your Git project directory and run:
  ```bash
  pip install aider-chat
  export ANTHROPIC_API_KEY="your-key-here"
  aider
  ```
  Then, type directly in the terminal: `幫我寫一個快取裝飾器，並加到 utils.py 裡，然後幫我寫測試。`
- **Comparison**: Compared to GitHub Copilot, which only offers single-line or single-file completions, `aider` is a true "agentic" tool. It can refactor and debug across multiple files while keeping you in full control of your terminal and Git workflow.
- **The Catch**: For legacy, massive codebases with huge contexts, if you don't configure `.aiderignore` properly, a single conversation can consume a terrifying amount of API token costs.
- **Who it's for**: Terminal enthusiasts and senior engineers who pursue ultimate development efficiency and hate manual copying and pasting.
- **Link**: [https://github.com/paul-gauthier/aider](https://github.com/paul-gauthier/aider)

## Wrapping Up
As we can see from the tools shared today, open-source AI tools are rapidly moving from "web chatboxes" into engineers' "terminals and core workflows." Whether it is optimizing low-level training, unifying API gateways, or directly refactoring code for you, the efficiency gains are multiplied several-fold. We hope these tools help you save some overtime. See you tomorrow!

---

## About this article and its author

Originally published on [Mark Ku's Tech Notes](https://blog.markkulab.net/en/open-source-weekly/open-source-weekly-2026-10-26)

License: [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) — when reusing or quoting, credit the author and link back to the original

### About the author

**[Mark Ku](https://blog.markkulab.net/en/author/mark-ku)** — Software engineer

- 10+ years as a software engineer
- Built North-American e-commerce and AI SaaS subscription billing

### Free tools built by the author

All of these are free to use:

- [Free PDF Sign Tool](https://blog.markkulab.net/en/tools/pdf-sign): Online PDF sign tool — draw, type, or upload a signature, then drag, resize, and download. Everything runs in your browser; nothing is uploaded.
- [VS Code Refactory](https://blog.markkulab.net/en/tools/refactory): Refactory is a VS Code refactoring extension: 34 actions plus a 37-rule code-smell inspection layer with a Code Health dashboard, across 18 languages, backed by 534 tests. It learns your repo's conventions: where interfaces live, where DI is registered, whether 'use client' belongs. It ranks files by git churn × complexity so you know what to fix first, and hands any smell to the Claude Code already on your machine. Free to use, and your source never leaves your computer.
- [DB-Kit Database Manager](https://blog.markkulab.net/en/tools/db-kit): DB-Kit is a lightweight, cross-platform database manager built with Tauri + Rust + React. Manage MySQL, MariaDB, PostgreSQL, SQL Server, Oracle, SQLite, MongoDB, Redis, Kafka, Elasticsearch and RabbitMQ from one consistent interface: passwords encrypted in the OS keychain, SSH tunnels, full CRUD, a visual query builder, stacked multi-statement result sets, cross-connection data transfer, schema & data compare with sync SQL and schema snapshots, Excel / CSV import & export, visualized execution plans, ER diagrams, scheduled backups, SQL stress testing with p50–p99 latency percentiles, a 15-rule SQL review engine, Review & Run (AI review plus per-statement backups and an auto-generated rollback script), Kafka message browsing with monitoring & alerts, an SSH terminal with SFTP / FTP, Docker / Kubernetes, remote desktop (RDP / VNC / RustDesk), file / folder compare, stored procedure integration tests, six UI languages, a built-in AI assistant (a local CLI or any Anthropic / OpenAI-compatible API; natural-language SQL, AI review and tuning advice) and the dbk CLI. Free and open source (MIT), with installers for Windows, macOS and Linux.
- [VS Code Super Mermaid](https://blog.markkulab.net/en/tools/super-mermaid): Super Mermaid is a VS Code extension for beautiful Mermaid diagrams out of the box: auto-colored live preview, mouse pan & zoom, high-res PNG / SVG export, 21 templates and multiple themes. Free and open source (MIT).
- [React Super Mermaid](https://blog.markkulab.net/en/tools/react-super-mermaid): react-super-mermaid is an open-source React component library: render beautiful Mermaid diagrams with a single <MermaidViewer>, with built-in colorful / sketch themes, pan & zoom, in-diagram search, and high-res SVG / PNG export. Lightweight, SSR-safe, fully typed. Free and open source (MIT).
- [Jira / Confluence Super Mermaid](https://blog.markkulab.net/en/tools/jira-super-mermaid): An Atlassian Forge app: write Mermaid syntax directly inside a Jira issue or a Confluence page and get flowcharts, sequence diagrams, state machines and Gantt charts. 11 diagram types, SVG / PNG export, light and dark themes, full CJK support. Runs on Atlassian: your diagrams live in your own site and the app calls no third-party service. Free, coming soon to the Atlassian Marketplace.
- [Mermaid Live Preview](https://blog.markkulab.net/en/tools/mermaid-preview): Write Mermaid in your browser, see it render instantly, and share the whole diagram as a single link. No sign-up, nothing uploaded to a server, and mermaid.live share links work as-is.
- [React Intl Phone Number](https://blog.markkulab.net/en/tools/react-intl-phone-number): react-intl-phone-number is an open-source React component: framework-agnostic and antd-free, with E.164 in/out, a searchable flag / country-code dropdown, configurable validation levels (strict / mobile-strict / loose), themeable CSS, and i18n — phone logic powered by google-libphonenumber. Lightweight and fully typed. Free and open source (MIT).
- [Uptime Kuma Cluster](https://blog.markkulab.net/en/tools/uptime-kuma-cluster): Turn single-node Uptime Kuma into a highly available cluster: OpenResty + Lua smart load balancing, shared MariaDB state, health checks and automatic failover, plus cluster-management REST APIs. One Docker Compose command to start. Free and open source (MIT).
- [AI Podcast Cut](https://blog.markkulab.net/en/tools/ai-podcast-cut): Drop in a recording and it removes fillers and stutters, levels loudness segment by segment, and sends a second agent to review every cut. Cut points snap to word boundaries and zero crossings, every splice gets a fade, and sentence-end breaths are preserved. Desktop app for Windows, macOS and Linux. MIT licensed; the Windows installer bundles ffmpeg.
- [AI Video Cut](https://blog.markkulab.net/en/tools/ai-video-cut): Pick something in your video: type it (face, license plate, phone screen, logo), click points or drag a box, or let Claude Code / Codex pick it. AI Video Cut tracks it frame by frame, then mosaics or blurs it, recolors it, pins stickers and text to it, swaps a screen or poster for your own image or video, or removes it using background that other frames actually captured. Plus sequence editing, local captions and an AI assistant. Free, open-source (MIT) Windows desktop app (macOS / Linux experimental) that runs locally, nothing uploaded.
- [open-pos restaurant POS](https://blog.markkulab.net/en/tools/open-pos): One computer and one receipt printer is enough to open the shop. Your data lives on your own disk, no subscription, no lock-in, MIT licensed. Money is integer New Taiwan dollars with tax split by the statutory formula, so sales + tax always equals the total. Printing goes straight over ESC/POS on TCP 9100, with no vendor driver. Tauri + Rust + SQLite desktop app, v1.0 in development.
- [.NET e-commerce API template](https://blog.markkulab.net/en/tools/dotnet-ecommerce-api): An e-commerce backend API template built on .NET 10: Controller → Service → Repository layering, Autofac convention-based registration, SqlSugar for MySQL / MariaDB and SQL Server, Redis cache and message queue, Hangfire jobs, JWT auth, and an extensible payment provider module (ECPay, Newebpay). Apache 2.0.
- [Open Token Monitor team AI usage](https://blog.markkulab.net/en/tools/open-token-monitor): Collect token usage and equivalent cost from every employee's AI coding tools into a hub you host yourself: PostgreSQL, org roster, dashboards compared by company and department, and a reports API. Electron and Rust/Tauri clients upload every 30 minutes. MIT licensed, one Docker command.
- [ai-security-monitor defensive security](https://blog.markkulab.net/en/tools/ai-security-monitor): ai-security-monitor is a single Rust binary for your computers and servers. It finds unusual connections, rates them safe / unknown / suspicious / dangerous against 26 public threat-intel sources, and checks firewalls, open ports, vulnerabilities and 60 hardening rules. Security Hub shows the whole fleet, and Claude Code / Codex explain it in plain words. MIT, free, self-hosted.

### Daily podcasts

- [Mark's Tech Insights — Daily AI News](https://blog.markkulab.net/en/category/tech-news): Daily curated AI and tech trends. Catch the latest developments via audio summaries — covering AI applications, software architecture, DevOps, and engineering practice. — RSS: https://blog.markkulab.net/feed.xml
- [AI股市蝦聊](https://blog.markkulab.net/en/category/ai-stock-chat): Every trading day, an AI-analyzed take on the Taiwan stock market, delivered as a two-host conversation covering the session and the next-day outlook. — RSS: https://blog.markkulab.net/ai-stock-chat/feed.xml
- [開源好物日報](https://blog.markkulab.net/en/category/open-source-weekly): A daily two-host pick of free open-source tools surfaced from real Hacker News, GitHub, and Reddit buzz — what pain they solve and the fastest way to get started. — RSS: https://blog.markkulab.net/open-source-weekly/feed.xml
- [AI 運動日報](https://blog.markkulab.net/en/category/sports-weekly): Two hosts talk NBA, MLB and world sport every day — scores, records and the stories behind them, from a Taiwanese fan perspective. — RSS: https://blog.markkulab.net/sports-weekly/feed.xml
- [AI 國際新聞快報](https://blog.markkulab.net/en/category/world-news): A daily two-host briefing that makes sense of the past 24 hours in world news — geopolitics, the global economy, conflict and security, disasters and climate — from a Taiwanese perspective, neutral and fully sourced. — RSS: https://blog.markkulab.net/world-news/feed.xml
- [地端 AI 實驗室](https://blog.markkulab.net/en/category/local-ai-lab): Every day, a two-host look at open-weight models and LoRAs you can actually run on your own machine: what they are for, whether your GPU can handle them, and how they compare — sourced from official model cards. — RSS: https://blog.markkulab.net/local-ai-lab/feed.xml
- [AI 行銷日報](https://blog.markkulab.net/en/category/ai-marketing): Every day, two hosts break down the latest in AI for marketing, advertising, SEO/GEO and content marketing: practical, never sponsored, every story sourced. — RSS: https://blog.markkulab.net/ai-marketing/feed.xml
- [AI 資安日報](https://blog.markkulab.net/en/category/ai-security): Every day, two hosts cover the latest where AI meets cybersecurity: LLM and AI agent vulnerabilities, prompt injection, model supply-chain risks and AI-powered attacks, from a defense-first angle with every story sourced. — RSS: https://blog.markkulab.net/ai-security/feed.xml

### Newsletter

[Subscribe to the newsletter](https://blog.markkulab.net/en/subscribe) — Be the first to know about new posts. No spam, unsubscribe anytime.