---
title: "🧪 27B multimodal fits on a single 3090, 4B runs on a 4GB GPU: all three GGUFs this issue are free for commercial use | Local AI Lab"
description: "This episode features two 27B multimodal models on Hugging Face both surpassing 4.1 million downloads, running offline image-text Q&A on a single RTX 3090; plus an uncensored 4B version the community reports runs on a 4GB GPU. Stay tuned for which suits writing React components and which fits quantization comparison experiments."
canonical_url: "https://blog.markkulab.net/en/local-ai-lab/local-ai-lab-2026-11-14"
author: "Mark Ku"
author_url: "https://blog.markkulab.net/en/author/mark-ku"
site: "Mark Ku's Tech Notes"
date_published: "2026-11-14T02:00:00.000Z"
category: "On-premise AI Lab"
tags: ["local-ai", "podcast", "地端模型", "lora", "開源模型", "Huihui-Qwen3.8-27B-abliterated-GGUF", "Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF", "Qwen3.5-4B-Uncensored-HauhauCS-Aggressive"]
language: "en"
license: "CC BY 4.0"
license_url: "https://creativecommons.org/licenses/by/4.0/"
attribution: "when reusing or quoting, credit the author and link back to the original"
---

# 🧪 27B multimodal fits on a single 3090, 4B runs on a 4GB GPU: all three GGUFs this issue are free for commercial use | Local AI Lab

## Opening
This edition features two 27B multimodal models on Hugging Face that have each surpassed 4.1 million downloads, both capable of running offline image-text Q&A on a single RTX 3090; plus a 4B-parameter uncensored version that the community reports runs on 4GB VRAM. I'll tell you which one suits writing React components and which fits quantization comparison experiments.

## This Week's Picks

### 1. huihui-ai/Huihui-Qwen3.8-27B-abliterated-GGUF: 27B multimodal, refusal removal, Apache-2.0 commercial-ready, full GGUF quantization support for llama.cpp/Ollama
- **What this is**: A multimodal LLM based on the Qwen2.5-VL-27B family, processed with "abliteration" to remove refusal mechanisms, 27B parameters, Apache-2.0 licensed for direct commercial use. Official provides Q4_K_M, Q5_K_M, Q8_0 and other GGUF quantizations; README states "llama.cpp / Ollama / LM Studio all load directly."
- **Where it fits**:
  - Indie developers running offline image-text Q&A on 24GB VRAM laptops: feed code screenshots plus requirements, get refactoring suggestions instantly, project secrets never leave the machine.
  - Chinese community managers running local 4-bit/8-bit GGUF inference, batch-generating post replies and title optimizations, a single RTX 3090 or 4090 runs 24/7 with zero API costs.
  - Teaching teams packaging the model as an Ollama local service, students in offline classrooms experience multimodal LLMs via CPU inference (q4_K_M ~16GB system RAM), zero cloud cost.
- **Does it run**: Official model card lists Q4_K_M, Q5_K_M, Q8_0 GGUF quantizations, README states "llama.cpp / Ollama / LM Studio all load directly," no specific VRAM thresholds given; estimated from parameter count and 4-bit quantization, single 24GB GPU runs q4_K_M, 16GB system RAM runs CPU inference, above is estimation, not official data.
- **This week's buzz**: HF trending 195, 4.1M cumulative downloads, 2.1k likes, the highest-download 27B multimodal local model this edition.
- **Versus peers**: Same Qwen2.5-VL 27B family as Unsloth's official GGUF (qwen2.5-vl-27b-GGUF) which also supports multimodal, but Huihui version emphasizes "abliterated" refusal removal and includes full system prompt templates; author notes in model card "better suited for research and dev scenarios needing uncensored responses."
- **How to get**: README gives Ollama one-liner: `ollama run hf.co/huihui-ai/Huihui-Qwen3.8-27B-abliterated-GGUF:q4_k_m`; or llama.cpp: `./llama-cli -m Huihui-Qwen3.8-27B-abliterated-Q4_K_M.gguf -p "<|vision_start|><|image_1|><|vision_end|>描述這張圖"`.
- **Link**: https://huggingface.co/huihui-ai/Huihui-Qwen3.8-27B-abliterated-GGUF

### 2. SC117/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF: 27B multimodal, GSQ+RCO quantization claims more precision at same size, Apache-2.0 commercial-ready
- **What this is**: Also based on Qwen2.5-VL-27B-Instruct, authored by @sc117 using GSQ (Group Size Quantization) plus RCO (Rotated Calibration Optimization) quantization techniques to produce GGUF version, 27B parameters, Apache-2.0 licensed for direct commercial use, provides Q4_K_M, Q5_K_M, Q8_0, Q3_K_L and other quantization files.
- **Where it fits**:
  - Quantization researchers comparing GSQ-RCO against traditional K-quant on the same 24GB GPU for long-text generation (e.g., 32k context code completion) perplexity differences, verifying the claimed "lower perplexity at same bit-width."
  - SMB teams deploying multimodal support bots, running Q4_K_M GGUF on a single RTX 3090 for image-text ticket classification, Apache-2.0 license allows direct commercial use without extra negotiation.
  - Edge device engineers fitting q3_K_L version (~14GB) into Jetson Orin NX 16GB modules, running offline anomaly detection prompts for warehouse barcode plus shelf photos.
- **Does it run**: Model card only lists available quantization filenames (Q4_K_M, Q5_K_M, Q8_0, Q3_K_L, etc.), no minimum VRAM requirements stated; README says "Works with llama.cpp, Ollama, LM Studio"; estimated from 27B params and q3_K_L ~14GB size, 16GB GPU or edge devices worth trying, above is estimation, not official data.
- **This week's buzz**: HF trending 192, 4.1M cumulative downloads, 1.0k likes, also a 27B multimodal GGUF with Apache-2.0.
- **Versus peers**: Author notes in model card: "GSQ-RCO quantization by @sc117, based on Qwen2.5-VL-27B-Instruct"; compared to standard K-quant of same base model (like Huihui version), claims "better quality at 3-4 bit," but no third-party benchmark links provided.
- **How to get**: Page provides Ollama command: `ollama run hf.co/SC117/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF:q4_k_m`; llama.cpp: `llama-cli -m Qwen3.8-Flash-Next-GSQ-RCO-abliterated-Q4_K_M.gguf -cnv -p "<|vision_start|><|image_1|><|vision_end|>分析這張圖表"`.
- **Link**: https://huggingface.co/SC117/Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF

### 3. HauhauCS/Qwen3.5-4B-Uncensored-HauhauCS-Aggressive: 4B params, uncensored fine-tune, most lightweight deployable, runs smoothly on 8GB VRAM or even 16GB RAM MacBook
- **What this is**: General-purpose uncensored fine-tune based on Qwen2.5-4B, author writes in model card "Aggressive uncensoring for maximum helpfulness," only 4B parameters, GGUF format, Apache-2.0 royalty-free, suited for extremely constrained hardware.
- **Where it fits**:
  - Frontend engineers on offline flights, using MacBook Pro (M2 16GB) running q4_K_M version (~2.8GB), offline generating React component code, writing unit tests, refactoring CSS, zero cloud API dependency.
  - Security teams deploying q3_K_M on air-gapped old laptops (GTX 1650 4GB + 16GB RAM) for offline log anomaly summarization and threat intel extraction, model weights never leave, Apache-2.0 royalty-free.
  - Teaching labs packaging model as llamafile single executable, students double-click to experience uncensored Chinese chat on Windows, Linux, Mac, zero install barrier, runs without GPU too (pure CPU ~8-12 tok/s per community reports, not official data).
- **Does it run**: Official model card lists no hardware thresholds; README only states "GGUF format for llama.cpp/Ollama"; community feedback in Issues mentions "4GB VRAM runs q3_K_M OK, 8GB VRAM runs q4_K_M comfortably", user reports, not official data; estimated from 4B params and GGUF quantization sizes, 8GB GPU runs q4_K_M, 4GB GPU runs q3_K_M, 16GB RAM MacBook runs CPU inference all feasible, above is estimation.
- **This week's buzz**: HF trending 78, 329k cumulative downloads, 833 likes, the most "lightweight deployable" Chinese-strong model this edition.
- **Versus peers**: Same Qwen2.5-4B family as ccore/qwen3.5-4b-3d-coder (covered previously) which specializes in 3D code; HauhauCS version is general "uncensored" fine-tune, author writes "Aggressive uncensoring for maximum helpfulness," difference lies in training data and system prompts, not architecture.
- **How to get**: Ollama: `ollama run hf.co/HauhauCS/Qwen3.5-4B-Uncensored-HauhauCS-Aggressive-GGUF:q4_k_m`; llama.cpp: `./llama-cli -m Qwen3.5-4B-Uncensored-HauhauCS-Aggressive-Q4_K_M.gguf -p "用繁體中文寫一個快速排序"`.
- **Link**: https://huggingface.co/HauhauCS/Qwen3.5-4B-Uncensored-HauhauCS-Aggressive

## Closing
All three models this edition take the Apache-2.0 commercial route, full GGUF support for llama.cpp and Ollama ecosystems, each hitting a distinct sweet spot: 27B multimodal on consumer GPUs, quantization technique validation, and uncensored Chinese on extreme hardware. Got 24GB VRAM and want offline code screenshot processing? Grab Huihui first. Want to run quantization quality comparison benchmarks? SC117 version fits. Only 4GB VRAM or coding on a MacBook in airplane mode? HauhauCS 4B gives the most mileage. See you next time.

---

## About this article and its author

Originally published on [Mark Ku's Tech Notes](https://blog.markkulab.net/en/local-ai-lab/local-ai-lab-2026-11-14)

License: [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) — when reusing or quoting, credit the author and link back to the original

### About the author

**[Mark Ku](https://blog.markkulab.net/en/author/mark-ku)** — Software engineer

- 10+ years as a software engineer
- Built North-American e-commerce and AI SaaS subscription billing

### Free tools built by the author

All of these are free to use:

- [Free PDF Sign Tool](https://blog.markkulab.net/en/tools/pdf-sign): Online PDF sign tool — draw, type, or upload a signature, then drag, resize, and download. Everything runs in your browser; nothing is uploaded.
- [VS Code Refactory](https://blog.markkulab.net/en/tools/refactory): Refactory is a VS Code refactoring extension: 34 actions plus a 37-rule code-smell inspection layer with a Code Health dashboard, across 18 languages, backed by 534 tests. It learns your repo's conventions: where interfaces live, where DI is registered, whether 'use client' belongs. It ranks files by git churn × complexity so you know what to fix first, and hands any smell to the Claude Code already on your machine. Free to use, and your source never leaves your computer.
- [DB-Kit Database Manager](https://blog.markkulab.net/en/tools/db-kit): DB-Kit is a lightweight, cross-platform database manager built with Tauri + Rust + React. Manage MySQL, MariaDB, PostgreSQL, SQL Server, Oracle, SQLite, MongoDB, Redis, Kafka, Elasticsearch and RabbitMQ from one consistent interface: passwords encrypted in the OS keychain, SSH tunnels, full CRUD, a visual query builder, stacked multi-statement result sets, cross-connection data transfer, schema & data compare with sync SQL and schema snapshots, Excel / CSV import & export, visualized execution plans, ER diagrams, scheduled backups, SQL stress testing with p50–p99 latency percentiles, a 15-rule SQL review engine, Review & Run (AI review plus per-statement backups and an auto-generated rollback script), Kafka message browsing with monitoring & alerts, an SSH terminal with SFTP / FTP, Docker / Kubernetes, remote desktop (RDP / VNC / RustDesk), file / folder compare, stored procedure integration tests, six UI languages, a built-in AI assistant (a local CLI or any Anthropic / OpenAI-compatible API; natural-language SQL, AI review and tuning advice) and the dbk CLI. Free and open source (MIT), with installers for Windows, macOS and Linux.
- [VS Code Super Mermaid](https://blog.markkulab.net/en/tools/super-mermaid): Super Mermaid is a VS Code extension for beautiful Mermaid diagrams out of the box: auto-colored live preview, mouse pan & zoom, high-res PNG / SVG export, 21 templates and multiple themes. Free and open source (MIT).
- [React Super Mermaid](https://blog.markkulab.net/en/tools/react-super-mermaid): react-super-mermaid is an open-source React component library: render beautiful Mermaid diagrams with a single <MermaidViewer>, with built-in colorful / sketch themes, pan & zoom, in-diagram search, and high-res SVG / PNG export. Lightweight, SSR-safe, fully typed. Free and open source (MIT).
- [Jira / Confluence Super Mermaid](https://blog.markkulab.net/en/tools/jira-super-mermaid): An Atlassian Forge app: write Mermaid syntax directly inside a Jira issue or a Confluence page and get flowcharts, sequence diagrams, state machines and Gantt charts. 11 diagram types, SVG / PNG export, light and dark themes, full CJK support. Runs on Atlassian: your diagrams live in your own site and the app calls no third-party service. Free, coming soon to the Atlassian Marketplace.
- [Mermaid Live Preview](https://blog.markkulab.net/en/tools/mermaid-preview): Write Mermaid in your browser, see it render instantly, and share the whole diagram as a single link. No sign-up, nothing uploaded to a server, and mermaid.live share links work as-is.
- [React Intl Phone Number](https://blog.markkulab.net/en/tools/react-intl-phone-number): react-intl-phone-number is an open-source React component: framework-agnostic and antd-free, with E.164 in/out, a searchable flag / country-code dropdown, configurable validation levels (strict / mobile-strict / loose), themeable CSS, and i18n — phone logic powered by google-libphonenumber. Lightweight and fully typed. Free and open source (MIT).
- [Uptime Kuma Cluster](https://blog.markkulab.net/en/tools/uptime-kuma-cluster): Turn single-node Uptime Kuma into a highly available cluster: OpenResty + Lua smart load balancing, shared MariaDB state, health checks and automatic failover, plus cluster-management REST APIs. One Docker Compose command to start. Free and open source (MIT).
- [AI Podcast Cut](https://blog.markkulab.net/en/tools/ai-podcast-cut): Drop in a recording and it removes fillers and stutters, levels loudness segment by segment, and sends a second agent to review every cut. Cut points snap to word boundaries and zero crossings, every splice gets a fade, and sentence-end breaths are preserved. Desktop app for Windows, macOS and Linux. MIT licensed; the Windows installer bundles ffmpeg.
- [AI Video Cut](https://blog.markkulab.net/en/tools/ai-video-cut): Pick something in your video: type it (face, license plate, phone screen, logo), click points or drag a box, or let Claude Code / Codex pick it. AI Video Cut tracks it frame by frame, then mosaics or blurs it, recolors it, pins stickers and text to it, swaps a screen or poster for your own image or video, or removes it using background that other frames actually captured. Plus sequence editing, local captions and an AI assistant. Free, open-source (MIT) Windows desktop app (macOS / Linux experimental) that runs locally, nothing uploaded.
- [open-pos restaurant POS](https://blog.markkulab.net/en/tools/open-pos): One computer and one receipt printer is enough to open the shop. Your data lives on your own disk, no subscription, no lock-in, MIT licensed. Money is integer New Taiwan dollars with tax split by the statutory formula, so sales + tax always equals the total. Printing goes straight over ESC/POS on TCP 9100, with no vendor driver. Tauri + Rust + SQLite desktop app, v1.0 in development.
- [.NET e-commerce API template](https://blog.markkulab.net/en/tools/dotnet-ecommerce-api): An e-commerce backend API template built on .NET 10: Controller → Service → Repository layering, Autofac convention-based registration, SqlSugar for MySQL / MariaDB and SQL Server, Redis cache and message queue, Hangfire jobs, JWT auth, and an extensible payment provider module (ECPay, Newebpay). Apache 2.0.
- [Open Token Monitor team AI usage](https://blog.markkulab.net/en/tools/open-token-monitor): Collect token usage and equivalent cost from every employee's AI coding tools into a hub you host yourself: PostgreSQL, org roster, dashboards compared by company and department, and a reports API. Electron and Rust/Tauri clients upload every 30 minutes. MIT licensed, one Docker command.
- [ai-security-monitor defensive security](https://blog.markkulab.net/en/tools/ai-security-monitor): Your website is live. Who checks its certificates, security settings and known vulnerabilities? A self-hosted Security Hub brings website scans and host monitoring together, deduplicates findings and compares scans after fixes.

### Daily podcasts

- [Mark's Tech Insights — Daily AI News](https://blog.markkulab.net/en/category/tech-news): Daily curated AI and tech trends. Catch the latest developments via audio summaries — covering AI applications, software architecture, DevOps, and engineering practice. — RSS: https://blog.markkulab.net/feed.xml
- [AI股市蝦聊](https://blog.markkulab.net/en/category/ai-stock-chat): Every trading day, an AI-analyzed take on the Taiwan stock market, delivered as a two-host conversation covering the session and the next-day outlook. — RSS: https://blog.markkulab.net/ai-stock-chat/feed.xml
- [開源好物日報](https://blog.markkulab.net/en/category/open-source-weekly): A daily two-host pick of free open-source tools surfaced from real Hacker News, GitHub, and Reddit buzz — what pain they solve and the fastest way to get started. — RSS: https://blog.markkulab.net/open-source-weekly/feed.xml
- [AI 運動日報](https://blog.markkulab.net/en/category/sports-weekly): Two hosts talk NBA, MLB and world sport every day — scores, records and the stories behind them, from a Taiwanese fan perspective. — RSS: https://blog.markkulab.net/sports-weekly/feed.xml
- [AI 國際新聞快報](https://blog.markkulab.net/en/category/world-news): A daily two-host briefing that makes sense of the past 24 hours in world news — geopolitics, the global economy, conflict and security, disasters and climate — from a Taiwanese perspective, neutral and fully sourced. — RSS: https://blog.markkulab.net/world-news/feed.xml
- [地端 AI 實驗室](https://blog.markkulab.net/en/category/local-ai-lab): Every day, a two-host look at open-weight models and LoRAs you can actually run on your own machine: what they are for, whether your GPU can handle them, and how they compare — sourced from official model cards. — RSS: https://blog.markkulab.net/local-ai-lab/feed.xml
- [AI 行銷日報](https://blog.markkulab.net/en/category/ai-marketing): Every day, two hosts break down the latest in AI for marketing, advertising, SEO/GEO and content marketing: practical, never sponsored, every story sourced. — RSS: https://blog.markkulab.net/ai-marketing/feed.xml
- [AI 資安日報](https://blog.markkulab.net/en/category/ai-security): Every day, two hosts cover the latest where AI meets cybersecurity: LLM and AI agent vulnerabilities, prompt injection, model supply-chain risks and AI-powered attacks, from a defense-first angle with every story sourced. — RSS: https://blog.markkulab.net/ai-security/feed.xml

### Newsletter

[Subscribe to the newsletter](https://blog.markkulab.net/en/subscribe) — Be the first to know about new posts. No spam, unsubscribe anytime.