Opening

On November 17, 2026, our radar picked up Haru-4B, a 4B-parameter audio-to-audio model with 4.4k downloads under Apache-2.0. The official model card claims 16GB VRAM for BF16, 8GB for 4-bit GGUF, and the ability to turn English meeting recordings directly into Chinese speech without text as an intermediate step. Also coming up: Inkling-2-Small, a compact vision-language model with 18.4k downloads that reportedly fits in 3.2GB at 4-bit GGUF, worth a look for engineers running invoice parsing on consumer GPUs; and fluid-speech-pro, a 0.5B ASR model that the authors clocked at 0.25 real-time factor on an Intel i7-12700 CPU, even running offline transcription on a Raspberry Pi 5, under CC-BY-4.0 for unrestricted commercial use.

This Week's Picks

1. higurashi-ai/Haru-4B: 4B End-to-End Speech-to-Speech, Commercial-Friendly, Fine-Tunable

  • What it is
    A 4B-parameter audio-to-audio model for Speech-to-Speech tasks, licensed Apache-2.0, commercial use and fine-tuning both allowed. The model card includes a fine-tune script and data format docs, so you can fork and experiment with LoRA for specific speaker styles (e.g., customer-service tone) right away.
  • Where it fits
    • Indie developers wanting an offline "speech-to-speech" demo on a laptop (16–24GB VRAM), e.g., turning English meeting recordings directly into Chinese speech output, no text intermediary.
    • Voice assistant product teams deploying low-latency multilingual conversational entry points on edge devices (Jetson Orin, Mac Studio); the model card notes streaming inference support.
    • Grad students fine-tuning specific speaker styles with LoRA; the model card provides the fine-tune script and data format, ready to fork.
  • Can it run?
    Official model card states: "BF16 on 16GB VRAM, 4-bit GGUF on 8GB VRAM"; also provides GGUF quantizations (Q4_K_M, Q8_0) for llama.cpp. If you only have 8GB VRAM, the official 4-bit GGUF path is likely the easiest entry point.
  • Buzz this week
    HF trending score 374, 4.4k downloads, 476 likes.
  • vs. peers
    The author compares against SeamlessM4T-v2-large (1.2B) and Whisper + TTS pipelines in the model card, claiming Haru-4B wins on "end-to-end latency" and "tone preservation"; this is self-reported, no independent third-party reproduction yet.
  • How to get it
    README gives two paths: ① pip install -U transformers accelerate then AutoModelForSpeechSeq2Seq.from_pretrained("higurashi-ai/Haru-4B"); ② download GGUF with llama.cpp --model haru-4b-q4_k_m.gguf --audio-in mic.
  • Link
    https://huggingface.co/higurashi-ai/Haru-4B

2. thinkingmachines/Inkling-2-Small: Compact VLM with 18.4k Downloads, 3.2GB at 4-bit

  • What it is
    A small vision-language model, Apache-2.0 licensed, supports 1024×1024 resolution, accepts PDF-rendered images directly. The model card includes train_lora.py and chat_template examples, ready to plug into Unsloth workflows for fine-tuning on your own corpus (e.g., traditional Chinese financial reports).
  • Where it fits
    • SaaS backend engineers running "invoice/receipt auto-parsing" on 12GB VRAM machines; model card notes 1024×1024 support and direct PDF-rendered image input.
    • Edge device engineers quantizing to 4-bit GGUF for deployment on Jetson Orin NX (16GB) for real-time "screen-content QA" test assistance.
    • Data scientists LoRA-fine-tuning on proprietary docs (traditional Chinese financial reports); model card provides train_lora.py and chat_template examples, directly compatible with Unsloth.
  • Can it run?
    Official model card: "FP16 needs 8GB VRAM, 8-bit needs 5GB, 4-bit GGUF ~3.2GB memory"; pre-quantized GGUF (Q4_K_M, Q5_K_M) provided. This means 8-bit should theoretically run on 8GB consumer GPUs, and laptops with enough system RAM could even run 4-bit GGUF on CPU.
  • Buzz this week
    HF trending score 347, 18.4k downloads, 434 likes.
  • vs. peers
    Author compares against Qwen2-VL-2B and MiniCPM-V-2_6 in the model card, claiming 3–5 point gains on DocVQA; self-reported numbers, no public third-party reproduction yet.
  • How to get it
    README instructs: from transformers import AutoModelForImageTextToText; model = AutoModelForImageTextToText.from_pretrained("thinkingmachines/Inkling-2-Small", load_in_8bit=True, device_map="auto"); or load GGUF with llama.cpp.
  • Link
    https://huggingface.co/thinkingmachines/Inkling-2-Small

3. FluidVoice-Labs/fluid-speech-pro: 0.5B ASR Built for CPU Inference, Runs Offline Transcription on Pi 5

  • What it is
    A 0.5B-class model specialized for Automatic Speech Recognition (ASR), CC-BY-4.0 licensed, commercial use and redistribution allowed, no application needed. Model card provides ONNX and CTranslate2 versions, emphasizing "CPU real-time factor < 0.3".
  • Where it fits
    • Full-stack engineers running "auto-generate podcast transcripts" in GPU-less CI/CD pipelines; model card provides ONNX and CTranslate2 versions, ready to pip install ctranslate2 for inference.
    • Embedded engineers deploying offline mixed Chinese-English recognition on Raspberry Pi 5 / Orange Pi 5 Plus; model card cites int8 quantization to 40MB, CPU real-time factor 0.25.
    • Privacy-sensitive teams (medical/legal) batch-transcribing recordings on air-gapped servers; CC-BY-4.0 permits commercial use and redistribution without application.
  • Can it run?
    Official model card: "FP16 0.5B ~1GB VRAM; INT8 quantized CPU real-time factor 0.25 (Intel i7-12700), 0.35 (Apple M2); ONNX Runtime / CTranslate2 both supported". Machines without discrete GPUs, even Pi 5-class devices, can theoretically run offline as long as CPU performance suffices and memory fits the 40MB model.
  • Buzz this week
    HF trending score 49, 9.1k downloads, 53 likes.
  • vs. peers
    Author compares against Whisper-large-v3 (1.5B) and distil-whisper-large-v3 (756M) in the model card, claiming WER on CommonVoice zh-TW test set within 0.8% while being 4x faster; self-reported, no public third-party reproduction yet.
  • How to get it
    README lists three ways: ① pip install fluid-voice then transcribe("audio.wav"); ② download ONNX with onnxruntime-genai; ③ download CTranslate2 model with ctranslate2.Translator("model_ct2").
  • Link
    https://huggingface.co/FluidVoice-Labs/fluid-speech-pro

Closing

All three models this week sit at the intersection of "small parameter count, solid quantization, permissive licensing": Haru-4B brings speech-to-speech down to a single consumer GPU, Inkling-2-Small compresses document understanding to 3.2GB, and fluid-speech-pro pushes transcription onto GPU-less CI/CD and Pi 5. If you have 16GB+ VRAM and want to try real-time speech translation, grab Haru-4B's GGUF for a quick feel; if you have zero GPU and just need transcripts, fluid-speech-pro's Python package is the lowest-friction path. See you next time.

Author

Mark Ku

10 年以上的軟體工程師,做過北美電商與 AI SaaS 訂閱收費系統。Read More

Found this useful?

The author's free tools, daily podcasts and newsletter are all here.

Mark Ku · This article is licensed under CC BY 4.0. Credit the author and link back to the original when reusing it.

Comments

Subscribe to Newsletter

Subscribe to get new posts delivered instantly — never miss a tech share.

By submitting, you agree to receive emails. You can anytime.

Popular Posts

View all
Mark Ku
··633

Oracle Cloud Always Free Tier: Linux Host and Static IP for a $0 Cloud Solution

Oracle Cloud Always Free Tier: Linux Host and Static IP for a $0 Cloud Solution
Mark Ku
··464

Say Goodbye to Postman's Fee Trap! A Hands-on Guide to Bruno, the Open-Source Git-Native API Testing Powerhouse.

Say Goodbye to Postman's Fee Trap! A Hands-on Guide to Bruno, the Open-Source Git-Native API Testing Powerhouse.
Mark Ku
··249

A Free, Open-Source, Notion-like Knowledge Base — A Complete Guide to Deploying and Backing Up Outline Wiki

A Free, Open-Source, Notion-like Knowledge Base — A Complete Guide to Deploying and Backing Up Outline Wiki
Mark Ku
··205

Building an Efficient API Management Platform: Deploying Kong Gateway from Scratch - Part 1

Building an Efficient API Management Platform: Deploying Kong Gateway from Scratch - Part 1
Mark Ku
··202

Setting Up Samba on Ubuntu to Share Folders with Windows 11

Setting Up Samba on Ubuntu to Share Folders with Windows 11
Mark Ku
··198

Training Your Own AI Voice: Hardware Requirements, Open-Source Model Comparison, and LoRA Fine-Tuning

Training Your Own AI Voice: Hardware Requirements, Open-Source Model Comparison, and LoRA Fine-Tuning