---
title: "🧪 HF Trending #1 Xing4.0-29B-A4B: A 29B model activating only 4B, fits on a single GPU | Local AI Lab"
description: "This week's Hugging Face trending champion is Xing4.0-29B-A4B, with 43K downloads and 1.7K likes; it's a 29B-parameter model but only activates 4B per token, and the official tutorial says a single GPU can fit it. This episode also covers native streaming speech recognition Audio8-ASR-Infinite, along with openjev, a model specialized for reranking and gatekeeping. Next, I'll walk you through the official hardware thresholds for all three models and the scenarios each one can solve."
canonical_url: "https://blog.markkulab.net/en/local-ai-lab/local-ai-lab-2026-09-25"
author: "Mark Ku"
author_url: "https://blog.markkulab.net/en/author/mark-ku"
site: "Mark Ku's Tech Notes"
date_published: "2026-09-25T02:00:00.000Z"
category: "Local AI Lab"
tags: ["local-ai", "podcast", "地端模型", "lora", "開源模型", "Xing4.0-29B-A4B", "Audio8-ASR-Infinite", "openjev", "MoE", "語音辨識", "RAG"]
language: "en"
license: "CC BY 4.0"
license_url: "https://creativecommons.org/licenses/by/4.0/"
attribution: "when reusing or quoting, credit the author and link back to the original"
---

# 🧪 HF Trending #1 Xing4.0-29B-A4B: A 29B model activating only 4B, fits on a single GPU | Local AI Lab

## Opening

This week's top trending model on the HF chart is Xing4.0-29B-A4B, with 43K downloads and 1.7K likes. It has 29B parameters but only activates 4B per token, and the official tutorial says it fits on a single GPU. This episode also covers Audio8-ASR-Infinite, a native streaming speech recognition model, and openjev, a model purpose-built for reranking and fact-checking. Let's walk through the hardware thresholds officially listed for each of these three models, and what scenarios each one can solve.

## Featured This Week

### 1. Xing4.0-29B-A4B: 29B total parameters, MoE activates only 4B, official GGUF tutorial fits on a single GPU

- **What it is**: Xing4.0-29B-A4B from XingChen-AGI is a language model with a MoE (Mixture of Experts) architecture. It has 29B total parameters, but only activates 4B per token during inference. It's released under the Apache-2.0 license in safetensors format. The official model card notes that the entire training run was done on Ascend NPUs with the MindSpore framework, and the publisher is an AI company under China Telecom.
- **Where it's useful**:
  - Backend engineers taking over legacy projects can feed an entire repo along with documentation into the officially listed 256K native context window (the card says it can extend to 512K), and plan cross-file modifications locally, without ever uploading company code to the cloud.
  - Anyone building their own coding agent can spin up a service locally using llama.cpp's llama-server. The model card states it supports multi-step planning and tool calling, and the official tutorial specifically notes you need to add the --jinja flag when invoking tools, which can connect to your own CLI or editor extensions.
  - Internal tooling teams can use it for the vertical applications the card specifically calls out: intent classification, table understanding, contract review, and knowledge base Q&A, doing lightweight customization on their own data without sending sensitive documents outside the company.
- **Can it actually run**: The official model card doesn't give a specific VRAM figure, only noting compatibility with vLLM, SGLang, and KTransformers. The official llama.cpp tutorial is more concrete: after IQ4_NL mixed-precision quantization, the GGUF file is about 18 GB, and the tutorial says it can run on a single consumer-grade GPU. It also gives headroom figures for two 12GB GPUs with Q8_0 KV cache: 5.2 GiB remaining at 65,536 tokens of context, 4.5 GiB at 131,072 tokens, and 3.1 GiB at 262,144 tokens (the officially listed native context limit). The GGUF version is hosted in a separate official repo, XingChen-AGI/Xing4.0-29B-A4B-GGUF.
- **This week's buzz**: HF trending score of 1223, with 43.0K downloads and 1.7K likes, making it the highest-trending of the three models featured this episode.
- **How it compares**: Of the two models trending higher this period, according to their own model cards, deepseek-ai/DeepSeek-V4.1-Flash claims a 552B backbone (8B active for prefill, 16B for decode), while XiaomiMiMo/MiMo-V2.6-Flash-RL claims 309B/15B and its deployment example explicitly specifies --tp 8; neither is single-machine scale. Xing4.0 is the only model this period that combines "MoE efficiency" with "only 29B total weights." The card self-reports SWE-bench Verified 75.00 and Terminal-Bench 2.1 57.50.
- **How to get it**: In the README's transformers example, remember to include trust_remote_code=True and device_map="auto". To spin up a service in one line, use vllm serve "XingChen-AGI/Xing4.0-29B-A4B", or python3 -m sglang.launch_server --model-path "XingChen-AGI/Xing4.0-29B-A4B". For local quantized deployment, grab the official GGUF repo; the tutorial says to place the GGUF file in the deploy directory alongside llama-server, and use -ngl 99 to offload layers to the GPU.
- **Link**: [XingChen-AGI/Xing4.0-29B-A4B](https://huggingface.co/XingChen-AGI/Xing4.0-29B-A4B)

### 2. Audio8-ASR-Infinite: 4B native streaming speech recognition, adjustable Chinese latency, no cloud required

- **What it is**: Audio8-ASR-Infinite from Edge0 is a 4B-parameter native streaming speech recognition model. The weight file model.safetensors is 8.17 GB in bfloat16, released under the Apache-2.0 license for commercial use, and supports both Chinese and English.
- **Where it's useful**:
  - Engineers building customer service or online meeting systems who need real-time captions without uploading recordings to a third party: the model card states transcription latency falls between 240 and 560 ms, and the audio clock can be set to 80, 120, or 160 ms, letting you find your own balance between latency and accuracy.
  - Scenarios requiring uninterrupted, long-duration audio capture, such as live-stream captions or overnight monitoring desks: the card notes the native context is only 30 seconds, but rolling KV cache extends it to unlimited length. The author explicitly states it was designed for 24/7 continuous operation.
  - Content teams with a batch of Chinese interviews or internal meeting recordings to transcribe can run them through the transformers pipeline in bulk, simply specifying zh as the language parameter.
- **Can it actually run**: The card doesn't specify a minimum VRAM requirement, only noting the 4B parameter count, the 8.17 GB model.safetensors file, and bfloat16 precision. The inference code uses .cuda(), indicating a CUDA GPU is required. The author recommends Docker Compose for production deployment, with streaming inference running over WebSocket paired with vLLM.
- **This week's buzz**: HF trending score of 426, with 2.9K downloads and 473 likes.
- **How it compares**: In a table the author included in the card (under a 480 ms latency, 80 ms frame length configuration), the comparison model is explicitly named as Voxtral: on AISHELL-1, the character error rate (CER) is 1.750% versus Voxtral's 16.795%, a substantial gap in Chinese. But on LibriSpeech test.clean, the word error rate (WER) is 3.042% versus Voxtral's 2.210%, and on test.other it's 6.808% versus 5.552%, showing it actually falls slightly behind on English. Based on the author's own figures, its selling point is Chinese recognition and streaming latency, not raw English accuracy.
- **How to get it**: The shortest path given in the README is transformers' pipeline("automatic-speech-recognition", model="Edge0/Audio8-ASR-Infinite", trust_remote_code=True). For streaming decoding, there's also a CLI example: python -m audio8_asr_infinite.examples.torch_streaming_decode --checkpoint /path/to/checkpoint --audio sample.wav --language zh --transcription-delay-ms 480.
- **Link**: [Edge0/Audio8-ASR-Infinite](https://huggingface.co/Edge0/Audio8-ASR-Infinite)

### 3. openjev: turning Qwen3.5 into a dedicated cross-encoder for reranking and fact-checking

- **What it is**: openjev by AlexWortega is a project that transforms Qwen3.5 into a single cross-encoder judgment model. Checkpoints range from 0.8B all the way up to 35B-A3B (MoE), under the MIT license, with no gated access required for download.
- **Where it's useful**:
  - Backend engineers building their own RAG system can feed a dozen or so retrieved candidates into it for reranking: the author's README includes a rerank example, and the 0.8B checkpoint adds barely any latency.
  - Anyone wanting to fact-check generated answers can treat the model's output as the hypothesis and the retrieved text as the premise, letting it judge entailment, contradiction, or neutral, and flag sentences lacking support, without calling on a large model to act as judge.
  - Content moderation or real-time interaction judgment: the card explicitly lists use cases including "guard content" and "play games in real time," achieving real-time responsiveness with small checkpoints.
- **Can it actually run**: The official card doesn't specify VRAM or hardware requirements. It only lists the available checkpoint sizes, 0.8B, 2B, 4B, and 35B-A3B (MoE), with the base model listed as Qwen3.5-4B. No official hardware figures are given here; the only reasonable inference is that the 0.8B to 4B range is far smaller than typical large conversational models and theoretically easier to fit on a consumer GPU, but that's an estimate based on parameter count, not an official stated threshold.
- **This week's buzz**: HF trending score of 441, with 0 downloads and 566 likes.
- **How it compares**: The author positions it as a cross-encoder that handles reranking, scoring, and fact-checking all at once, reading in a premise and hypothesis and directly outputting a three-way classification, taking a different approach from the common bi-encoder vector-based reranker. The reported scores are self-reported by the author on MNLI, ANLI, and a self-built benchmark called JevBench.
- **How to get it**: The loading example given in the card is AutoTokenizer and AutoModelForSequenceClassification.from_pretrained("AlexWortega/openjev", subfolder="qwen3.5-4b-nli-v5"), with weights stored under their respective subfolders. The author notes that v5 is suited for output classification decisions, while v2 is used for multimodal scenarios. The README also includes examples for SGLang deployment, reranking, and hypothesis prediction.
- **Link**: [AlexWortega/openjev](https://huggingface.co/AlexWortega/openjev)

## Closing

This episode's three models each address a different local-deployment pain point: Xing4.0-29B-A4B fills the gap for flagship-level agent capability that still fits on a single GPU, Audio8-ASR-Infinite fills the gap for real-time captions that can't be sent to the cloud, and openjev fills the gap that self-hosted RAG systems need most and yet shouldn't be brute-forced with a large model, reranking and fact-checking. If you can only try one first, openjev's 0.8B checkpoint has the lowest barrier to entry and can be plugged into an existing RAG pipeline the fastest for testing. See you next episode.

---

## About this article and its author

Originally published on [Mark Ku's Tech Notes](https://blog.markkulab.net/en/local-ai-lab/local-ai-lab-2026-09-25)

License: [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) — when reusing or quoting, credit the author and link back to the original

### About the author

**[Mark Ku](https://blog.markkulab.net/en/author/mark-ku)** — 226 Network Co. · Software engineer

- 10+ years as a software engineer, now running 226 Network
- Built North-American e-commerce and AI SaaS subscription billing
- Systems, websites, LINE bots and AI automation for small companies

### Free tools built by the author

All of these are free to use:

- [Free PDF Sign Tool](https://blog.markkulab.net/en/tools/pdf-sign): Online PDF sign tool — draw, type, or upload a signature, then drag, resize, and download. Everything runs in your browser; nothing is uploaded.
- [VS Code Refactory](https://blog.markkulab.net/en/tools/refactory): Refactory is a VS Code refactoring extension: 34 actions plus a 37-rule code-smell inspection layer with a Code Health dashboard, across 18 languages, backed by 534 tests. It learns your repo's conventions: where interfaces live, where DI is registered, whether 'use client' belongs. It ranks files by git churn × complexity so you know what to fix first, and hands any smell to the Claude Code already on your machine. Free to use, and your source never leaves your computer.
- [DB-Kit Database Manager](https://blog.markkulab.net/en/tools/db-kit): DB-Kit is a lightweight, cross-platform database manager built with Tauri + Rust + React. Manage MySQL, MariaDB, PostgreSQL, SQL Server, Oracle, SQLite, MongoDB, Redis, Kafka, Elasticsearch and RabbitMQ from one consistent interface: passwords encrypted in the OS keychain, SSH tunnels, full CRUD, a visual query builder, stacked multi-statement result sets, cross-connection data transfer, schema & data compare with sync SQL and schema snapshots, Excel / CSV import & export, visualized execution plans, ER diagrams, scheduled backups, SQL stress testing with p50–p99 latency percentiles, a 15-rule SQL review engine, Review & Run (AI review plus per-statement backups and an auto-generated rollback script), Kafka message browsing with monitoring & alerts, a bilingual UI (Traditional Chinese / English), a built-in AI assistant (a local CLI or any Anthropic / OpenAI-compatible API; natural-language SQL, AI review and tuning advice) and the dbk CLI. Free and open source (MIT), with installers for Windows, macOS and Linux.
- [VS Code Super Mermaid](https://blog.markkulab.net/en/tools/super-mermaid): Super Mermaid is a VS Code extension for beautiful Mermaid diagrams out of the box: auto-colored live preview, mouse pan & zoom, high-res PNG / SVG export, 21 templates and multiple themes. Free and open source (MIT).
- [React Super Mermaid](https://blog.markkulab.net/en/tools/react-super-mermaid): react-super-mermaid is an open-source React component library: render beautiful Mermaid diagrams with a single <MermaidViewer>, with built-in colorful / sketch themes, pan & zoom, in-diagram search, and high-res SVG / PNG export. Lightweight, SSR-safe, fully typed. Free and open source (MIT).
- [Jira / Confluence Super Mermaid](https://blog.markkulab.net/en/tools/jira-super-mermaid): An Atlassian Forge app: write Mermaid syntax directly inside a Jira issue or a Confluence page and get flowcharts, sequence diagrams, state machines and Gantt charts. 11 diagram types, SVG / PNG export, light and dark themes, full CJK support. Runs on Atlassian: your diagrams live in your own site and the app calls no third-party service. Free, coming soon to the Atlassian Marketplace.
- [Mermaid Live Preview](https://blog.markkulab.net/en/tools/mermaid-preview): Write Mermaid in your browser, see it render instantly, and share the whole diagram as a single link. No sign-up, nothing uploaded to a server, and mermaid.live share links work as-is.
- [React Intl Phone Number](https://blog.markkulab.net/en/tools/react-intl-phone-number): react-intl-phone-number is an open-source React component: framework-agnostic and antd-free, with E.164 in/out, a searchable flag / country-code dropdown, configurable validation levels (strict / mobile-strict / loose), themeable CSS, and i18n — phone logic powered by google-libphonenumber. Lightweight and fully typed. Free and open source (MIT).
- [Uptime Kuma Cluster](https://blog.markkulab.net/en/tools/uptime-kuma-cluster): Turn single-node Uptime Kuma into a highly available cluster: OpenResty + Lua smart load balancing, shared MariaDB state, health checks and automatic failover, plus cluster-management REST APIs. One Docker Compose command to start. Free and open source (MIT).
- [AI Podcast Cut](https://blog.markkulab.net/en/tools/ai-podcast-cut): Drop in a recording and it removes fillers and stutters, levels loudness segment by segment, and sends a second agent to review every cut. Cut points snap to word boundaries and zero crossings, every splice gets a fade, and sentence-end breaths are preserved. Desktop app for Windows, macOS and Linux. MIT licensed; the Windows installer bundles ffmpeg.
- [open-pos restaurant POS](https://blog.markkulab.net/en/tools/open-pos): One computer and one receipt printer is enough to open the shop. Your data lives on your own disk, no subscription, no lock-in, MIT licensed. Money is integer New Taiwan dollars with tax split by the statutory formula, so sales + tax always equals the total. Printing goes straight over ESC/POS on TCP 9100, with no vendor driver. Tauri + Rust + SQLite desktop app, v1.0 in development.
- [Special Education](https://blog.markkulab.net/en/education): Learning materials crafted for special education students

### Daily podcasts

- [Mark's Tech Insights — Daily AI News](https://blog.markkulab.net/en/category/tech-news): Daily curated AI and tech trends. Catch the latest developments via audio summaries — covering AI applications, software architecture, DevOps, and engineering practice. — RSS: https://blog.markkulab.net/feed.xml
- [AI股市蝦聊](https://blog.markkulab.net/en/category/ai-stock-chat): Every trading day, an AI-analyzed take on the Taiwan stock market, delivered as a two-host conversation covering the session and the next-day outlook. — RSS: https://blog.markkulab.net/ai-stock-chat/feed.xml
- [開源好物週報](https://blog.markkulab.net/en/category/open-source-weekly): A weekly two-host pick of free open-source tools surfaced from real Hacker News, GitHub, and Reddit buzz — what pain they solve and the fastest way to get started. — RSS: https://blog.markkulab.net/open-source-weekly/feed.xml
- [AI 運動週報](https://blog.markkulab.net/en/category/sports-weekly): Two hosts talk NBA, MLB and world sport three times a week — scores, records and the stories behind them, from a Taiwanese fan perspective. — RSS: https://blog.markkulab.net/sports-weekly/feed.xml
- [AI 國際新聞快報](https://blog.markkulab.net/en/category/world-news): A daily two-host briefing that makes sense of the past 24 hours in world news — geopolitics, the global economy, conflict and security, disasters and climate — from a Taiwanese perspective, neutral and fully sourced. — RSS: https://blog.markkulab.net/world-news/feed.xml
- [地端 AI 實驗室](https://blog.markkulab.net/en/category/local-ai-lab): Twice a week, a two-host look at open-weight models and LoRAs you can actually run on your own machine: what they are for, whether your GPU can handle them, and how they compare — sourced from official model cards. — RSS: https://blog.markkulab.net/local-ai-lab/feed.xml

### Deals

- [Saily eSIM](https://blog.markkulab.net/en/saily): Travel eSIM by the NordVPN team，Promo code：KUKU
- [NordVPN](https://blog.markkulab.net/en/nordvpn): The world's leading VPN, independently audited
- [PremLogin](https://blog.markkulab.net/en/premlogin): Subscription sharing for streaming and AI seats，Promo code：markku666

### Newsletter

[Subscribe to the newsletter](https://blog.markkulab.net/en/subscribe) — Be the first to know about new posts. No spam, unsubscribe anytime.