Opening

This week's top trending model on the HF chart is Xing4.0-29B-A4B, with 43K downloads and 1.7K likes. It has 29B parameters but only activates 4B per token, and the official tutorial says it fits on a single GPU. This episode also covers Audio8-ASR-Infinite, a native streaming speech recognition model, and openjev, a model purpose-built for reranking and fact-checking. Let's walk through the hardware thresholds officially listed for each of these three models, and what scenarios each one can solve.

Featured This Week

1. Xing4.0-29B-A4B: 29B total parameters, MoE activates only 4B, official GGUF tutorial fits on a single GPU

• What it is: Xing4.0-29B-A4B from XingChen-AGI is a language model with a MoE (Mixture of Experts) architecture. It has 29B total parameters, but only activates 4B per token during inference. It's released under the Apache-2.0 license in safetensors format. The official model card notes that the entire training run was done on Ascend NPUs with the MindSpore framework, and the publisher is an AI company under China Telecom.
• Where it's useful:
  • Backend engineers taking over legacy projects can feed an entire repo along with documentation into the officially listed 256K native context window (the card says it can extend to 512K), and plan cross-file modifications locally, without ever uploading company code to the cloud.
  • Anyone building their own coding agent can spin up a service locally using llama.cpp's llama-server. The model card states it supports multi-step planning and tool calling, and the official tutorial specifically notes you need to add the --jinja flag when invoking tools, which can connect to your own CLI or editor extensions.
  • Internal tooling teams can use it for the vertical applications the card specifically calls out: intent classification, table understanding, contract review, and knowledge base Q&A, doing lightweight customization on their own data without sending sensitive documents outside the company.
• Can it actually run: The official model card doesn't give a specific VRAM figure, only noting compatibility with vLLM, SGLang, and KTransformers. The official llama.cpp tutorial is more concrete: after IQ4_NL mixed-precision quantization, the GGUF file is about 18 GB, and the tutorial says it can run on a single consumer-grade GPU. It also gives headroom figures for two 12GB GPUs with Q8_0 KV cache: 5.2 GiB remaining at 65,536 tokens of context, 4.5 GiB at 131,072 tokens, and 3.1 GiB at 262,144 tokens (the officially listed native context limit). The GGUF version is hosted in a separate official repo, XingChen-AGI/Xing4.0-29B-A4B-GGUF.
• This week's buzz: HF trending score of 1223, with 43.0K downloads and 1.7K likes, making it the highest-trending of the three models featured this episode.
• How it compares: Of the two models trending higher this period, according to their own model cards, deepseek-ai/DeepSeek-V4.1-Flash claims a 552B backbone (8B active for prefill, 16B for decode), while XiaomiMiMo/MiMo-V2.6-Flash-RL claims 309B/15B and its deployment example explicitly specifies --tp 8; neither is single-machine scale. Xing4.0 is the only model this period that combines "MoE efficiency" with "only 29B total weights." The card self-reports SWE-bench Verified 75.00 and Terminal-Bench 2.1 57.50.
• How to get it: In the README's transformers example, remember to include trust_remote_code=True and device_map="auto". To spin up a service in one line, use vllm serve "XingChen-AGI/Xing4.0-29B-A4B", or python3 -m sglang.launch_server --model-path "XingChen-AGI/Xing4.0-29B-A4B". For local quantized deployment, grab the official GGUF repo; the tutorial says to place the GGUF file in the deploy directory alongside llama-server, and use -ngl 99 to offload layers to the GPU.
• Link: XingChen-AGI/Xing4.0-29B-A4B

2. Audio8-ASR-Infinite: 4B native streaming speech recognition, adjustable Chinese latency, no cloud required

• What it is: Audio8-ASR-Infinite from Edge0 is a 4B-parameter native streaming speech recognition model. The weight file model.safetensors is 8.17 GB in bfloat16, released under the Apache-2.0 license for commercial use, and supports both Chinese and English.
• Where it's useful:
  • Engineers building customer service or online meeting systems who need real-time captions without uploading recordings to a third party: the model card states transcription latency falls between 240 and 560 ms, and the audio clock can be set to 80, 120, or 160 ms, letting you find your own balance between latency and accuracy.
  • Scenarios requiring uninterrupted, long-duration audio capture, such as live-stream captions or overnight monitoring desks: the card notes the native context is only 30 seconds, but rolling KV cache extends it to unlimited length. The author explicitly states it was designed for 24/7 continuous operation.
  • Content teams with a batch of Chinese interviews or internal meeting recordings to transcribe can run them through the transformers pipeline in bulk, simply specifying zh as the language parameter.
• Can it actually run: The card doesn't specify a minimum VRAM requirement, only noting the 4B parameter count, the 8.17 GB model.safetensors file, and bfloat16 precision. The inference code uses .cuda(), indicating a CUDA GPU is required. The author recommends Docker Compose for production deployment, with streaming inference running over WebSocket paired with vLLM.
• This week's buzz: HF trending score of 426, with 2.9K downloads and 473 likes.
• How it compares: In a table the author included in the card (under a 480 ms latency, 80 ms frame length configuration), the comparison model is explicitly named as Voxtral: on AISHELL-1, the character error rate (CER) is 1.750% versus Voxtral's 16.795%, a substantial gap in Chinese. But on LibriSpeech test.clean, the word error rate (WER) is 3.042% versus Voxtral's 2.210%, and on test.other it's 6.808% versus 5.552%, showing it actually falls slightly behind on English. Based on the author's own figures, its selling point is Chinese recognition and streaming latency, not raw English accuracy.
• How to get it: The shortest path given in the README is transformers' pipeline("automatic-speech-recognition", model="Edge0/Audio8-ASR-Infinite", trust_remote_code=True). For streaming decoding, there's also a CLI example: python -m audio8_asr_infinite.examples.torch_streaming_decode --checkpoint /path/to/checkpoint --audio sample.wav --language zh --transcription-delay-ms 480.
• Link: Edge0/Audio8-ASR-Infinite

3. openjev: turning Qwen3.5 into a dedicated cross-encoder for reranking and fact-checking

• What it is: openjev by AlexWortega is a project that transforms Qwen3.5 into a single cross-encoder judgment model. Checkpoints range from 0.8B all the way up to 35B-A3B (MoE), under the MIT license, with no gated access required for download.
• Where it's useful:
  • Backend engineers building their own RAG system can feed a dozen or so retrieved candidates into it for reranking: the author's README includes a rerank example, and the 0.8B checkpoint adds barely any latency.
  • Anyone wanting to fact-check generated answers can treat the model's output as the hypothesis and the retrieved text as the premise, letting it judge entailment, contradiction, or neutral, and flag sentences lacking support, without calling on a large model to act as judge.
  • Content moderation or real-time interaction judgment: the card explicitly lists use cases including "guard content" and "play games in real time," achieving real-time responsiveness with small checkpoints.
• Can it actually run: The official card doesn't specify VRAM or hardware requirements. It only lists the available checkpoint sizes, 0.8B, 2B, 4B, and 35B-A3B (MoE), with the base model listed as Qwen3.5-4B. No official hardware figures are given here; the only reasonable inference is that the 0.8B to 4B range is far smaller than typical large conversational models and theoretically easier to fit on a consumer GPU, but that's an estimate based on parameter count, not an official stated threshold.
• This week's buzz: HF trending score of 441, with 0 downloads and 566 likes.
• How it compares: The author positions it as a cross-encoder that handles reranking, scoring, and fact-checking all at once, reading in a premise and hypothesis and directly outputting a three-way classification, taking a different approach from the common bi-encoder vector-based reranker. The reported scores are self-reported by the author on MNLI, ANLI, and a self-built benchmark called JevBench.
• How to get it: The loading example given in the card is AutoTokenizer and AutoModelForSequenceClassification.from_pretrained("AlexWortega/openjev", subfolder="qwen3.5-4b-nli-v5"), with weights stored under their respective subfolders. The author notes that v5 is suited for output classification decisions, while v2 is used for multimodal scenarios. The README also includes examples for SGLang deployment, reranking, and hypothesis prediction.
• Link: AlexWortega/openjev

Closing

This episode's three models each address a different local-deployment pain point: Xing4.0-29B-A4B fills the gap for flagship-level agent capability that still fits on a single GPU, Audio8-ASR-Infinite fills the gap for real-time captions that can't be sent to the cloud, and openjev fills the gap that self-hosted RAG systems need most and yet shouldn't be brute-forced with a large model, reranking and fact-checking. If you can only try one first, openjev's 0.8B checkpoint has the lowest barrier to entry and can be plugged into an existing RAG pipeline the fastest for testing. See you next episode.