--- title: "🧪 Controlling Screens with a 4GB GPU! Building Local Agents with Mage-VL-Cowork: Local AI Lab" description: "Microsoft's latest open-source 4B ultra-lightweight model Mage-VL-Cowork can actually run on a budget graphics card with 4GB VRAM, helping you automatically write Python to organize data and even directly understand the screen to simulate mouse operations. In this episode, we will also introduce the Clover-35B-A3B-Preview diffusion language model, which scored a high 73.6 on SWE-bench: how does it achieve the extreme speed of over 100 tokens per second? We'll tell you in a moment!" canonical_url: "https://blog.markkulab.net/en/local-ai-lab/local-ai-lab-2026-11-01" author: "Mark Ku" author_url: "https://blog.markkulab.net/en/author/mark-ku" site: "Mark Ku's Tech Notes" date_published: "2026-11-01T02:00:00.000Z" category: "On-premises AI Lab" tags: ["local-ai", "podcast", "地端模型", "lora", "開源模型", "Mage-VL-Cowork", "Clover-35B-A3B-Preview", "CodingAgent", "電腦操作Agent", "AI實驗室"] language: "en" license: "CC BY 4.0" license_url: "https://creativecommons.org/licenses/by/4.0/" attribution: "when reusing or quoting, credit the author and link back to the original" --- # 🧪 Controlling Screens with a 4GB GPU! Building Local Agents with Mage-VL-Cowork: Local AI Lab Microsoft's newly open-sourced 4B ultra-lightweight model, Mage-VL-Cowork, can actually run on a budget graphics card with just 4GB of VRAM to automatically write Python code for data organization, and even directly understand screen layouts to simulate mouse operations. In this issue, we also bring you Clover-35B-A3B-Preview, a diffusion language model that scored a whopping 73.6 on SWE-bench. How does it achieve an extreme speed of over 100 tokens per second? We will tell you in just a moment! ## Featured This Week ### 1. microsoft/Mage-VL-Cowork: An Ultra-Lightweight Local Hybrid Computer-Use Agent - **What it is**: This is an ultra-lightweight 4B (4 billion parameters) multimodal computer-use Agent model open-sourced by Microsoft under the developer-friendly apache-2.0 license. Its biggest highlight is its ability to autonomously switch between graphical user interface (GUI) and command-line interface (CLI) operations within a single decision-making mechanism. - **Use Cases**: - **Scenario 1: Automating repetitive data organization tasks**. When software engineers need to merge complex fields across multiple spreadsheets, they previously had to click through software manually. Now, Mage-VL-Cowork can directly write and execute Python scripts (such as calling the openpyxl library) in the terminal, efficiently handling file editing in the background via code and bypassing tedious GUI clicks. - **Scenario 2: Operating web or GUI applications without APIs**. When developers need to control closed systems or legacy software, the model automatically switches to the computer_use pipeline, recognizing screenshots and simulating mouse clicks and keyboard inputs to complete interface operations just like a human. - **Scenario 3: Running system-level automation tasks in secure, isolated environments**. If a team needs to execute system-level tasks without leaking sensitive code or trade secrets, they can fully deploy this model locally in a sandbox, Docker container, or virtual machine because of its compact 4B size. This allows the Agent to execute arbitrary shell commands and file operations in a secure, privacy-preserving environment. - **Hardware Requirements**: The official model card lists its full BF16 weight size at around 9.48 GB. However, according to evaluations by the third-party analysis platform modelfit.io, its Q4_K_M quantized GGUF format combined with the vision projector has a total size of only about 3 GB. Therefore, an NVIDIA graphics card with at least 4 GB of VRAM (such as a GTX 1650 or RTX 3050) is sufficient to run it, and it will run perfectly fine on a 16GB Mac. - **Current Traction**: It currently has a trending score of 104 on Hugging Face, with 752 cumulative downloads and 105 likes. - **Comparison with Peers**: According to Microsoft's official data on the OSWorld-Verified benchmark, Mage-VL-Cowork achieved a 66.0% task success rate. This score not only surpasses the larger Muse Glimmer-30B (65.9%) and Qwen3.6-27B (63.9%), but also far outperforms Fara1.5-4B (13.9%), which is also a 4B model but solely focused on GUI. - **How to Get It**: The official README states that developers can load the model directly via Transformers (requiring `trust_remote_code=True` to be configured in the code) or deploy it using the official SGLang container. - **Link**: [microsoft/Mage-VL-Cowork](https://huggingface.co/microsoft/Mage-VL-Cowork) --- ### 2. Diffuspace/Clover-35B-A3B-Preview: The High-Speed Diffusion Powerhouse for the Strongest Local Coding Agent - **What it is**: This is a 35B Mixture of Experts (MoE) diffusion language model released by DiffuSpace under the apache-2.0 license. It is currently the first open-source diffusion language model, deeply optimized for Agent tasks, and particularly excels at autonomous coding, tool calling, and web browsing. - **Use Cases**: - **Scenario 1: Building a local autonomous Coding Agent**. When software engineers want to build a local assistant that can automatically fix bugs in GitHub projects, Clover-35B-A3B-Preview demonstrates an extremely high task completion rate. It can autonomously write code, call external tools, and browse the web to efficiently resolve repository issues. - **Scenario 2: Constructing ultra-low latency local AI assistants**. If developers need extremely fast response times to handle complex Agent workflows, the "Block Diffusion" technology used by this model can denoise and generate multiple tokens in parallel, providing a smooth generation experience with ultra-high throughput in complex automation pipelines. - **Hardware Requirements**: According to the third-party deployment guide on mindstudio.ai and the model specifications, the full BF16 weights require about 69GB of VRAM. However, the official README mentions that they already provide 8-bit and 4-bit GGUF quantized versions, allowing developers to easily run local inference via llama.cpp. Additionally, the model supports Self-Speculative Decoding for further acceleration. - **Current Traction**: It currently has a high trending score of 195 on Hugging Face, with 2.4k cumulative downloads and 199 likes. - **Comparison with Peers**: According to official data from DiffuSpace, Clover-35B-A3B-Preview achieved an impressive score of 73.6 on SWE-bench Verified and 51.7 on Terminal-Bench 2.1, outperforming the autoregressive model Qwen3.6-35B-A3B of a similar parameter scale. Furthermore, the official team notes that its generation speed for Agent tasks reaches 515 tokens/s on an H200, which is significantly faster than the Qwen model using DFlash2 speculative decoding (365 tokens/s). - **How to Get It**: The official README states that developers can load it by installing llama.cpp and starting an OpenAI-compatible local server in the terminal: `llama serve -hf Diffuspace/Clover-35B-A3B-Preview-GGUF:Q4_K_M`. - **Link**: [Diffuspace/Clover-35B-A3B-Preview](https://huggingface.co/Diffuspace/Clover-35B-A3B-Preview) --- ## Conclusion From this week's selection, it is clear that the development of local Agent models has entered a brand-new phase of high efficiency and multi-modal operations. Whether it is Mage-VL-Cowork, which flexibly switches between GUI and CLI with just 4GB of VRAM, or Clover-35B-A3B-Preview, which supercharges generation speeds using block diffusion technology, both empower engineers to build highly productive local automated workflows while keeping their data completely private. That is all for today's introduction, see you in the next episode! --- ## About this article and its author Originally published on [Mark Ku's Tech Notes](https://blog.markkulab.net/en/local-ai-lab/local-ai-lab-2026-11-01) License: [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) — when reusing or quoting, credit the author and link back to the original ### About the author **[Mark Ku](https://blog.markkulab.net/en/author/mark-ku)** — Software engineer - 10+ years as a software engineer - Built North-American e-commerce and AI SaaS subscription billing ### Free tools built by the author All of these are free to use: - [Free PDF Sign Tool](https://blog.markkulab.net/en/tools/pdf-sign): Online PDF sign tool — draw, type, or upload a signature, then drag, resize, and download. Everything runs in your browser; nothing is uploaded. - [VS Code Refactory](https://blog.markkulab.net/en/tools/refactory): Refactory is a VS Code refactoring extension: 34 actions plus a 37-rule code-smell inspection layer with a Code Health dashboard, across 18 languages, backed by 534 tests. It learns your repo's conventions: where interfaces live, where DI is registered, whether 'use client' belongs. It ranks files by git churn × complexity so you know what to fix first, and hands any smell to the Claude Code already on your machine. Free to use, and your source never leaves your computer. - [DB-Kit Database Manager](https://blog.markkulab.net/en/tools/db-kit): DB-Kit is a lightweight, cross-platform database manager built with Tauri + Rust + React. Manage MySQL, MariaDB, PostgreSQL, SQL Server, Oracle, SQLite, MongoDB, Redis, Kafka, Elasticsearch and RabbitMQ from one consistent interface: passwords encrypted in the OS keychain, SSH tunnels, full CRUD, a visual query builder, stacked multi-statement result sets, cross-connection data transfer, schema & data compare with sync SQL and schema snapshots, Excel / CSV import & export, visualized execution plans, ER diagrams, scheduled backups, SQL stress testing with p50–p99 latency percentiles, a 15-rule SQL review engine, Review & Run (AI review plus per-statement backups and an auto-generated rollback script), Kafka message browsing with monitoring & alerts, an SSH terminal with SFTP / FTP, Docker / Kubernetes, remote desktop (RDP / VNC / RustDesk), file / folder compare, stored procedure integration tests, six UI languages, a built-in AI assistant (a local CLI or any Anthropic / OpenAI-compatible API; natural-language SQL, AI review and tuning advice) and the dbk CLI. Free and open source (MIT), with installers for Windows, macOS and Linux. - [VS Code Super Mermaid](https://blog.markkulab.net/en/tools/super-mermaid): Super Mermaid is a VS Code extension for beautiful Mermaid diagrams out of the box: auto-colored live preview, mouse pan & zoom, high-res PNG / SVG export, 21 templates and multiple themes. Free and open source (MIT). - [React Super Mermaid](https://blog.markkulab.net/en/tools/react-super-mermaid): react-super-mermaid is an open-source React component library: render beautiful Mermaid diagrams with a single , with built-in colorful / sketch themes, pan & zoom, in-diagram search, and high-res SVG / PNG export. Lightweight, SSR-safe, fully typed. Free and open source (MIT). - [Jira / Confluence Super Mermaid](https://blog.markkulab.net/en/tools/jira-super-mermaid): An Atlassian Forge app: write Mermaid syntax directly inside a Jira issue or a Confluence page and get flowcharts, sequence diagrams, state machines and Gantt charts. 11 diagram types, SVG / PNG export, light and dark themes, full CJK support. Runs on Atlassian: your diagrams live in your own site and the app calls no third-party service. Free, coming soon to the Atlassian Marketplace. - [Mermaid Live Preview](https://blog.markkulab.net/en/tools/mermaid-preview): Write Mermaid in your browser, see it render instantly, and share the whole diagram as a single link. No sign-up, nothing uploaded to a server, and mermaid.live share links work as-is. - [React Intl Phone Number](https://blog.markkulab.net/en/tools/react-intl-phone-number): react-intl-phone-number is an open-source React component: framework-agnostic and antd-free, with E.164 in/out, a searchable flag / country-code dropdown, configurable validation levels (strict / mobile-strict / loose), themeable CSS, and i18n — phone logic powered by google-libphonenumber. Lightweight and fully typed. Free and open source (MIT). - [Uptime Kuma Cluster](https://blog.markkulab.net/en/tools/uptime-kuma-cluster): Turn single-node Uptime Kuma into a highly available cluster: OpenResty + Lua smart load balancing, shared MariaDB state, health checks and automatic failover, plus cluster-management REST APIs. One Docker Compose command to start. Free and open source (MIT). - [AI Podcast Cut](https://blog.markkulab.net/en/tools/ai-podcast-cut): Drop in a recording and it removes fillers and stutters, levels loudness segment by segment, and sends a second agent to review every cut. Cut points snap to word boundaries and zero crossings, every splice gets a fade, and sentence-end breaths are preserved. Desktop app for Windows, macOS and Linux. MIT licensed; the Windows installer bundles ffmpeg. - [AI Video Cut](https://blog.markkulab.net/en/tools/ai-video-cut): Pick something in your video: type it (face, license plate, phone screen, logo), click points or drag a box, or let Claude Code / Codex pick it. AI Video Cut tracks it frame by frame, then mosaics or blurs it, recolors it, pins stickers and text to it, swaps a screen or poster for your own image or video, or removes it using background that other frames actually captured. Plus sequence editing, local captions and an AI assistant. Free, open-source (MIT) Windows desktop app (macOS / Linux experimental) that runs locally, nothing uploaded. - [open-pos restaurant POS](https://blog.markkulab.net/en/tools/open-pos): One computer and one receipt printer is enough to open the shop. Your data lives on your own disk, no subscription, no lock-in, MIT licensed. Money is integer New Taiwan dollars with tax split by the statutory formula, so sales + tax always equals the total. Printing goes straight over ESC/POS on TCP 9100, with no vendor driver. Tauri + Rust + SQLite desktop app, v1.0 in development. - [.NET e-commerce API template](https://blog.markkulab.net/en/tools/dotnet-ecommerce-api): An e-commerce backend API template built on .NET 10: Controller → Service → Repository layering, Autofac convention-based registration, SqlSugar for MySQL / MariaDB and SQL Server, Redis cache and message queue, Hangfire jobs, JWT auth, and an extensible payment provider module (ECPay, Newebpay). Apache 2.0. - [Open Token Monitor team AI usage](https://blog.markkulab.net/en/tools/open-token-monitor): Collect token usage and equivalent cost from every employee's AI coding tools into a hub you host yourself: PostgreSQL, org roster, dashboards compared by company and department, and a reports API. Electron and Rust/Tauri clients upload every 30 minutes. MIT licensed, one Docker command. - [ai-security-monitor defensive security](https://blog.markkulab.net/en/tools/ai-security-monitor): Your website is live. Who checks its certificates, security settings and known vulnerabilities? A self-hosted Security Hub brings website scans and host monitoring together, deduplicates findings and compares scans after fixes. ### Daily podcasts - [Mark's Tech Insights — Daily AI News](https://blog.markkulab.net/en/category/tech-news): Daily curated AI and tech trends. Catch the latest developments via audio summaries — covering AI applications, software architecture, DevOps, and engineering practice. — RSS: https://blog.markkulab.net/feed.xml - [AI股市蝦聊](https://blog.markkulab.net/en/category/ai-stock-chat): Every trading day, an AI-analyzed take on the Taiwan stock market, delivered as a two-host conversation covering the session and the next-day outlook. — RSS: https://blog.markkulab.net/ai-stock-chat/feed.xml - [開源好物日報](https://blog.markkulab.net/en/category/open-source-weekly): A daily two-host pick of free open-source tools surfaced from real Hacker News, GitHub, and Reddit buzz — what pain they solve and the fastest way to get started. — RSS: https://blog.markkulab.net/open-source-weekly/feed.xml - [AI 運動日報](https://blog.markkulab.net/en/category/sports-weekly): Two hosts talk NBA, MLB and world sport every day — scores, records and the stories behind them, from a Taiwanese fan perspective. — RSS: https://blog.markkulab.net/sports-weekly/feed.xml - [AI 國際新聞快報](https://blog.markkulab.net/en/category/world-news): A daily two-host briefing that makes sense of the past 24 hours in world news — geopolitics, the global economy, conflict and security, disasters and climate — from a Taiwanese perspective, neutral and fully sourced. — RSS: https://blog.markkulab.net/world-news/feed.xml - [地端 AI 實驗室](https://blog.markkulab.net/en/category/local-ai-lab): Every day, a two-host look at open-weight models and LoRAs you can actually run on your own machine: what they are for, whether your GPU can handle them, and how they compare — sourced from official model cards. — RSS: https://blog.markkulab.net/local-ai-lab/feed.xml - [AI 行銷日報](https://blog.markkulab.net/en/category/ai-marketing): Every day, two hosts break down the latest in AI for marketing, advertising, SEO/GEO and content marketing: practical, never sponsored, every story sourced. — RSS: https://blog.markkulab.net/ai-marketing/feed.xml - [AI 資安日報](https://blog.markkulab.net/en/category/ai-security): Every day, two hosts cover the latest where AI meets cybersecurity: LLM and AI agent vulnerabilities, prompt injection, model supply-chain risks and AI-powered attacks, from a defense-first angle with every story sourced. — RSS: https://blog.markkulab.net/ai-security/feed.xml ### Newsletter [Subscribe to the newsletter](https://blog.markkulab.net/en/subscribe) — Be the first to know about new posts. No spam, unsubscribe anytime.