--- title: "A Brief Discussion on High-Concurrency Web Architecture Design" description: "Key architectural design points for high-concurrency websites, covering aspects such as frontend optimization, backend distributed architecture, database sharding, caching strategies, load balancing, and security protection." canonical_url: "https://blog.markkulab.net/en/post/how-to-build-high-concurrency-website" author: "Mark Ku" author_url: "https://blog.markkulab.net/en/author/mark-ku" site: "Mark Ku's Tech Notes" date_published: "2024-11-10 20:01:35 +0800" category: "Learning" tags: ["architecture", "concurrency", "load balancer", "cache", "redis", "kubernetes", "database"] language: "en" license: "CC BY 4.0" license_url: "https://creativecommons.org/licenses/by/4.0/" attribution: "when reusing or quoting, credit the author and link back to the original" --- # A Brief Discussion on High-Concurrency Web Architecture Design ## Introduction I've compiled some notes on high-traffic website architecture design. The common theme seems to be "divide and conquer"—breaking down large components into smaller ones. This involves splitting up tasks or bottlenecks to make the system or services easy to scale horizontally, avoiding overload at a single point which can create a system bottleneck. This is actually similar to large organizations in the real world; when they hit a growth bottleneck, they have to restructure. ### Frontend * [Frontend optimizations, such as optimizing images, JS, CSS, lazy loading, and static page generation](https://blog.markkulab.net/enhance-nextjs-performance/). * Use a CDN to host static content, reducing the I/O pressure on the main server. ### Backend * Adopt mature programming language frameworks and write asynchronous code to maximize CPU and I/O utilization. Don't send unnecessary data to the frontend. If data can be fetched in a single request, don't use multiple requests. * Adopt a distributed architecture - break down the application into multiple independent services to prevent one service from getting stuck and causing a system bottleneck. For example: a payment module, a logistics module. * Use queuing technologies - when the system receives a large number of requests in a short time, a queue can temporarily store these requests and then process them gradually according to the system's capacity, avoiding sudden spikes in usage that could impact the system. * [You can use certain algorithmic patterns to handle sudden performance bottlenecks: circuit breaking, bulkheading, retries, graceful degradation, timeouts, and rate limiting (token bucket, waiting room, leaky bucket)](https://blog.csdn.net/github_38592071/article/details/140196235) ## Database * Database performance optimization. For example: creating indexes, optimizing queries, denormalization, reducing joins, vertical table partitioning, or using a caching server to cache data that doesn't change often, avoiding fetching it from the DB every time. * The database can be sharded or partitioned into multiple databases to distribute data processing. * Read-write splitting - Separate database reads and writes, using [Master-Slave Replication](https://ithelp.ithome.com.tw/articles/10267454) to handle a large volume of read requests. * Use NoSQL databases. NoSQL is effective for partitioning and sharding, has horizontal scaling capabilities, and is particularly suitable for handling large-scale data, such as user profiles. * [Ensure transactional consistency using locking mechanisms](https://ouch1978.github.io/blog/2022/09/21/common-locks-in-programming-world) ## Caching * Frontend Caching - Enable client-side caching. If a file hasn't changed, the client doesn't need to request it again. * Backend Caching - For data that is frequently queried but rarely changes, a caching server can be used, or use a [Redis Cluster](https://isdaniel.github.io/redis-cluster-introduce-01/) for data sharding (suitable for scenarios with contiguous Redis keys). ## Operations * Load Balancer - The core principle of load balancing is to distribute traffic and tasks across multiple servers using a reverse proxy mechanism. If the QPS is less than 10k, it's recommended to use a software reverse proxy (like Nginx or YARP). Hardware load balancers have specially designed chips that can handle much higher traffic loads. When QPS exceeds 10k, you should consider purchasing a hardware load balancer, such as an [F5](https://www.f5.com/glossary/load-balancer). * Adopt a modern system operations architecture (K8s) to scale out and in based on QPS or system resources. * Service Monitoring (Health Check) - Promptly detect and respond to service anomalies to ensure stable operation. For example: [Uptime-Kuma](https://blog.markkulab.net/2021/12/11/application-monitor-uptime-kuma/) * Application Observability - Provides visualization and analysis of application performance, facilitating problem diagnosis and performance optimization. For example: [Signoz](https://www.cnblogs.com/hacker-linner/p/17581720.html), Prometheus * Log Server - Centralizes log management and analysis, making it easier to find and trace the source of issues. For example: Grafana, SEQ ## Security * Use a website protection service like Cloudflare to prevent malicious attacks, such as [DDoS](https://blog.markkulab.net/prevent-ddos-part2/). * [Prevent CSRF attacks to enhance system security.](https://www.telerik.com/blogs/protecting-nextjs-applications-cross-site-request-forgery-csrf-attacks) ## Finally To be honest, there are some features here that I haven't implemented myself, but I've compiled this list to give myself a conceptual map. This way, when I encounter these problems, I'll at least have a starting point for a solution. However, not every website needs this kind of architecture. A design that is too granular can increase maintenance costs. Instead, it's more appropriate to consider future business volume when evaluating this type of website architecture. ## References * [My former colleague, Ouch](https://ouch1978.github.io/) * [tixCraft Ticketing System](https://aws.amazon.com/tw/solutions/case-studies/tixcraft/?fbclid=IwZXh0bgNhZW0CMTAAAR1OkXmUsFGJZ4x086DzwYbK4Ox1JEuJWDA6umSIx2edlKKucWm4jWU6PGY_aem_aztLGVHLLc9EvEnDe6WTMg) --- ## About this article and its author Originally published on [Mark Ku's Tech Notes](https://blog.markkulab.net/en/post/how-to-build-high-concurrency-website) License: [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) — when reusing or quoting, credit the author and link back to the original ### About the author **[Mark Ku](https://blog.markkulab.net/en/author/mark-ku)** — Software Solution Provider - 10+ years senior software engineer, now an AI Builder - Focused on large-platform architecture — North-American e-commerce, AI SaaS subscription billing - Combining AI Agents and automation to build evolvable product foundations ### Free tools built by the author All of these are free to use: - [Free PDF Sign Tool](https://blog.markkulab.net/en/tools/pdf-sign): Online PDF sign tool — draw, type, or upload a signature, then drag, resize, and download. Everything runs in your browser; nothing is uploaded. - [VS Code Refactory](https://blog.markkulab.net/en/tools/refactory): Refactory is a VS Code refactoring extension: 34 actions plus a 37-rule code-smell inspection layer with a Code Health dashboard, across 18 languages, backed by 534 tests. It learns your repo's conventions: where interfaces live, where DI is registered, whether 'use client' belongs. It ranks files by git churn × complexity so you know what to fix first, and hands any smell to the Claude Code already on your machine. Free to use, and your source never leaves your computer. - [DB-Kit Database Manager](https://blog.markkulab.net/en/tools/db-kit): DB-Kit is a lightweight, cross-platform database manager built with Tauri + Rust + React. Manage MySQL, MariaDB, PostgreSQL, SQL Server, Oracle, SQLite, MongoDB, Redis, Kafka, Elasticsearch and RabbitMQ from one consistent interface: passwords encrypted in the OS keychain, SSH tunnels, full CRUD, a visual query builder, stacked multi-statement result sets, cross-connection data transfer and compare/sync, Excel / CSV import & export, visualized execution plans, ER diagrams, scheduled backups, SQL stress testing with p50–p99 latency percentiles, a 15-rule SQL review engine, Kafka message browsing with monitoring & alerts, a bilingual UI (Traditional Chinese / English), a built-in AI assistant (natural-language SQL, AI review and tuning advice) and the dbk CLI. Free and open source (MIT), with installers for Windows, macOS and Linux. - [VS Code Super Mermaid](https://blog.markkulab.net/en/tools/super-mermaid): Super Mermaid is a VS Code extension for beautiful Mermaid diagrams out of the box: auto-colored live preview, mouse pan & zoom, high-res PNG / SVG export, 21 templates and multiple themes. Free and open source (MIT). - [React Super Mermaid](https://blog.markkulab.net/en/tools/react-super-mermaid): react-super-mermaid is an open-source React component library: render beautiful Mermaid diagrams with a single , with built-in colorful / sketch themes, pan & zoom, in-diagram search, and high-res SVG / PNG export. Lightweight, SSR-safe, fully typed. Free and open source (MIT). - [Jira / Confluence Super Mermaid](https://blog.markkulab.net/en/tools/jira-super-mermaid): An Atlassian Forge app: write Mermaid syntax directly inside a Jira issue or a Confluence page and get flowcharts, sequence diagrams, state machines and Gantt charts. 11 diagram types, SVG / PNG export, light and dark themes, full CJK support. Runs on Atlassian: your diagrams live in your own site and the app calls no third-party service. Free, coming soon to the Atlassian Marketplace. - [Mermaid Live Preview](https://blog.markkulab.net/en/tools/mermaid-preview): Write Mermaid in your browser, see it render instantly, and share the whole diagram as a single link. No sign-up, nothing uploaded to a server, and mermaid.live share links work as-is. - [React Intl Phone Number](https://blog.markkulab.net/en/tools/react-intl-phone-number): react-intl-phone-number is an open-source React component: framework-agnostic and antd-free, with E.164 in/out, a searchable flag / country-code dropdown, configurable validation levels (strict / mobile-strict / loose), themeable CSS, and i18n — phone logic powered by google-libphonenumber. Lightweight and fully typed. Free and open source (MIT). - [Uptime Kuma Cluster](https://blog.markkulab.net/en/tools/uptime-kuma-cluster): Turn single-node Uptime Kuma into a highly available cluster: OpenResty + Lua smart load balancing, shared MariaDB state, health checks and automatic failover, plus cluster-management REST APIs. One Docker Compose command to start. Free and open source (MIT). - [Special Education](https://blog.markkulab.net/en/education): Learning materials crafted for special education students ### Daily podcasts - [Mark's Tech Insights — Daily AI News](https://blog.markkulab.net/en/category/tech-news): Daily curated AI and tech trends. Catch the latest developments via audio summaries — covering AI applications, software architecture, DevOps, and engineering practice. — RSS: https://blog.markkulab.net/feed.xml - [AI股市蝦聊](https://blog.markkulab.net/en/category/ai-stock-chat): Every trading day, an AI-analyzed take on the Taiwan stock market, delivered as a two-host conversation covering the session and the next-day outlook. — RSS: https://blog.markkulab.net/ai-stock-chat/feed.xml - [開源好物週報](https://blog.markkulab.net/en/category/open-source-weekly): A weekly two-host pick of free open-source tools surfaced from real Hacker News, GitHub, and Reddit buzz — what pain they solve and the fastest way to get started. — RSS: https://blog.markkulab.net/open-source-weekly/feed.xml ### Newsletter [Subscribe to the newsletter](https://blog.markkulab.net/en/subscribe) — Be the first to know about new posts. No spam, unsubscribe anytime.