System Goals
The main goal of this system overhaul was to optimize and rewrite the German website. Through this revamp, we not only addressed insurmountable problems with the old system architecture and code that was difficult to maintain and extend, but also hoped to boost sales for the German site. In the future, we can also use the newly written website to quickly expand into other countries.
State of the System at the Time
- The system was outdated, source code was missing, and no one could run it locally.
- It was difficult for users to maintain the existing data.
- The old API system architecture lacked flexibility, couldn't be extended, and couldn't be containerized. It still required manual deployment.
- The old system architecture and its extensibility were also detrimental to SEO optimization.
- The old website couldn't sell pre-built PCs and accessories; everything was crammed into the PC configurator page for sale.
- The old website frequently had failed transactions, requiring manual conversion of temporary orders into actual orders.
- The old website was not modular, used an outdated technical framework that couldn't be containerized, and couldn't be easily adapted for use in other countries.
Challenges and Difficulties
I was originally recruited to plan and rebuild the German website. However, because the US website constantly had issues and there were more urgent projects, the US site only approached stability after a challenging year and a half.
Just when I thought we could finally start on the German website, there were differing major internal goals. Since we only had three developers juggling several projects, running them all concurrently would slow everything down, and context-switching between projects was painful for everyone. After several rounds of communication, we finally agreed to focus on one project at a time.
German E-commerce Website - Architecture Diagram
P.S. Due to our various resource constraints, we bought a theme for the frontend from a foreign website and adapted an open-source .NET Core project for the backend.
The technologies we adopted this time were Next.js App Router + .NET Core 8. I was primarily responsible for analyzing and clarifying requirements, planning the architecture, and breaking down the work into a task list. I also participated in developing core features and independently set up the entire deployment environment for both development and production.
Next, a Look Back at the Technical Details and Special Designs of the German E-commerce Site
Frontend Website (Next.js)
Choosing Between App Router and Pages Router
At that point in time, Next.js was releasing updates very quickly, with significant changes happening monthly. The App Router had just become stable, so I spent a while considering it. Given the benefits of Server Components, which can reduce the JavaScript bundle size for non-interactive components, I tried upgrading the purchased project template from Pages Router to App Router. I realized the side effects for a new project shouldn't be that significant, and upgrading after building the entire project with the Pages Router would likely be even more painful.
To Accelerate Development, We Introduced a Headless CMS - Strapi
In the past, we stored website settings, SEO parameters, and multi-language strings in the backend, manipulating them via JSON data. While this approach allowed for rapid development, it wasn't user-friendly. People without a programming background were confused about the purpose of these JSON settings. Uploading images and configuring them were separate steps, and the site often displayed broken images due to incorrect path settings.
Initially, I wanted to use our self-built Low-code CMS for the German website. However, due to its immaturity and time constraints, we ultimately chose Strapi CMS to solve the data maintenance problem.
Multi-language Support
Initially, the multi-language strings were managed in Strapi. However, during the development phase, we found that adding multiple languages in Strapi was cumbersome for users. We later moved the entire maintenance function to Excel 365 and wrote a script to load the data into Redis. This allows users to maintain multiple languages much more easily and quickly in one place.
Proxy API (/api/proxy/[...path].ts)
Previously, when using Next.js, every time we needed to call a C# API, we had to implement a corresponding API endpoint within Next.js. Now, by using a custom proxy server with the http-proxy-middleware package, we no longer need to spend time maintaining these extra APIs in Next.js, which significantly saves development time and resolves Cross-Origin Resource Sharing (CORS) issues.
import { CookieKeys } from '@/const/keys';
import { clientSideLog } from '@utils/log/client-side-log';
import { LogLevel } from '@utils/log/const-logging';
import { createProxyMiddleware } from 'http-proxy-middleware';
export const config = {
api: {
externalResolver: true, // true, means not use the default settings of Next.js
bodyParser: false, // false, means not use Next.js bodyParser(same as express)
},
};
const proxy: any = createProxyMiddleware({
target: process.env.NEXT_PUBLIC_API,
changeOrigin: true, // change `request domain` to `target`
pathRewrite: { '^/api/proxy': '' }, // remove `/api/proxy` prefix
onProxyReq: relayRequestHeaders,
onError: (err: any, req: any) => {
clientSideLog(
`[Proxy] Error for ${req.method} '${req.url}' to '${process.env.NEXT_PUBLIC_API}': ${err.message}`,
LogLevel.Error
);
},
});
function proxyServer(req: any, res: any) {
proxy(req, res, (err: any) => {
if (err) {
throw err;
}
throw new Error(`Request '${req.url}' is not proxied! We should never reach here!`);
});
}
export default proxyServer;
function relayRequestHeaders(proxyReq: any, req: any, res: any) {
let cookie = req.headers.cookie;
if (cookie) {
proxyReq.setHeader('cookie', cookie);
}
}
⚠️ Note: The above method only works with the Pages Router. The App Router uses the Web Fetch API (
Request/Response), not Node.js'sIncomingMessage/ServerResponse. Sincehttp-proxy-middlewaredirectly manipulates low-level Node.js objects, it cannot be injected. If you're using the App Router, simply wrappingfetchis sufficient; no extra packages are needed.
Solving the Unreleased Redis Connection Pool Issue in Next.js
We used Redis extensively for page rendering. Originally, we used node-redis, but because the connection pool wasn't being released properly, Redis had to be manually restarted from time to time. After asking a colleague to run a stress test with JMeter, we switched to the ioredis package, and the problem never occurred again.
Backend (.NET Core)
Establishing a Unified API Interface and Error Codes
By defining a consistent API interface, the frontend can handle responses uniformly.
{
"count": 0,
"data": null,
"isSuccess": true,
"code": 20000,
"errors": []
}
Issues Extending C# Asynchronous Operations
To achieve maximum utilization of both CPU and I/O, we allowed waiting threads to perform other tasks. Therefore, all our backend APIs were written asynchronously.
However, this asynchronous processing in the backend could lead to chaotic thread handling when querying log files, making them difficult to trace.
To solve this, we used the CorrelationId package. Each request generates a CorrelationId, which is displayed in the response headers. By using this CorrelationId package and configuring it in NLog, we can link related logs together, which helps in tracing urgent issues that occur in the production environment.
Token-based Authentication - JWT Token
The session-based approach we used in the past was not easy to scale horizontally. To enable horizontal scaling for our backend API servers in the future and reduce API coupling, we adopted token-based authentication (JWT) to identify users.
Modular Payment Gateway Integration
We automatically read the config to inject the relevant payment gateway services. By defining all necessary payment interfaces and using Autofac to resolve them, we achieved dependency inversion, easy extensibility, and decoupling. We can now automatically swap different payment logic just by passing in a payment_code.
var paymentConfigs = AppSettingsHelper.GetSection<List<PaymentConfig>>("Payment", "PaymentOptions").Where(x => !x.Disable).ToList();
foreach (var item in paymentConfigs)
{
var serviceTypeName = item.PaymentCode;
Type serviceType = assembly.GetType($"MemberSite.Services.{serviceTypeName}Service");
if (serviceType == null)
{
throw new Exception($"Payment {serviceType} service not found");
}
builder.RegisterType(serviceType)
.WithProperty("PaymentConfig", item)
.WithParameter(new ResolvedParameter(
(pi, ctx) => pi.ParameterType == typeof(IUnitOfWork),
(pi, ctx) => ctx.Resolve<IUnitOfWork>()))
.PropertiesAutowired() // 啟用屬性自動注入
.Named<IPaymentOptionService>(item.PaymentCode);
}
Passing in the payment_code allows for automatically swapping different payment logic based on the provided name.
var params = new Dictionary<string,string>();
var paymentService = _context.ResolveNamed<IPaymentOptionService>(checkout.PaymentTypeCode);
paymentService.pay(params);
Temporary Orders (Using Abstract Classes and Inheritance for Code Reuse, Maintaining a Single Codebase)
Because some foreign payment gateways involve multiple redirects, transaction failures are quite common. In the past, a temporary order was saved before checkout and only written to the real order table upon successful payment. This resulted in two similar sets of logic. Furthermore, the temporary order and the final order shared the same fields.
Inheritance Relationship
[SugarTable("TempOrders")]
public class TempOrders: Orders
{
}
Abstract Class
AbstractInsertOrderService
{
...
public T CreateOrderInstance<T>(Checkout checkout) where T : Orders, new()
{
return new T()
{
Name = checkout.Name,
Email = checkout.Email,
...
};
}
public int InsertOrders<T>(Checkout checkout) where T : Orders, new() // 泛型約束 (Generic Constraint)
{
var order = CreateCustomerInstance<T>(checkout);
return _unitOfWork.GetDbClient().Insertable<T>(order).ExecuteReturnIdentity();
}
...
}
As we all know, abstract classes are used to provide a common implementation for a series of classes. By extracting the shared implementation of temporary and final orders into an abstract class, we used inheritance to reduce two separate pieces of code into a single one, making development and maintenance much easier.
HTTPS Secure Distribution with Nginx
When I first joined the company, the server architecture was based on nested Hyper-V. However, since each server could only use one HTTPS (443) port, every new web application required spinning up a new virtual machine. This was not only inefficient but also wasted a lot of server resources (like CPU, memory, disk space, and external IPs). Today, a single Docker command can deploy multiple services, eliminating the need for OS installation and tedious environment configuration as in the past. Since we couldn't migrate to K8s in the short term, we decided to first simplify the server architecture and improve deployment efficiency using Nginx.
Code Generator
To speed up development, the backend was built on an open-source framework that I organized myself. We used a code generator to create models and the data access layer, accelerating backend development.
Solving SFTP Instability Issues
The old FTP server connection would frequently drop. After trying several Docker-based FTP solutions, I finally set up an open-source SFTP server with a web UI, sftpgo.
Replacing Google CAPTCHA with Cloudflare Turnstile
To prevent bot attacks on pages like login or order lookup, we replaced the original Google CAPTCHA with Cloudflare Turnstile. This saved money (it's free), was easier to integrate, provided a better user experience, and had fewer false positives than Google CAPTCHA v3.
SEO Efforts
Europe places a high value on personal data privacy, so we can't just use member data for marketing. This makes SEO especially important.
- Submitted the website sitemap to Google Search Console and Bing Webmaster Tools.
- Provided metadata tags to help crawlers build indexes more effectively.
- Provided structured data (Google Schema) to crawlers, which helps them better understand our site and potentially gain rich snippets in search results.
- Addressed suggestions and errors from Google Search Console and Bing Webmaster Tools, adjusting the HTML to be more crawler-friendly (e.g., titles 50-60 characters, descriptions 150-160 characters, alt attributes, only one h1 tag...).
- Used Azure AI to generate SEO meta tags for product pages.
- When pages were updated, we used Bing's IndexNow API to notify crawlers to visit our site sooner.
- Set up permanent redirects (HTTP status 308) for old site pages and submitted pages that no longer needed to exist to search engines for removal.
Website ranking, from 4350 to 3745 shortly after launch.
Unit Testing
To avoid breaking existing functionality when modifying features in the future, we wrote tests for both the frontend and backend, specifically for checkout amount and shipping cost calculations. The tests were written following the AAA (Arrange-Act-Assert) pattern.
- Arrange: Initialize the target object, dependencies, and method parameters.
- Act: Call the method of the target object.
- Assert: Verify that the outcome matches the expectation.
Functional Aspects
Customer Support Software
Since the German website was not large-scale, it previously had no customer support software. After looking at the market share of communication apps in Europe, I suggested the German company use WhatsApp as their customer support tool to increase opportunities for user interaction.
Integrating the Stripe Payment Gateway
Integrating Stripe's payment gateway is quite easy. Once you integrate one method, you can enable over a dozen European payment options through their dashboard. This allowed us to quickly expand to new payment methods within two weeks of launch. We could also enable 3D Secure (SMS verification) for more secure transactions.
Another Point to Note is GDPR
According to GDPR regulations, any website visited by individuals within the EU must explicitly inform users about the use of Google Analytics via a cookie banner and obtain their consent. This consent must be specific, voluntary, informed, and retractable.
Fraud Prevention Insurance
Online fraud is a serious problem in Europe and the US. In the past, we had to disable PayPal because chargebacks were too easy. While the US has fraud insurance services to mitigate this, they don't serve the European market. Through a friend of an e-commerce acquaintance, I found out about Riskified. However, due to its high monthly base fee and an additional percentage per transaction, we decided against it for now.
Some Unique Problems Encountered in DevOps Before Go-Live
Since we encountered quite a few issues, I wrote a separate blog post about them.
Two Months After Launch, the Company Plans to Expand to Other Countries
The multi-language support we initially wrote for the frontend ran into issues when expanding to other European countries. Some countries have 5+ official languages and over 10 unofficial ones. We hadn't considered the scalability for such a large number of languages. Therefore, we refactored the language settings into a centralized const in the frontend. We also decoupled some features from specific languages to make them as generic as possible, or extracted non-generic parts into configurations.
Conclusion
The German project didn't actually start in mid-February. Because I've always been interested in architectural planning, I had already begun researching and planning the architecture in my spare time ever since I joined the company. I deduced requirements from the old codebase and thought about how to shorten development time. Despite encountering many small hiccups during development, we successfully launched on July 1st. The German website is running smoothly as planned, and the European market is gradually getting on track.
This successful launch was made possible with the help of many people, and I'm very grateful to everyone. Since we never had a dedicated PM, the design lead and I took on that role. With limited resources, we had to plan and develop simultaneously while also coordinating external matters. Being able to achieve this goal was truly a remarkable feat.
Comments