---
title: "Building an AI Bot API in Next.js with Langchain and Open-Source Llama AI Part 1 - Getting Started with Langchain"
description: "An introduction to the basic concepts of the LangChain framework, demonstrating how to integrate an Ollama Llama model in a Next.js API and build an AI application using Prompt Templates and Chains."
canonical_url: "https://blog.markkulab.net/en/post/ai-bot-api-parts-1"
author: "Mark Ku"
author_url: "https://blog.markkulab.net/en/author/mark-ku"
site: "Mark Ku's Tech Notes"
date_published: "2024-10-01 01:01:35 +0800"
category: "AI"
tags: ["llama", "ai", "langchain", "node", "nextjs", "ollama"]
language: "en"
license: "CC BY 4.0"
license_url: "https://creativecommons.org/licenses/by/4.0/"
attribution: "when reusing or quoting, credit the author and link back to the original"
---

# Building an AI Bot API in Next.js with Langchain and Open-Source Llama AI Part 1 - Getting Started with Langchain

## Introduction
Previously, I used [Langchain to optimize SEO for a German e-commerce website](https://blog.markkulab.net/generate-seo-metadata-by-azure-ai/), using Azure AI at the time. However, since the budget always ran out at the end of the month, I started looking for alternatives and eventually found Facebook's open-source AI model, Llama, as a replacement.

I had actually built a fortune-telling AI bot using Python and Langchain. But considering my team members are primarily proficient in JavaScript, not Python, I decided to try developing this AI-related feature using Next.js's API to unify our tech stack and make it easier for everyone to back each other up. This ultimately became an AI Agent Bot API.

![Line AI Bot](https://blog.markkulab.net/content/markku/posts/ai-bot-api-parts-1/images/line-ai-bot.png)

## A Quick Review - What is Langchain
Langchain is a framework that can integrate various large AI models, making it easy to call different types of large language models. The framework provides templates, parsers, dynamic routing, and can be integrated with backend languages like Python, Node.js, and C#.

## Why Should Applications Interact with AI?
We all know that AI is exceptionally good with text and highly efficient. When applications interact with AI, it opens up many extended applications, such as:
* 1. Checking if reviews are positive or negative.
* 2. SEO generation / translation / marketing copy.
* 4. Product consultation / product recommendation bots.
* 5. Code review tools.
* 6. Checking documents for typos / verifying the correctness of monetary amounts.
* 8. Internal domain-specific response bots. For example, HR leave policies, product design workflows.

## So Why Use Langchain?
If your use case is simple, involving only a single question-and-answer exchange with a large language model, then I think you might not need Langchain. However, if you want the AI to decide what to do based on the semantics of the user's input, or if you need to interact with the LLM multiple times within a single request—for instance, checking your own vector database and then interacting with an external API if no relevant data is found—then Langchain is a perfect fit for this scenario.

![Why use Langchain](https://blog.markkulab.net/content/markku/posts/ai-bot-api-parts-1/images/why-use-langchain.png)

## Prerequisites
* Install Ollama ([refer to my previous article](https://blog.markkulab.net/build-your-ollama-ai-with-hardware-acceleration/))
* Have a Next.js project ready
* Install the relevant Langchain packages. This series of articles uses version (0.3).

```
npm install langchain @langchain/azure-openai @langchain/ollama @langchain/openai --save
```
## Example 1: Generating Store Names with AI
## Originally, generating a shoe store name using Azure OpenAI

```
import { AzureChatOpenAI } from '@langchain/openai';
import { NextApiRequest, NextApiResponse } from 'next';

export default async function handler(req: NextApiRequest, res: NextApiResponse) {
    const model = new AzureChatOpenAI({
        azureOpenAIApiKey: process.env.AZURE_OPENAI_API_KEY,
        azureOpenAIApiDeploymentName: 'gpt-4',
        azureOpenAIApiInstanceName: 'your-instance-name',
        azureOpenAIApiVersion: '2023-03-15-preview',
        temperature: 0, // 用來調控AI 的隨機性
        maxTokens: 500,
    });

    try {
        const result = await model.invoke('為一個鞋店起一個好的名字。');
        res.status(200).json({ result });
    } catch (error) {
        res.status(500).json({ error: 'Failed to generate name' });
    }
}


```
Now, generating a shoe store name using ChatOllama
Isn't Langchain convenient? It has a well-designed interface. You can just swap `AzureChatOpenAI` with `ChatOllama` to change the model without altering the rest of the code.
```
import { ChatOllama } from '@langchain/ollama';
import { NextApiRequest, NextApiResponse } from 'next';

export default async function handler(req: NextApiRequest, res: NextApiResponse) {
    const model = new ChatOllama({
        model: 'llama3.2',
        temperature: 0,
        maxRetries: 2,
        baseUrl: 'http://localhost:11434',
    });

    try {
        const result = await model.invoke('為一個鞋店起一個好的名字。');
        const content = JSON.stringify(result.content);
        res.status(200).json(content);
    } catch (error) {
        res.status(500).json({ error: 'Failed to generate name' });
    }
}

```
## Example 2: Using chatTemplate

```
import { ChatOllama } from '@langchain/ollama';
import { NextApiRequest, NextApiResponse } from 'next';

export default async function handler(req: NextApiRequest, res: NextApiResponse) {
    const { brandName, modelVersion = 'llama3.2', temperature = 0 } = req.body;

    const model = new ChatOllama({
        model: modelVersion,
        temperature,
        maxRetries: 2,
        baseUrl: 'http://localhost:11434',
    });

    try {
        const result = await model.invoke(`為一個鞋店起一個好的名字：${brandName}`);
        const content = JSON.stringify(result.content);
        res.status(200).json(content);
    } catch (error) {
        res.status(500).json({ error: 'Failed to generate name' });
    }
}
```
## Example 3: Using `pipe` to make the AI do more things, easily enabling multiple back-and-forth calls

```
import { StringOutputParser } from '@langchain/core/output_parsers';
import { ChatPromptTemplate } from '@langchain/core/prompts';
import { ChatOllama } from '@langchain/ollama';
import { NextApiRequest, NextApiResponse } from 'next/types';

// 定義 API route
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
    const { modelVersion = 'llama3.2', temperature = 0 } = req.body;

    try {
     const question = 'What is the weather in Taipei?';
        // 1. 初始化模型
        const model = new ChatOllama({
            model: modelVersion,
            temperature,
            maxRetries: 2,
            baseUrl: 'http://localhost:11434',
        });

        // 2. 定義一個處理步驟的 prompt
        const prompt = ChatPromptTemplate.fromMessages([['human', 'Ask the weather for {question}']]);

        // 2. 定義 output parser
        const outputParser = new StringOutputParser();

        // 3. 使用 pipe 來串聯步驟：prompt -> model -> parser
        const pipeChain = prompt.pipe(model).pipe(outputParser);

        // 4. 執行 chain 並取得結果
        const weatherResponse = await pipeChain.invoke({ question });

        // 5. 回應 API 結果
        res.status(200).json({ weather: weatherResponse });
    } catch (error) {
        
        res.status(500).json({ message: 'Internal Server Error' });
    }
}

```
## Example 4: Using a chain to have the AI call an API for me

```
import { ChatOllama } from '@langchain/ollama';
import { APIChain } from 'langchain/chains';
import { NextApiRequest, NextApiResponse } from 'next/types';

const OPEN_METEO_DOCS = `api document ...`;

// Next.js API route
export default async function handler(req: NextApiRequest, res: NextApiResponse) {
    const { question, modelVersion = 'llama3.2', temperature = 0 } = req.body;

    try {
        const model = new ChatOllama({
            model: modelVersion,
            temperature,
            maxRetries: 2,
            baseUrl: 'http://localhost:11434',
        });

        const chain = APIChain.fromLLMAndAPIDocs(model, OPEN_METEO_DOCS, {
            headers: {
                // API-specific headers if required
            },
        });

        const weatherResponse = await chain.invoke({ question });

        res.status(200).json({ weather: weatherResponse });
    } catch (error) {	
        console.error('Error fetching weather:', error);
        res.status(500).json({ message: 'Internal Server Error' });
    }
}

```

## Related Articles in This Series
* [Building an AI Bot API in Next.js with Langchain and Open-Source Llama AI Part 1 - Getting to Know Langchain](https://blog.markkulab.net/ai-bot-api-parts-1/)
* [Building an AI Bot API in Next.js with Langchain and Open-Source Llama AI Part 2 - Creating an AI Agent with Memory](https://blog.markkulab.net/ai-bot-api-parts-2/)
* [Building an AI Bot API in Next.js with Langchain and Open-Source Llama AI Part 3 - Adding a Vector Database to Give the AI an Extra Brain](https://blog.markkulab.net/ai-bot-api-parts-3/)
* [Building an AI Bot API in Next.js with Langchain and Open-Source Llama AI Part 4 - AI Product Recommendation API](https://blog.markkulab.net/ai-bot-api-parts-4/)

---

## About this article and its author

Originally published on [Mark Ku's Tech Notes](https://blog.markkulab.net/en/post/ai-bot-api-parts-1)

License: [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) — when reusing or quoting, credit the author and link back to the original

### About the author

**[Mark Ku](https://blog.markkulab.net/en/author/mark-ku)** — Software Solution Provider

- 10+ years senior software engineer, now an AI Builder
- Focused on large-platform architecture — North-American e-commerce, AI SaaS subscription billing
- Combining AI Agents and automation to build evolvable product foundations

### Free tools built by the author

All of these are free to use:

- [Free PDF Sign Tool](https://blog.markkulab.net/en/tools/pdf-sign): Online PDF sign tool — draw, type, or upload a signature, then drag, resize, and download. Everything runs in your browser; nothing is uploaded.
- [VS Code Refactory](https://blog.markkulab.net/en/tools/refactory): Refactory is a VS Code refactoring extension: 34 actions plus a 37-rule code-smell inspection layer with a Code Health dashboard, across 18 languages, backed by 534 tests. It learns your repo's conventions: where interfaces live, where DI is registered, whether 'use client' belongs. It ranks files by git churn × complexity so you know what to fix first, and hands any smell to the Claude Code already on your machine. Free to use, and your source never leaves your computer.
- [DB-Kit Database Manager](https://blog.markkulab.net/en/tools/db-kit): DB-Kit is a lightweight, cross-platform database manager built with Tauri + Rust + React. Manage MySQL, MariaDB, PostgreSQL, SQL Server, Oracle, SQLite, MongoDB, Redis, Kafka, Elasticsearch and RabbitMQ from one consistent interface: passwords encrypted in the OS keychain, SSH tunnels, full CRUD, a visual query builder, stacked multi-statement result sets, cross-connection data transfer and compare/sync, Excel / CSV import & export, visualized execution plans, ER diagrams, scheduled backups, SQL stress testing with p50–p99 latency percentiles, a 15-rule SQL review engine, Kafka message browsing with monitoring & alerts, a bilingual UI (Traditional Chinese / English), a built-in AI assistant (natural-language SQL, AI review and tuning advice) and the dbk CLI. Free and open source (MIT), with installers for Windows, macOS and Linux.
- [VS Code Super Mermaid](https://blog.markkulab.net/en/tools/super-mermaid): Super Mermaid is a VS Code extension for beautiful Mermaid diagrams out of the box: auto-colored live preview, mouse pan & zoom, high-res PNG / SVG export, 21 templates and multiple themes. Free and open source (MIT).
- [React Super Mermaid](https://blog.markkulab.net/en/tools/react-super-mermaid): react-super-mermaid is an open-source React component library: render beautiful Mermaid diagrams with a single <MermaidViewer>, with built-in colorful / sketch themes, pan & zoom, in-diagram search, and high-res SVG / PNG export. Lightweight, SSR-safe, fully typed. Free and open source (MIT).
- [Jira / Confluence Super Mermaid](https://blog.markkulab.net/en/tools/jira-super-mermaid): An Atlassian Forge app: write Mermaid syntax directly inside a Jira issue or a Confluence page and get flowcharts, sequence diagrams, state machines and Gantt charts. 11 diagram types, SVG / PNG export, light and dark themes, full CJK support. Runs on Atlassian: your diagrams live in your own site and the app calls no third-party service. Free, coming soon to the Atlassian Marketplace.
- [Mermaid Live Preview](https://blog.markkulab.net/en/tools/mermaid-preview): Write Mermaid in your browser, see it render instantly, and share the whole diagram as a single link. No sign-up, nothing uploaded to a server, and mermaid.live share links work as-is.
- [React Intl Phone Number](https://blog.markkulab.net/en/tools/react-intl-phone-number): react-intl-phone-number is an open-source React component: framework-agnostic and antd-free, with E.164 in/out, a searchable flag / country-code dropdown, configurable validation levels (strict / mobile-strict / loose), themeable CSS, and i18n — phone logic powered by google-libphonenumber. Lightweight and fully typed. Free and open source (MIT).
- [Uptime Kuma Cluster](https://blog.markkulab.net/en/tools/uptime-kuma-cluster): Turn single-node Uptime Kuma into a highly available cluster: OpenResty + Lua smart load balancing, shared MariaDB state, health checks and automatic failover, plus cluster-management REST APIs. One Docker Compose command to start. Free and open source (MIT).
- [Special Education](https://blog.markkulab.net/en/education): Learning materials crafted for special education students

### Daily podcasts

- [Mark's Tech Insights — Daily AI News](https://blog.markkulab.net/en/category/tech-news): Daily curated AI and tech trends. Catch the latest developments via audio summaries — covering AI applications, software architecture, DevOps, and engineering practice. — RSS: https://blog.markkulab.net/feed.xml
- [AI股市蝦聊](https://blog.markkulab.net/en/category/ai-stock-chat): Every trading day, an AI-analyzed take on the Taiwan stock market, delivered as a two-host conversation covering the session and the next-day outlook. — RSS: https://blog.markkulab.net/ai-stock-chat/feed.xml
- [開源好物週報](https://blog.markkulab.net/en/category/open-source-weekly): A weekly two-host pick of free open-source tools surfaced from real Hacker News, GitHub, and Reddit buzz — what pain they solve and the fastest way to get started. — RSS: https://blog.markkulab.net/open-source-weekly/feed.xml

### Newsletter

[Subscribe to the newsletter](https://blog.markkulab.net/en/subscribe) — Be the first to know about new posts. No spam, unsubscribe anytime.