--- title: "Building an AI Bot API in Next.js with Langchain and Open-Source Llama AI Part 2 - Creating an AI Agent with Memory" description: "Demonstrates how to use the Langchain AgentExecutor with custom tools and SerpAPI, and integrate Redis to add memory to an AI Agent, creating a stateful conversational bot." canonical_url: "https://blog.markkulab.net/en/post/ai-bot-api-parts-2" author: "Mark Ku" author_url: "https://blog.markkulab.net/en/author/mark-ku" site: "Mark Ku's Tech Notes" date_published: "2024-10-13 01:01:35 +0800" category: "AI" tags: ["llama", "ai", "langchain", "redis", "node", "nextjs"] language: "en" license: "CC BY 4.0" license_url: "https://creativecommons.org/licenses/by/4.0/" attribution: "when reusing or quoting, credit the author and link back to the original" --- # Building an AI Bot API in Next.js with Langchain and Open-Source Llama AI Part 2 - Creating an AI Agent with Memory ## Introduction I [previously tried switching from the Python version of Langchain to the Node.js version](https://blog.markkulab.net/ai-bot-api-parts-1/), but the documentation for Langchain's Node.js port is quite limited. This made the transition challenging because even when documentation existed, it often omitted parameters, causing the program to crash. In the end, I still had to cross-reference the Python version. ## Prerequisites * Redis server * Ollama * A Next.js project ## The Agent Component First, let's look at the agent component (AgentExecutor). The agent component is a core part of the Langchain framework. Based on user input, the agent can dynamically decide which tool to use and provide a final answer. ### Initializing the Agent Component ``` // 初化模式 const llm = new ChatOllama({ model: 'llama3.2', temperature: 0, maxRetries: 2, baseUrl: 'http://localhost:11434', }); const tools = []; // 工具清單 // 初始化聊天範本 const prompt = ChatPromptTemplate.fromMessages([ ['system', 'You are a helpful assistant'], // default role ['placeholder', '{chat_history}'], // default role ['human', '{input}'], ['placeholder', '{agent_scratchpad}'], ]); const agent = createToolCallingAgent({ llm, tools, prompt, }); const result = await agentExecutor.invoke({ input: 'your question ' }); ``` ## Tools (Dynamic Routing) As mentioned earlier, when the agent component receives a user's question, it will use the tool's description to let the AI decide which tool to use, execute the tool, and finally return the result to the user. ### Customizing Tools ``` import { tool } from '@langchain/core/tools'; ... const greetingTool = tool( async ({ input }: { input: string }) => { return input + ' =='; }, { name: 'greetingTool', description: 'if some one say hello', schema: z.object({ input: z.string(), }) } ); ... ``` Parameter Descriptions * `name`: The name of the tool. * `description`: A description of the tool's functionality. The AI uses this description to decide which tool to use. * `func`: The core functionality of the tool. Here, it's an asynchronous function that returns a value based on the input. * `schema`: Uses Zod to define the input parameter format for the tool. ### Using a Third-Party Tool - SerpAPI SerpAPI is an API specifically designed for interacting with search engines, particularly Google Search. SerpAPI automates the web scraping process, freeing developers from having to manage the parsing and data extraction of search engine pages themselves, thus simplifying the process of obtaining search results. ``` import { SerpAPI } from '@langchain/community/tools/serpapi'; let serpApiTool = new SerpAPI('your key ', { location: 'Austin,Texas,United States', // ，代表你要模擬的搜尋位置。在這裡，它指定了搜索應基於美國德州奧斯汀的位置，這可以影響搜尋結果的區域相關性。 hl: 'en', // 語言參數，代表要以哪種語言來顯示搜尋結果。'en' 表示結果會以英文顯示。 gl: 'us', // 國家/地區代碼，用來設定你希望搜尋結果的區域依據。'us' 表示結果會根據美國的地區來產生。 }), ``` ## Extending the AI Agent's Memory Large language models themselves do not have memory. The reason they can understand the context of a conversation is because the entire discussion history is passed back to the LLM with each new query. Langchain's memory feature also operates on this principle. By storing each user query in Redis, we can equip our AI Bot with memory capabilities. ### Installing Related Packages - [Reference Documentation](https://js.langchain.com/docs/integrations/vectorstores/redis/) ``` npm i @langchain/redis @langchain/core redis @langchain/openai --save ``` ### Using Redis Memory ``` import { RedisChatMessageHistory } from '@langchain/community/stores/message/ioredis'; import { BufferMemory } from 'langchain/memory'; ... const client = new Redis('redis://localhost:30001'); const memory = new BufferMemory({ chatHistory: new RedisChatMessageHistory({ sessionId: 'sessionId:' + new Date().toISOString(), sessionTTL: 300, client, }), aiPrefix: 'ollama', outputKey: 'output', // 沒有會爆錯 memoryKey: 'chat_history', // 沒有會爆錯 inputKey: 'input', // 沒有會爆錯 returnMessages: true, const agentExecutor = new AgentExecutor({ agent, tools, verbose: true, // enable debug handleParsingErrors: true, memory, // redis 記憶功能 returnIntermediateSteps: true, }); ... ``` ## Complete Code (Next.js API) ``` import { RedisChatMessageHistory } from '@langchain/community/stores/message/ioredis'; import { Calculator } from '@langchain/community/tools/calculator'; import { SerpAPI } from '@langchain/community/tools/serpapi'; import { ChatPromptTemplate } from '@langchain/core/prompts'; import { tool } from '@langchain/core/tools'; import { ChatOllama } from '@langchain/ollama'; import Redis from 'ioredis'; import { AgentExecutor, createToolCallingAgent } from 'langchain/agents'; import { BufferMemory } from 'langchain/memory'; import { NextApiRequest, NextApiResponse } from 'next/types'; import { z } from 'zod'; export default async function handler(req: NextApiRequest, res: NextApiResponse) { if (req.method !== 'GET') { return res.status(405).json({ message: 'Only GET requests are allowed' }); } try { const magicTool = tool( async ({ input }: { input: number }) => { return `${input + 2}`; }, { name: 'magic_function', description: 'Applies a magic function to an input.', schema: z.object({ input: z.number(), }), } ); const llm = new ChatOllama({ model: 'llama3.2', temperature: 0, maxRetries: 2, baseUrl: 'http://localhost:11434', // other params... }); const tools = [ magicTool, // custom tool new Calculator(), new SerpAPI('your serp key', { location: 'Austin,Texas,United States', hl: 'en', gl: 'us', }), ]; const prompt = ChatPromptTemplate.fromMessages([ ['system', 'You are a helpful assistant'], // default role ['placeholder', '{chat_history}'], // default role ['human', '{input}'], ['placeholder', '{agent_scratchpad}'], ]); const agent = createToolCallingAgent({ llm, tools, prompt, }); const client = new Redis('redis://localhost:30001'); const memory = new BufferMemory({ chatHistory: new RedisChatMessageHistory({ sessionId: 'sessionId:' + new Date().toISOString(), sessionTTL: 300, client, }), aiPrefix: 'ollama', outputKey: 'output', // 沒有會爆錯 memoryKey: 'chat_history', // 沒有會爆錯 inputKey: 'input', // 沒有會爆錯 returnMessages: true, }); const agentExecutor = new AgentExecutor({ agent, tools, verbose: true, // enable debug handleParsingErrors: true, memory, // redis 記憶功能 returnIntermediateSteps: true, }); const result = await agentExecutor.invoke({ input: 'what is the value of magic_function(3)?' }); const serpResult2 = await agentExecutor.invoke({ input: 'what is the Pokomon?' }); const result3 = await agentExecutor.invoke({ input: 'what is 5 + 2 *5 =' }); const result4 = await agentExecutor.invoke({ input: 'hello', outputKey: 'key1' }); const result5 = await agentExecutor.invoke({ input: '幫我推薦電腦' }); return res.status(200).json({ result, serpResult2, result3, result4, result5 }); } catch (error) { return res.status(500).json({ message: 'Internal server error' }); } } ``` ## References * [Official Documentation](https://js.langchain.com/docs/how_to/migrate_agent/#basic-usage) * [Official API Documentation](https://v03.api.js.langchain.com/hierarchy.html#@langchain/community.tools/serpapi.SerpAPI) * [Building AI Solutions with LangChain and Node.js: A Comprehensive Guide](https://medium.com/widle-studio/building-ai-solutions-with-langchain-and-node-js-a-comprehensive-guide-widle-studio-4812753aedff) ## Related Articles in This Series * [Building an AI Bot API with Langchain and Open-Source Llama AI in Next.js Part 1 - Getting Started with Langchain](https://blog.markkulab.net/ai-bot-api-parts-1/) * [Building an AI Bot API with Langchain and Open-Source Llama AI in Next.js Part 2 - Creating an AI Agent with Memory](https://blog.markkulab.net/ai-bot-api-parts-2/) * [Building an AI Bot API with Langchain and Open-Source Llama AI in Next.js Part 3 - Adding a Vector Database to Give the AI an Extra Brain](https://blog.markkulab.net/ai-bot-api-parts-3/) * [Building an AI Bot API with Langchain and Open-Source Llama AI in Next.js Part 4 - AI Product Recommendation API](https://blog.markkulab.net/ai-bot-api-parts-4/) --- ## About this article and its author Originally published on [Mark Ku's Tech Notes](https://blog.markkulab.net/en/post/ai-bot-api-parts-2) License: [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/) — when reusing or quoting, credit the author and link back to the original ### About the author **[Mark Ku](https://blog.markkulab.net/en/author/mark-ku)** — Software Solution Provider - 10+ years senior software engineer, now an AI Builder - Focused on large-platform architecture — North-American e-commerce, AI SaaS subscription billing - Combining AI Agents and automation to build evolvable product foundations ### Free tools built by the author All of these are free to use: - [Free PDF Sign Tool](https://blog.markkulab.net/en/tools/pdf-sign): Online PDF sign tool — draw, type, or upload a signature, then drag, resize, and download. Everything runs in your browser; nothing is uploaded. - [VS Code Refactory](https://blog.markkulab.net/en/tools/refactory): Refactory is a VS Code refactoring extension: 34 actions plus a 37-rule code-smell inspection layer with a Code Health dashboard, across 18 languages, backed by 534 tests. It learns your repo's conventions: where interfaces live, where DI is registered, whether 'use client' belongs. It ranks files by git churn × complexity so you know what to fix first, and hands any smell to the Claude Code already on your machine. Free to use, and your source never leaves your computer. - [DB-Kit Database Manager](https://blog.markkulab.net/en/tools/db-kit): DB-Kit is a lightweight, cross-platform database manager built with Tauri + Rust + React. Manage MySQL, MariaDB, PostgreSQL, SQL Server, Oracle, SQLite, MongoDB, Redis, Kafka, Elasticsearch and RabbitMQ from one consistent interface: passwords encrypted in the OS keychain, SSH tunnels, full CRUD, a visual query builder, stacked multi-statement result sets, cross-connection data transfer and compare/sync, Excel / CSV import & export, visualized execution plans, ER diagrams, scheduled backups, SQL stress testing with p50–p99 latency percentiles, a 15-rule SQL review engine, Kafka message browsing with monitoring & alerts, a bilingual UI (Traditional Chinese / English), a built-in AI assistant (natural-language SQL, AI review and tuning advice) and the dbk CLI. Free and open source (MIT), with installers for Windows, macOS and Linux. - [VS Code Super Mermaid](https://blog.markkulab.net/en/tools/super-mermaid): Super Mermaid is a VS Code extension for beautiful Mermaid diagrams out of the box: auto-colored live preview, mouse pan & zoom, high-res PNG / SVG export, 21 templates and multiple themes. Free and open source (MIT). - [React Super Mermaid](https://blog.markkulab.net/en/tools/react-super-mermaid): react-super-mermaid is an open-source React component library: render beautiful Mermaid diagrams with a single , with built-in colorful / sketch themes, pan & zoom, in-diagram search, and high-res SVG / PNG export. Lightweight, SSR-safe, fully typed. Free and open source (MIT). - [Jira / Confluence Super Mermaid](https://blog.markkulab.net/en/tools/jira-super-mermaid): An Atlassian Forge app: write Mermaid syntax directly inside a Jira issue or a Confluence page and get flowcharts, sequence diagrams, state machines and Gantt charts. 11 diagram types, SVG / PNG export, light and dark themes, full CJK support. Runs on Atlassian: your diagrams live in your own site and the app calls no third-party service. Free, coming soon to the Atlassian Marketplace. - [Mermaid Live Preview](https://blog.markkulab.net/en/tools/mermaid-preview): Write Mermaid in your browser, see it render instantly, and share the whole diagram as a single link. No sign-up, nothing uploaded to a server, and mermaid.live share links work as-is. - [React Intl Phone Number](https://blog.markkulab.net/en/tools/react-intl-phone-number): react-intl-phone-number is an open-source React component: framework-agnostic and antd-free, with E.164 in/out, a searchable flag / country-code dropdown, configurable validation levels (strict / mobile-strict / loose), themeable CSS, and i18n — phone logic powered by google-libphonenumber. Lightweight and fully typed. Free and open source (MIT). - [Uptime Kuma Cluster](https://blog.markkulab.net/en/tools/uptime-kuma-cluster): Turn single-node Uptime Kuma into a highly available cluster: OpenResty + Lua smart load balancing, shared MariaDB state, health checks and automatic failover, plus cluster-management REST APIs. One Docker Compose command to start. Free and open source (MIT). - [Special Education](https://blog.markkulab.net/en/education): Learning materials crafted for special education students ### Daily podcasts - [Mark's Tech Insights — Daily AI News](https://blog.markkulab.net/en/category/tech-news): Daily curated AI and tech trends. Catch the latest developments via audio summaries — covering AI applications, software architecture, DevOps, and engineering practice. — RSS: https://blog.markkulab.net/feed.xml - [AI股市蝦聊](https://blog.markkulab.net/en/category/ai-stock-chat): Every trading day, an AI-analyzed take on the Taiwan stock market, delivered as a two-host conversation covering the session and the next-day outlook. — RSS: https://blog.markkulab.net/ai-stock-chat/feed.xml - [開源好物週報](https://blog.markkulab.net/en/category/open-source-weekly): A weekly two-host pick of free open-source tools surfaced from real Hacker News, GitHub, and Reddit buzz — what pain they solve and the fastest way to get started. — RSS: https://blog.markkulab.net/open-source-weekly/feed.xml ### Newsletter [Subscribe to the newsletter](https://blog.markkulab.net/en/subscribe) — Be the first to know about new posts. No spam, unsubscribe anytime.