Introduction

I previously tried switching from the Python version of Langchain to the Node.js version, but the documentation for Langchain's Node.js port is quite limited. This made the transition challenging because even when documentation existed, it often omitted parameters, causing the program to crash. In the end, I still had to cross-reference the Python version.

Prerequisites

• Redis server
• Ollama
• A Next.js project

The Agent Component

First, let's look at the agent component (AgentExecutor). The agent component is a core part of the Langchain framework. Based on user input, the agent can dynamically decide which tool to use and provide a final answer.

Initializing the Agent Component

// 初化模式
const llm = new ChatOllama({
            model: 'llama3.2',
            temperature: 0,
            maxRetries: 2,
            baseUrl: 'http://localhost:11434',
        });

const tools = []; // 工具清單

// 初始化聊天範本
const prompt = ChatPromptTemplate.fromMessages([
            ['system', 'You are a helpful assistant'], // default role
            ['placeholder', '{chat_history}'], // default role
            ['human', '{input}'],
            ['placeholder', '{agent_scratchpad}'],
        ]);

const agent = createToolCallingAgent({
            llm,
            tools,
            prompt,
        });  
        
const result = await agentExecutor.invoke({ input: 'your question ' });        

Tools (Dynamic Routing)

As mentioned earlier, when the agent component receives a user's question, it will use the tool's description to let the AI decide which tool to use, execute the tool, and finally return the result to the user.

Customizing Tools

import { tool } from '@langchain/core/tools';
...
 const greetingTool = tool(
            async ({ input }: { input: string }) => {
                return input + ' ==';
            },
            {
                name: 'greetingTool',
                description: 'if some one say hello',
                schema: z.object({
                    input: z.string(),
                })                
            }
        );
...        

Parameter Descriptions

• name: The name of the tool.
• description: A description of the tool's functionality. The AI uses this description to decide which tool to use.
• func: The core functionality of the tool. Here, it's an asynchronous function that returns a value based on the input.
• schema: Uses Zod to define the input parameter format for the tool.

Using a Third-Party Tool - SerpAPI

SerpAPI is an API specifically designed for interacting with search engines, particularly Google Search. SerpAPI automates the web scraping process, freeing developers from having to manage the parsing and data extraction of search engine pages themselves, thus simplifying the process of obtaining search results.

import { SerpAPI } from '@langchain/community/tools/serpapi';

 let serpApiTool = new SerpAPI('your key ', {
           location: 'Austin,Texas,United States', // ，代表你要模擬的搜尋位置。在這裡，它指定了搜索應基於美國德州奧斯汀的位置，這可以影響搜尋結果的區域相關性。
           hl: 'en', //  語言參數，代表要以哪種語言來顯示搜尋結果。'en' 表示結果會以英文顯示。
           gl: 'us', // 國家/地區代碼，用來設定你希望搜尋結果的區域依據。'us' 表示結果會根據美國的地區來產生。
           }),

Extending the AI Agent's Memory

Large language models themselves do not have memory. The reason they can understand the context of a conversation is because the entire discussion history is passed back to the LLM with each new query.

Langchain's memory feature also operates on this principle. By storing each user query in Redis, we can equip our AI Bot with memory capabilities.

Installing Related Packages - Reference Documentation

npm i @langchain/redis @langchain/core redis @langchain/openai --save

Using Redis Memory

import { RedisChatMessageHistory } from '@langchain/community/stores/message/ioredis';
import { BufferMemory } from 'langchain/memory';
...
 const client = new Redis('redis://localhost:30001');

        const memory = new BufferMemory({
            chatHistory: new RedisChatMessageHistory({
                sessionId: 'sessionId:' + new Date().toISOString(),
                sessionTTL: 300,
                client,
            }),
            aiPrefix: 'ollama',
            outputKey: 'output', // 沒有會爆錯
            memoryKey: 'chat_history', // 沒有會爆錯
            inputKey: 'input', // 沒有會爆錯
            returnMessages: true,
            
            
const agentExecutor = new AgentExecutor({
            agent,
            tools,
            verbose: true, // enable debug
            handleParsingErrors: true,
            memory, // redis 記憶功能
            returnIntermediateSteps: true,
        });
...

Complete Code (Next.js API)

import { RedisChatMessageHistory } from '@langchain/community/stores/message/ioredis';
import { Calculator } from '@langchain/community/tools/calculator';
import { SerpAPI } from '@langchain/community/tools/serpapi';
import { ChatPromptTemplate } from '@langchain/core/prompts';
import { tool } from '@langchain/core/tools';
import { ChatOllama } from '@langchain/ollama';
import Redis from 'ioredis';
import { AgentExecutor, createToolCallingAgent } from 'langchain/agents';
import { BufferMemory } from 'langchain/memory';
import { NextApiRequest, NextApiResponse } from 'next/types';
import { z } from 'zod';

export default async function handler(req: NextApiRequest, res: NextApiResponse) {
    if (req.method !== 'GET') {
        return res.status(405).json({ message: 'Only GET requests are allowed' });
    }

    try {
        const magicTool = tool(
            async ({ input }: { input: number }) => {
                return `${input + 2}`;
            },
            {
                name: 'magic_function',
                description: 'Applies a magic function to an input.',
                schema: z.object({
                    input: z.number(),
                }),
            }
        );

        const llm = new ChatOllama({
            model: 'llama3.2',
            temperature: 0,
            maxRetries: 2,
            baseUrl: 'http://localhost:11434',
            // other params...
        });

        const tools = [
            magicTool, // custom tool
            new Calculator(),
            new SerpAPI('your serp key', {
                location: 'Austin,Texas,United States',
                hl: 'en',
                gl: 'us',
            }),
        ];

        const prompt = ChatPromptTemplate.fromMessages([
            ['system', 'You are a helpful assistant'], // default role
            ['placeholder', '{chat_history}'], // default role
            ['human', '{input}'],
            ['placeholder', '{agent_scratchpad}'],
        ]);

        const agent = createToolCallingAgent({
            llm,
            tools,
            prompt,
        });

        const client = new Redis('redis://localhost:30001');

        const memory = new BufferMemory({
            chatHistory: new RedisChatMessageHistory({
                sessionId: 'sessionId:' + new Date().toISOString(),
                sessionTTL: 300,
                client,
            }),
            aiPrefix: 'ollama',
            outputKey: 'output', // 沒有會爆錯
            memoryKey: 'chat_history', // 沒有會爆錯
            inputKey: 'input', // 沒有會爆錯
            returnMessages: true,
        });

        const agentExecutor = new AgentExecutor({
            agent,
            tools,
            verbose: true, // enable debug
            handleParsingErrors: true,
            memory, // redis 記憶功能
            returnIntermediateSteps: true,
        });

        const result = await agentExecutor.invoke({ input: 'what is the value of magic_function(3)?' });
        const serpResult2 = await agentExecutor.invoke({ input: 'what is the Pokomon?' });
        const result3 = await agentExecutor.invoke({ input: 'what is 5 + 2 *5 =' });
        const result4 = await agentExecutor.invoke({ input: 'hello', outputKey: 'key1' });
        const result5 = await agentExecutor.invoke({ input: '幫我推薦電腦' });

        return res.status(200).json({ result, serpResult2, result3, result4, result5 });
    } catch (error) {
        return res.status(500).json({ message: 'Internal server error' });
    }
}

References

• Official Documentation
• Official API Documentation
• Building AI Solutions with LangChain and Node.js: A Comprehensive Guide

Related Articles in This Series

• Building an AI Bot API with Langchain and Open-Source Llama AI in Next.js Part 1 - Getting Started with Langchain
• Building an AI Bot API with Langchain and Open-Source Llama AI in Next.js Part 2 - Creating an AI Agent with Memory
• Building an AI Bot API with Langchain and Open-Source Llama AI in Next.js Part 3 - Adding a Vector Database to Give the AI an Extra Brain
• Building an AI Bot API with Langchain and Open-Source Llama AI in Next.js Part 4 - AI Product Recommendation API