In this post, we'll explore how to effectively use the /api/chat endpoint for maintaining conversations with an AI assistant. Whether you're building a chatbot, virtual assistant, or interactive application, managing chat history and crafting structured requests are crucial for delivering a smooth user experience. We’ll dive into the request structure, provide practical examples, and share best practices for achieving conversational continuity.
The /api/chat endpoint is designed to handle interactive, context-aware conversations. By passing the conversation history along with each request, you can enable the AI model to respond naturally, maintaining context across multiple turns. This guide will walk you through how to structure your requests and manage chat history for effective interactions.
1. Understanding the Roles
The /api/chat endpoint uses the following roles to differentiate between parts of the conversation:
§ system: Provides initial instructions for the assistant's behaviour.
§ user: Represents messages from the user.
§ assistant: Represents responses from the AI assistant.
2. Starting a Conversation
To initiate a conversation, include a single user message in the messages array.
Example Request:
curl http://localhost:11434/api/chat -d '{
"model": "llama3.2",
"messages": [
{
"role": "user",
"content": "What is the capital of India?"
}
],
"stream": false
}'
$curl http://localhost:11434/api/chat -d '{
> "model": "llama3.2",
> "messages": [
> {
> "role": "user",
> "content": "What is the capital of India?"
> }
> ],
> "stream": false
> }'
{"model":"llama3.2","created_at":"2025-01-18T06:22:11.304346Z","message":{"role":"assistant","content":"The capital of India is New Delhi."},"done_reason":"stop","done":true,"total_duration":318585334,"load_duration":29283459,"prompt_eval_count":32,"prompt_eval_duration":162000000,"eval_count":9,"eval_duration":126000000}
Formatted response looks like below.
{
"model": "llama3.2",
"created_at": "2025-01-18T06:22:11.304346Z",
"message": {
"role": "assistant",
"content": "The capital of India is New Delhi."
},
"done_reason": "stop",
"done": true,
"total_duration": 318585334,
"load_duration": 29283459,
"prompt_eval_count": 32,
"prompt_eval_duration": 162000000,
"eval_count": 9,
"eval_duration": 126000000
}
3. Continuing the Conversation
To maintain context, include the entire conversation history in subsequent requests.
Example Request:
curl http://localhost:11434/api/chat -d '{
"model": "llama3.2",
"messages": [
{
"role": "user",
"content": "What is the capital of India?"
},
{
"role": "assistant",
"content": "The capital of India is New Delhi."
},
{
"role": "user",
"content": "Can you tell me a fact about New Delhi?"
}
],
"stream": false
}'
$curl http://localhost:11434/api/chat -d '{
> "model": "llama3.2",
> "messages": [
> {
> "role": "user",
> "content": "What is the capital of India?"
> },
> {
> "role": "assistant",
> "content": "The capital of India is New Delhi."
> },
> {
> "role": "user",
> "content": "Can you tell me a fact about New Delhi?"
> }
> ],
> "stream": false
> }'
{"model":"llama3.2","created_at":"2025-01-18T06:23:36.949495Z","message":{"role":"assistant","content":"New Delhi was the capital of British India from 1911 to 1947 and was designed by architects Edwin Lutyens and Herbert Baker in the early 20th century, with the intention of creating a grand and elegant city that would be a symbol of British colonial power."},"done_reason":"stop","done":true,"total_duration":1098513292,"load_duration":30183542,"prompt_eval_count":60,"prompt_eval_duration":185000000,"eval_count":57,"eval_duration":882000000}
Formatted response looks like below.
{
"model": "llama3.2",
"created_at": "2025-01-18T06:23:36.949495Z",
"message": {
"role": "assistant",
"content": "New Delhi was the capital of British India from 1911 to 1947 and was designed by architects Edwin Lutyens and Herbert Baker in the early 20th century, with the intention of creating a grand and elegant city that would be a symbol of British colonial power."
},
"done_reason": "stop",
"done": true,
"total_duration": 1098513292,
"load_duration": 30183542,
"prompt_eval_count": 60,
"prompt_eval_duration": 185000000,
"eval_count": 57,
"eval_duration": 882000000
}
4: Setting Assistant Behavior with system Role
Use the system role to guide the assistant's behavior.
For example:
curl http://localhost:11434/api/chat -d '{
"model": "llama3.2",
"messages": [
{
"role": "system",
"content": "You are a helpful assistant who explains things simply and Expert in History"
},
{
"role": "user",
"content": "What is the Capital Of India"
}
],
"stream": false
}'
$curl http://localhost:11434/api/chat -d '{
> "model": "llama3.2",
> "messages": [
> {
> "role": "system",
> "content": "You are a helpful assistant who explains things simply and Expert in History"
> },
> {
> "role": "user",
> "content": "What is the Capital Of India"
> }
> ],
> "stream": false
> }'
{"model":"llama3.2","created_at":"2025-01-18T06:25:12.964391Z","message":{"role":"assistant","content":"The capital of India is New Delhi. However, it's worth noting that India has two capitals:\n\n1. **New Delhi**: This is the seat of the President of India and the official residence of the Prime Minister of India.\n2. **Rashtrapati Bhavan (Presidential Palace)**: Located in New Delhi, this is where the President of India lives.\n\nBut, if you're thinking of the administrative capital of India, it's:\n\n**Kolkata (West Bengal) for the state government**\nand \n**Pune for some state institutions**\n\nHowever, **New Delhi** remains the official capital city of India and is home to many important government institutions, embassies, and museums.\n\nWould you like to know more about India or its history?"},"done_reason":"stop","done":true,"total_duration":2592336416,"load_duration":28333750,"prompt_eval_count":44,"prompt_eval_duration":150000000,"eval_count":155,"eval_duration":2413000000}
Formatted response looks like below.
{
"model": "llama3.2",
"created_at": "2025-01-18T06:25:12.964391Z",
"message": {
"role": "assistant",
"content": "The capital of India is New Delhi. However, it's worth noting that India has two capitals:\n\n1. **New Delhi**: This is the seat of the President of India and the official residence of the Prime Minister of India.\n2. **Rashtrapati Bhavan (Presidential Palace)**: Located in New Delhi, this is where the President of India lives.\n\nBut, if you're thinking of the administrative capital of India, it's:\n\n**Kolkata (West Bengal) for the state government**\nand \n**Pune for some state institutions**\n\nHowever, **New Delhi** remains the official capital city of India and is home to many important government institutions, embassies, and museums.\n\nWould you like to know more about India or its history?"
},
"done_reason": "stop",
"done": true,
"total_duration": 2592336416,
"load_duration": 28333750,
"prompt_eval_count": 44,
"prompt_eval_duration": 150000000,
"eval_count": 155,
"eval_duration": 2413000000
}
5. Best Practices
§ Only include relevant parts of the conversation to optimize payload size.
§ Use the stream Parameter. Set "stream": false for a complete response in one payload. Set "stream": true for real-time responses split into chunks.
§ Structure Your Messages: Follow the role conventions (system, user, assistant) to ensure clarity.
The /api/chat endpoint provides a flexible way to build conversational AI systems. By including chat history and leveraging the system role, you can create engaging and contextually aware experiences. Try these examples to enhance your application's interactivity!
No comments:
Post a Comment