Chat in HUF

Reference: develop @ aefaac5 · Branch: feature/jsx-in-chat · Date: 2026-02-13 09:24 UTC

This document describes how the chat feature works in HUF, including UI architecture, message capabilities, APIs, and real-time updates.

Overview

The HUF chat provides an agent-based conversational interface. Users select an agent, send messages, and receive responses that can include plain text, images, tool outputs, and rich visualizations (JSX previews, web previews, artifacts). The system uses REST APIs for message send/receive and Socket.IO for real-time tool and message updates.

UI Architecture

Page Structure

ChatPageV2 (pages/ChatPageV2.tsx)
├── ChatListing (sidebar, collapsible)
│   ├── ChatListHeader (title "Chat", AgentModelSelector)
│   └── Tabs: Recents | By Agent
│       ├── RecentsConversationList (grouped by TODAY/YESTERDAY/THIS WEEK/OLDER)
│       └── AgentConversationItem (accordion per agent)
└── ChatWindow (ChatWindowV2)
    ├── ChatWindowHeader (agent avatar, name, model badge)
    └── ChatMessageList
        ├── Scrollable message area (useInfiniteScroll)
        └── ChatInput (textarea, submit)

Key Components

Component	Path	Role
ChatPageV2	`pages/ChatPageV2.tsx`	Top-level layout: collapsible sidebar + chat window. Sidebar toggle (PanelLeftClose/Open).
ChatListing	`components/chat/ChatListing.tsx`	Sidebar: "Chat" title, agent selector, Recents/By Agent tabs. Compact layout (w-80).
ChatWindowV2	`components/chat/ChatWindowV2.tsx`	Header + message list. Closes app sidebar on mount.
ChatMessageList	`components/chat/ChatMessageList.tsx`	Fetches messages (useInfiniteScroll), merges socket updates, renders ChatMessage list.
ChatInput	`components/chat/ChatInput.tsx`	Textarea, Enter to send, Shift+Enter for newline. Calls newConversation or sendMessageToConversation.
ChatMessage	`components/chat/ChatMessage.tsx`	Single message: avatar, content via MessageContentWithArtifacts, tools, feedback actions.

AI Elements (`components/ai-elements/`)

Reusable building blocks for chat and other AI UIs:

message.tsx — Message, MessageContent, MessageResponse (Streamdown for markdown)
tool.tsx — Tool, ToolHeader, ToolContent, ToolInput, ToolOutput for tool calls
image.tsx — Image for generated images (base64 or URL, download button)
artifact.tsx — Artifact, ArtifactHeader, ArtifactContent for code/html/svg blocks
web-preview.tsx — WebPreview, WebPreviewBody for iframe previews
code-block.tsx — CodeBlock with Shiki syntax highlighting

Message Capabilities

1. Plain Text & Markdown

Default rendering via MessageResponse (Streamdown). Supports standard markdown.

2. Image Generation

Backend: Agent uses generate_image tool; backend creates Agent Message with kind="Image" and generated_image URL.
Frontend: ChatMessage checks message.kind === 'Image' and renders Image from ai-elements/image.tsx with download support.
Real-time: Socket emits new_agent_message with kind, generated_image; useChatSocket → upsertAgentMessageFromSocket updates UI before next poll.

3. Tool Calls

Backend: Tools run during agent execution; Agent Message records tool_name, tool_status, tool_args, tool_result.
Frontend: ChatMessage renders Tool (ToolHeader, ToolInput, ToolOutput) for each tool in message.tools.
Real-time: Socket emits tool_call_started, tool_call_completed, tool_call_failed; useChatSocket → upsertToolUpdateFromSocket updates tool state live.

4. JSX Preview

Format: Agent outputs <jsx-preview title="..."> tags with React/JSX (Recharts, shadcn/ui, Lucide icons).
Parsing: jsxPreviewParser extracts tags; unwraps markdown code fences (```xml) so wrapped tags are detected.
Rendering: JSXPreviewRenderer → JSXPreview (react-jsx-parser). Full-width in assistant messages.
Export: PNG via html-to-image (oklch-compatible); SVG from chart SVGs. Export buttons hidden during PNG capture.

5. Web Preview

Format: <web-preview url="..."> tags.
Parsing: webPreviewParser.
Rendering: WebPreviewRenderer — iframe with navigation (back/forward/refresh), URL bar, open in new tab.

6. Artifacts

Format: <artifact type="code|html|svg|mermaid|..." language="..."> blocks.
Parsing: artifactParser.
Rendering: ArtifactRenderer — code (Shiki), HTML, SVG, Mermaid diagrams, etc.

Content Pipeline

MessageContentWithArtifacts orchestrates parsing and rendering:

Decode HTML entities (< → <).
Parse in order: JSX previews → web previews → artifacts (avoids nested capture).
Render remaining text as MessageResponse (markdown).
Render JSX, web previews, artifacts as their respective components.

APIs

Frappe SDK (`lib/frappe-sdk.ts`)

import { FrappeApp } from 'frappe-js-sdk';
const frappe = new FrappeApp(frappeUrl);
export const db = frappe.db();
export const call = frappe.call();

db — DocType CRUD: getDoc, getDocList, createDoc, updateDoc.
call — Whitelisted method calls: call.post('module.method', args).

Chat API (`services/chatApi.ts`)

Function	Backend	Purpose
`getConversations`	`db.getDocList('Agent Conversation')`	Paginated conversation list (search, filters).
`getAgentsWithConversationCounts`	`db.getDocList('Agent')` + `fetchDocCount`	Agents with conversation counts for "By Agent" tab.
`getConversationsByAgent`	`db.getDocList('Agent Conversation', filters)`	Conversations for a specific agent.
`getAllConversationsForRecents`	`db.getDocList('Agent Conversation')`	All conversations for Recents date grouping.
`getConversation`	`db.getDoc('Agent Conversation', id)`	Single conversation (agent, model).
`getConversationMessages`	`db.getDocList('Agent Message')`	Paginated messages for a conversation.
`newConversation`	`call.post('huf.ai.agent_chat.new_conversation', {agent, message})`	Create conversation and run first message.
`sendMessageToConversation`	`call.post('huf.ai.agent_chat.send_message_to_conversation', {conversation, message})`	Send message to existing conversation.
`createAgentRunFeedback`	`db.createDoc('Agent Run Feedback', {...})`	Thumbs up/down feedback.
`updateConversationTitle`	`db.updateDoc('Agent Conversation', id, {title})`	Rename conversation.

Backend Endpoints (`huf/ai/agent_chat.py`)

new_conversation(agent, message) — Creates Agent Conversation, runs run_agent_sync, returns {success, conversation_id, run}. run includes response, conversation_id, agent_run_id, etc.
send_message_to_conversation(conversation, message) — Validates conversation, runs run_agent_sync, returns result (includes response, conversation_id).

Socket & Polling

Socket.IO (Real-time)

Connection: createFrappeSocket in utils/socket.ts — Socket.IO client to protocol://host:port/siteName. Uses frappe.boot.sitename and socketio_port (default 9000). Transports: websocket first, polling fallback.

Subscription: useChatSocket subscribes to conversation:{conversationId} when conversationId is set.

Events:

Event	Payload	Handler
`tool_call_started`	`ToolCallEvent` (tool_name, tool_status, tool_args, etc.)	`onToolUpdate` → `upsertToolUpdateFromSocket`
`tool_call_completed`	Same + `tool_result`	Same
`tool_call_failed`	Same + `error`	Same
`new_agent_message`	`NewAgentMessageEvent` (message_id, kind, content, generated_image)	`onNewMessage` → `upsertAgentMessageFromSocket`

Backend emission: agent_integration.py and litellm.py emit via frappe.publish_realtime with event='conversation:{id}' when tools complete or new agent messages (e.g. images) are created.

Polling (Message Fetch)

useInfiniteScroll — Fetches getConversationMessages with limit/start. Direction reverse (newest first, prepend older). Sentinel element triggers load-more when visible. Used for initial load and "Load previous messages."

Merge strategy: mergeConversationItemsIntoMessages maps API ChatMessage[] to MessageType[]. Socket updates are merged in: tool events update existing messages by agent_run_id; new_agent_message updates or appends by message_id.

Data Flow

Sending a Message

User types and submits in ChatInput.
User message appended to local state immediately.
Placeholder assistant message added.
newConversation or sendMessageToConversation called via call.post.
Response text applied to placeholder (no streaming in current implementation).
On new conversation: navigate to /chat/{conversationId}.
Socket connects to conversation:{id}; tool/image events update UI in real time.

Loading a Conversation

Route /chat/:chatId loads ChatPageV2 with chatId.
ChatMessageList calls getConversationMessages via useInfiniteScroll.
Messages mapped to MessageType, merged with any socket-originated messages.
Each ChatMessage renders content via MessageContentWithArtifacts, which parses and renders text, JSX, web previews, artifacts.

File Reference

Area	Key Files
Pages	`frontend/src/pages/ChatPageV2.tsx`
Chat UI	`frontend/src/components/chat/` (ChatListing, ChatWindowV2, ChatMessageList, ChatInput, ChatMessage, ChatWindowHeader)
Content	`MessageContentWithArtifacts`, `ArtifactRenderer`, `WebPreviewRenderer`, `JSXPreviewRenderer`
Parsers	`utils/artifactParser`, `utils/webPreviewParser`, `utils/jsxPreviewParser`
AI Elements	`frontend/src/components/ai-elements/` (message, tool, image, artifact, web-preview, code-block)
API	`services/chatApi.ts`, `lib/frappe-sdk.ts`
Socket	`hooks/useChatSocket.tsx`, `utils/socket.ts`
Backend	`huf/ai/agent_chat.py`, `huf/ai/agent_integration.py`

Using Chat Over API (External Frontends)

You can build a chat UI outside the HUF frontend by calling the same REST APIs and connecting to Socket.IO for real-time updates. All endpoints require authentication.

Authentication

Option 1: Token (recommended for API clients)

Generate API Key and API Secret in User → Settings → API Access. Use:

Authorization: token api_key:api_secret

Option 2: Password (session cookie)

# Login (save cookie to file)
curl -X POST "https://your-site.com/api/method/login" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -c cookies.txt \
  -d '{"usr":"user@example.com","pwd":"your-password"}'

Use -b cookies.txt for later requests.

Base URL

Replace https://your-site.com with your HUF instance (e.g. http://localhost:8080 for bench). For multi-tenant, include the site: https://huf.example.com or http://huf.localhost:8080.

1. Start a New Conversation

Creates a conversation and sends the first message. Returns conversation_id and the agent response.

curl -X POST "https://your-site.com/api/method/huf.ai.agent_chat.new_conversation" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: token api_key:api_secret" \
  -d '{
    "agent": "Your-Agent-Name",
    "message": "Hello, what can you help me with?"
  }'

Response:

{
  "message": {
    "success": true,
    "conversation_id": "abc123xyz",
    "run": {
      "response": "I can help you with...",
      "conversation_id": "abc123xyz",
      "agent_run_id": "run_xyz",
      "provider": "openai",
      "session_id": "..."
    }
  }
}

2. Send Message to Existing Conversation

curl -X POST "https://your-site.com/api/method/huf.ai.agent_chat.send_message_to_conversation" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: token api_key:api_secret" \
  -d '{
    "conversation": "abc123xyz",
    "message": "Tell me more about that."
  }'

Response:

{
  "message": {
    "response": "Certainly! Here is more detail...",
    "conversation_id": "abc123xyz",
    "agent_run_id": "run_xyz",
    "provider": "openai",
    "session_id": "..."
  }
}

3. Get Conversation History (Simple)

Returns messages as {role, content, creation, user, conversation_index}. Use for context or simple display.

curl -X GET "https://your-site.com/api/method/huf.ai.agent_chat.get_history?conversation_id=abc123xyz&limit=200" \
  -H "Accept: application/json" \
  -H "Authorization: token api_key:api_secret"

Response:

{
  "message": [
    {"role": "user", "content": "Hello", "creation": "2026-02-13 10:00:00", "user": "user@example.com", "conversation_index": 0},
    {"role": "assistant", "content": "Hi! How can I help?", "creation": "2026-02-13 10:00:05", "user": null, "conversation_index": 1}
  ]
}

4. List Conversations (REST Resource)

Paginated list of Agent Conversation documents. Filter by channel = "Chat" for chat conversations.

curl -X GET "https://your-site.com/api/resource/Agent%20Conversation?fields=[\"name\",\"title\",\"agent\",\"last_activity\",\"modified\"]&filters=[[\"channel\",\"=\",\"Chat\"]]&order_by=modified%20desc&limit_page_length=20&limit_start=0" \
  -H "Accept: application/json" \
  -H "Authorization: token api_key:api_secret"

Response:

{
  "data": [
    {"name": "abc123xyz", "title": "Chat with Agent", "agent": "My-Agent", "last_activity": "2026-02-13 10:05:00", "modified": "2026-02-13 10:05:00"}
  ]
}

5. Get a Single Conversation

curl -X GET "https://your-site.com/api/resource/Agent%20Conversation/abc123xyz" \
  -H "Accept: application/json" \
  -H "Authorization: token api_key:api_secret"

6. List Messages (Full Detail)

For rich display (tools, images, kind). Use Agent Message with filters.

curl -X GET "https://your-site.com/api/resource/Agent%20Message?fields=[\"name\",\"conversation\",\"content\",\"is_agent_message\",\"kind\",\"generated_image\",\"tool_name\",\"tool_status\",\"tool_args\",\"creation\",\"modified\"]&filters=[[\"conversation\",\"=\",\"abc123xyz\"]]&order_by=creation%20asc&limit_page_length=30&limit_start=0" \
  -H "Accept: application/json" \
  -H "Authorization: token api_key:api_secret"

Response: Array of message objects. is_agent_message=1 for assistant, 0 for user. kind="Image" with generated_image for images. Tool messages have tool_name, tool_status, tool_args, etc.

7. Conversations by Agent

curl -X GET "https://your-site.com/api/resource/Agent%20Conversation?fields=[\"name\",\"title\",\"agent\",\"last_activity\",\"modified\"]&filters=[[\"agent\",\"=\",\"My-Agent\"],[\"channel\",\"=\",\"Chat\"]]&order_by=modified%20desc&limit_page_length=100" \
  -H "Accept: application/json" \
  -H "Authorization: token api_key:api_secret"

8. Submit Feedback (Thumbs Up/Down)

Requires agent, feedback, and agent_message (the message ID from the assistant response).

curl -X POST "https://your-site.com/api/resource/Agent%20Run%20Feedback" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: token api_key:api_secret" \
  -d '{
    "agent": "My-Agent",
    "feedback": "Thumbs Up",
    "agent_message": "msg_xyz",
    "comments": "Optional comment"
  }'

9. Update Conversation Title

curl -X PUT "https://your-site.com/api/resource/Agent%20Conversation/abc123xyz" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: token api_key:api_secret" \
  -d '{"title": "My Chat Title"}'

10. Socket.IO (Real-time Updates)

For live tool progress and new messages (e.g. images), connect to Frappe's Socket.IO server.

URL format: {protocol}://{host}{:port}/{site_name}

protocol: http or https
host: e.g. localhost or huf.example.com
port: from site_config.json → socketio_port (often 9000 or 9001)
site_name: e.g. huf.localhost (from frappe.boot.sitename or site URL)

Example (Node.js with socket.io-client):

const { io } = require("socket.io-client");

const siteName = "huf.localhost";  // or your site
const port = "9000";  // socketio_port from site_config.json
const url = `http://localhost:${port}/${siteName}`;

const socket = io(url, {
  withCredentials: true,
  transports: ["websocket", "polling"],
});

socket.on("connect", () => console.log("Connected"));
socket.on("connect_error", (err) => console.error("Error:", err));

// Subscribe to conversation events
const conversationId = "abc123xyz";
socket.on(`conversation:${conversationId}`, (data) => {
  if (data.type === "new_agent_message") {
    console.log("New message:", data.message_id, data.kind, data.generated_image);
  } else {
    console.log("Tool update:", data.tool_name, data.tool_status, data.tool_result);
  }
});

Event payloads:

Event	`data.type`	Key fields
Tool started	`tool_call_started`	`tool_name`, `tool_status`, `tool_args`, `agent_run_id`
Tool completed	`tool_call_completed`	`tool_name`, `tool_result`, `agent_run_id`
Tool failed	`tool_call_failed`	`tool_name`, `error`, `agent_run_id`
New message	`new_agent_message`	`message_id`, `kind`, `content`, `generated_image`

Auth: Socket.IO uses the same session/cookie as the web app. For token-based clients, you may need to pass credentials (e.g. withCredentials: true) and ensure the server accepts your auth. Frappe's socket typically relies on the session established via the web app; for headless API clients, verify your setup supports socket auth.

Suggested Flow for External Chat UI

Login or use token for all requests.
New chat: POST /api/method/huf.ai.agent_chat.new_conversation → get conversation_id.
Continue chat: POST /api/method/huf.ai.agent_chat.send_message_to_conversation.
Load history: GET /api/method/huf.ai.agent_chat.get_history or GET /api/resource/Agent Message with filters.
List chats: GET /api/resource/Agent Conversation with channel=Chat.
Real-time: Connect Socket.IO to conversation:{id} for tool and image updates.
Polling fallback: If socket is unavailable, poll Agent Message or get_history periodically.

Optional: Voice Message (Upload & Transcribe)

Requires an Agent Chat doc (docname). This is typically used by the built-in chat UI; external API clients may need to create an Agent Chat doc first or use a different integration path.

curl -X POST "https://your-site.com/api/method/huf.ai.agent_chat.upload_audio_and_transcribe" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "Authorization: token api_key:api_secret" \
  -d '{
    "docname": "your-agent-chat-id",
    "filename": "recording.webm",
    "b64data": "data:audio/webm;base64,...",
    "agent": "My-Agent",
    "conversation": "abc123xyz"
  }'

Returns { "success": true, "transcript": "...", "run": { ... } } on success.

esafwan/ChatInHuf.md

Chat in HUF

Overview

UI Architecture

Page Structure

Key Components

AI Elements (components/ai-elements/)

Message Capabilities

1. Plain Text & Markdown

2. Image Generation

3. Tool Calls

4. JSX Preview

5. Web Preview

6. Artifacts

Content Pipeline

APIs

Frappe SDK (lib/frappe-sdk.ts)

Chat API (services/chatApi.ts)

Backend Endpoints (huf/ai/agent_chat.py)

Socket & Polling

Socket.IO (Real-time)

Polling (Message Fetch)

Data Flow

Sending a Message

Loading a Conversation

File Reference

Using Chat Over API (External Frontends)

Authentication

Base URL

1. Start a New Conversation

2. Send Message to Existing Conversation

3. Get Conversation History (Simple)

4. List Conversations (REST Resource)

5. Get a Single Conversation

6. List Messages (Full Detail)

7. Conversations by Agent

8. Submit Feedback (Thumbs Up/Down)

9. Update Conversation Title

10. Socket.IO (Real-time Updates)

Suggested Flow for External Chat UI

Optional: Voice Message (Upload & Transcribe)

AI Elements (`components/ai-elements/`)

Frappe SDK (`lib/frappe-sdk.ts`)

Chat API (`services/chatApi.ts`)

Backend Endpoints (`huf/ai/agent_chat.py`)