Skip to content

Instantly share code, notes, and snippets.

@alonsosilvaallende

alonsosilvaallende/understanding_chat_templates.ipynb

Created January 25, 2026 16:53
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save alonsosilvaallende/6734fab9789bd0bfd69813b0fd9ca893 to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save alonsosilvaallende/6734fab9789bd0bfd69813b0fd9ca893 to your computer and use it in GitHub Desktop.
Download ZIP
Understanding_Chat_Templates.ipynb
Display the source blob
Display the rendered blob
Raw
understanding_chat_templates.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"provenance": [],
"authorship_tag": "ABX9TyNB4BTBkrk4va1gDc/GCAkh",
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"language_info": {
"name": "python"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/alonsosilvaallende/6734fab9789bd0bfd69813b0fd9ca893/understanding_chat_templates.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"source": [
"If you have worked with language models, you might have worked with lists of `messages`. Indeed, when you send a request to a language model, we can call it a `user messsage` and the response can be called an `assistant message`. A conversation would consist of a list of a `user message` followed by an `assistant message` followed by a `user message`, etc.\n",
"\n",
"However, a language model takes one text and outputs another text, so you might be wondering what's the input text that's being passed to the language model. So what's going on?"
],
"metadata": {
"id": "P4WGQGfJhtYk"
}
},
{
"cell_type": "markdown",
"source": [
"A user message will be something like this:"
],
"metadata": {
"id": "I5wkqmV-iT6i"
}
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "rErnS-guhfnB",
"outputId": "8fdba4a9-598a-4740-bd13-929ada1ece05"
},
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"[{'role': 'user', 'content': 'Hi!'}]"
]
},
"metadata": {},
"execution_count": 1
}
],
"source": [
"user_message = [{\"role\": \"user\", \"content\": \"Hi!\"}]\n",
"user_message"
]
},
{
"cell_type": "markdown",
"source": [
"The assistant message will be something like this:"
],
"metadata": {
"id": "priJk7kdifkp"
}
},
{
"cell_type": "code",
"source": [
"assistant_message = [{\"role\": \"user\", \"content\": \"Hello! How can I assist you today?\"}]\n",
"assistant_message"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "5kWXmpE0iY9d",
"outputId": "a0405974-8a7c-46b4-ae85-fc4d198d4ab9"
},
"execution_count": 2,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"[{'role': 'user', 'content': 'Hello! How can I assist you today?'}]"
]
},
"metadata": {},
"execution_count": 2
}
]
},
{
"cell_type": "markdown",
"source": [
"To have a conversation, you can pass it a list of messages:"
],
"metadata": {
"id": "ifTQap4qinob"
}
},
{
"cell_type": "code",
"source": [
"messages = [\n",
" {\"role\": \"user\", \"content\": \"Hi!\"},\n",
" {\"role\": \"assistant\", \"content\": \"Hello! How can I assist you today?\"},\n",
" {\"role\": \"user\", \"content\": \"What's the capital of France?\"},\n",
"]\n",
"messages"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "nvyuKiR0iku1",
"outputId": "a49a3181-e4ae-4193-ddbc-dfdc6dfdc741"
},
"execution_count": 3,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"[{'role': 'user', 'content': 'Hi!'},\n",
" {'role': 'assistant', 'content': 'Hello! How can I assist you today?'},\n",
" {'role': 'user', 'content': \"What's the capital of France?\"}]"
]
},
"metadata": {},
"execution_count": 3
}
]
},
{
"cell_type": "markdown",
"source": [
"How do we convert a list of messages to an input text?"
],
"metadata": {
"id": "5ZwK7UWUi24P"
}
},
{
"cell_type": "markdown",
"source": [
"## The Chat Template"
],
"metadata": {
"id": "sCsf5y4ci61N"
}
},
{
"cell_type": "markdown",
"source": [
"The **chat template** takes as input a list of messages (and tools but we will talk about it later) and convert them into one single string.\n",
"\n",
"Let's look at a [chat template app](https://huggingface.co/spaces/alonsosilva/apply_chat_template)\n",
"\n",
"Let's see what the chat template does with an example.\n",
"\n",
"Let's download a tokenizer:"
],
"metadata": {
"id": "4sVcCAaejARW"
}
},
{
"cell_type": "code",
"source": [
"from transformers import AutoTokenizer\n",
"\n",
"MODEL_ID = 'Qwen/Qwen2.5-0.5B-Instruct'\n",
"tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "k0spjf4cirVd",
"outputId": "d935caab-598a-4f1a-9beb-5fc1b5b8c383"
},
"execution_count": 4,
"outputs": [
{
"output_type": "stream",
"name": "stderr",
"text": [
"/usr/local/lib/python3.12/dist-packages/huggingface_hub/utils/_auth.py:86: UserWarning: \n",
"Access to the secret `HF_TOKEN` has not been granted on this notebook.\n",
"You will not be requested again.\n",
"Please restart the session if you want to be prompted again.\n",
" warnings.warn(\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Let's pass it a user message:"
],
"metadata": {
"id": "HwCoszjqjw8k"
}
},
{
"cell_type": "code",
"source": [
"message = [{\"role\": \"user\", \"content\": \"Hi!\"}]\n",
"print(tokenizer.apply_chat_template(message, tokenize=False))"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "lW43Gjl0jnRm",
"outputId": "ad4d8256-a0ce-4615-d577-9beb4b1cdf04"
},
"execution_count": 5,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"<|im_start|>system\n",
"You are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\n",
"<|im_start|>user\n",
"Hi!<|im_end|>\n",
"\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Notice that when you don't provide a `system message`, the model `Qwen/Qwen2.5-0.5B-Instruct` adds a `system message` (\"You are Qwen, created by Alibaba Cloud. You are a helpful assistant.\"). Perhaps you want to change that to a different `system message`:"
],
"metadata": {
"id": "Y0zmXATzkAOj"
}
},
{
"cell_type": "code",
"source": [
"messages = [\n",
" {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
" {\"role\": \"user\", \"content\": \"Hi!\"},\n",
"]\n",
"print(tokenizer.apply_chat_template(messages, tokenize=False))"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "yJP89wPLj4xO",
"outputId": "4bf1afbd-9125-4e95-a382-13bef9e79352"
},
"execution_count": 6,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"<|im_start|>system\n",
"You are a helpful assistant.<|im_end|>\n",
"<|im_start|>user\n",
"Hi!<|im_end|>\n",
"\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"We can also see how does the chat template convert a conversation like this one:"
],
"metadata": {
"id": "9bRnzNImlNRt"
}
},
{
"cell_type": "code",
"source": [
"messages = [\n",
" {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
" {\"role\": \"user\", \"content\": \"Hi!\"},\n",
" {\"role\": \"assistant\", \"content\": \"Hello! How can I assist you today?\"},\n",
" {\"role\": \"user\", \"content\": \"What's the capital of France?\"},\n",
"]\n",
"print(tokenizer.apply_chat_template(messages, tokenize=False))"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "fvqtoisykHhm",
"outputId": "87d44ca2-242c-47bd-b558-352e494b3e05"
},
"execution_count": 7,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"<|im_start|>system\n",
"You are a helpful assistant.<|im_end|>\n",
"<|im_start|>user\n",
"Hi!<|im_end|>\n",
"<|im_start|>assistant\n",
"Hello! How can I assist you today?<|im_end|>\n",
"<|im_start|>user\n",
"What's the capital of France?<|im_end|>\n",
"\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Each model has its own chat template. Let's take a look at `Mistral-7B-v0.3` chat template:"
],
"metadata": {
"id": "e3luE09_la96"
}
},
{
"cell_type": "code",
"source": [
"MISTRAL_MODEL_ID = \"mistralai/Mistral-7B-Instruct-v0.3\" # if it doesn't work try \"MaziyarPanahi/Mistral-7B-v0.3\"\n",
"mistral_tokenizer = AutoTokenizer.from_pretrained(MISTRAL_MODEL_ID)"
],
"metadata": {
"id": "KUiUaB5mlSZl"
},
"execution_count": 8,
"outputs": []
},
{
"cell_type": "code",
"source": [
"messages = [\n",
" {\"role\": \"user\", \"content\": \"Hi!\"},\n",
" {\"role\": \"assistant\", \"content\": \"Hello! How can I assist you today?\"},\n",
" {\"role\": \"user\", \"content\": \"What's the capital of France?\"},\n",
"]\n",
"print(mistral_tokenizer.apply_chat_template(messages, tokenize=False))"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "PqKwYhtRlguH",
"outputId": "00ebd58b-8eae-4280-ecb5-a408823f764b"
},
"execution_count": 9,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"<s>[INST] Hi![/INST] Hello! How can I assist you today?</s>[INST] What's the capital of France?[/INST]\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Since after sending a `user message`, you expect an `assistant message`, you can help the model by basically saying \"Now, it's your turn!\". This is so useful that it has been incorporated into the chat template itself."
],
"metadata": {
"id": "lAyme0FHl4l3"
}
},
{
"cell_type": "markdown",
"source": [
"The parameter is `add_generation_prompt=True`, then the whole instruction becomes:\n",
"```\n",
"tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)\n",
"```"
],
"metadata": {
"id": "6KyL4I3Ll-8r"
}
},
{
"cell_type": "code",
"source": [
"messages = [\n",
" {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
" {\"role\": \"user\", \"content\": \"Hi!\"},\n",
"]\n",
"print(tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True))"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "9mlFUzl-l-Qm",
"outputId": "30e6af0b-3a02-4c67-e4fb-dc605536487f"
},
"execution_count": 10,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"<|im_start|>system\n",
"You are a helpful assistant.<|im_end|>\n",
"<|im_start|>user\n",
"Hi!<|im_end|>\n",
"<|im_start|>assistant\n",
"\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"## Tool calls\n",
"\n",
"The chat template also handles tool calls. That means that we can provide a list of tools (let's do one as an example):"
],
"metadata": {
"id": "eXJZLXJPmZ-5"
}
},
{
"cell_type": "code",
"source": [
"tools = [\n",
" {\n",
" \"type\": \"function\",\n",
" \"function\": {\n",
" \"name\": \"Python_REPL\",\n",
" \"description\": \"A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.\",\n",
" \"parameters\": {\n",
" \"properties\": {\n",
" \"python_code\": {\n",
" \"description\": \"Valid python command.\",\n",
" \"type\": \"string\",\n",
" }\n",
" },\n",
" \"required\": [\"python_code\"],\n",
" \"type\": \"object\",\n",
" },\n",
" },\n",
" }\n",
"]"
],
"metadata": {
"id": "Vez--HHwmFB1"
},
"execution_count": 11,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"The parameter is `tools=...`, then the whole instruction becomes:\n",
"```\n",
"tokenizer.apply_chat_template(messages, tokenize=False, tools=tools, add_generation_prompt=True)\n",
"```"
],
"metadata": {
"id": "ihODUrjsmnTR"
}
},
{
"cell_type": "code",
"source": [
"messages = [\n",
" {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
" {\"role\": \"user\", \"content\": \"What's 2 to the power of 5?\"},\n",
"]\n",
"\n",
"tools = [\n",
" {\n",
" \"type\": \"function\",\n",
" \"function\": {\n",
" \"name\": \"Python_REPL\",\n",
" \"description\": \"A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.\",\n",
" \"parameters\": {\n",
" \"properties\": {\n",
" \"python_code\": {\n",
" \"description\": \"Valid python command.\",\n",
" \"type\": \"string\",\n",
" }\n",
" },\n",
" \"required\": [\"python_code\"],\n",
" \"type\": \"object\",\n",
" },\n",
" },\n",
" }\n",
"]\n",
"\n",
"print(tokenizer.apply_chat_template(messages, tokenize=False, tools=tools, add_generation_prompt=True))"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "tXFZL8uVmj0O",
"outputId": "30ae6c8c-cecc-47c8-968c-e395a0b22382"
},
"execution_count": 12,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"<|im_start|>system\n",
"You are a helpful assistant.\n",
"\n",
"# Tools\n",
"\n",
"You may call one or more functions to assist with the user query.\n",
"\n",
"You are provided with function signatures within <tools></tools> XML tags:\n",
"<tools>\n",
"{\"type\": \"function\", \"function\": {\"name\": \"Python_REPL\", \"description\": \"A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.\", \"parameters\": {\"properties\": {\"python_code\": {\"description\": \"Valid python command.\", \"type\": \"string\"}}, \"required\": [\"python_code\"], \"type\": \"object\"}}}\n",
"</tools>\n",
"\n",
"For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n",
"<tool_call>\n",
"{\"name\": <function-name>, \"arguments\": <args-json-object>}\n",
"</tool_call><|im_end|>\n",
"<|im_start|>user\n",
"What's 2 to the power of 5?<|im_end|>\n",
"<|im_start|>assistant\n",
"\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"For Mistral tokenizer it becomes:"
],
"metadata": {
"id": "ZpmQAclCnSwc"
}
},
{
"cell_type": "code",
"source": [
"print(mistral_tokenizer.apply_chat_template(messages, tokenize=False, tools=tools, add_generation_prompt=True))"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "uTDktbB3nGc3",
"outputId": "c5c7407a-9144-47c9-bfa9-ce2ab6235227"
},
"execution_count": 13,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"<s>[AVAILABLE_TOOLS] [{\"type\": \"function\", \"function\": {\"name\": \"Python_REPL\", \"description\": \"A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.\", \"parameters\": {\"properties\": {\"python_code\": {\"description\": \"Valid python command.\", \"type\": \"string\"}}, \"required\": [\"python_code\"], \"type\": \"object\"}}}][/AVAILABLE_TOOLS][INST] You are a helpful assistant.\n",
"\n",
"What's 2 to the power of 5?[/INST]\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Quite complex string indeed.\n",
"\n",
"Let's look at some other model `NousResearch/Hermes-3-Llama-3.1-8B`:"
],
"metadata": {
"id": "a5WbNA8Rndiv"
}
},
{
"cell_type": "code",
"source": [
"NOUS_MODEL_ID = \"NousResearch/Hermes-3-Llama-3.1-8B\" # if it doesn't work try \"MaziyarPanahi/Mistral-7B-v0.3\"\n",
"nous_tokenizer = AutoTokenizer.from_pretrained(NOUS_MODEL_ID)\n",
"print(nous_tokenizer.apply_chat_template(messages, tokenize=False, tools=tools, add_generation_prompt=True))"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "TfTkxIDmnPkl",
"outputId": "a01fc2ba-5269-4b60-b689-2eb095776190"
},
"execution_count": 14,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"<|begin_of_text|><|im_start|>system\n",
"You are a function calling AI model. You are provided with function signatures within <tools></tools> XML tags. You may call one or more functions to assist with the user query. Don't make assumptions about what values to plug into functions. Here are the available tools: <tools> {\"type\": \"function\", \"function\": {\"name\": \"Python_REPL\", \"description\": \"Python_REPL(python_code: str) - A Python shell. Use this to execute python commands. Input should be a valid python command. If you want to see the output of a value, you should print it out with `print(...)`.\n",
"\n",
" Args:\n",
" python_code(str): Valid python command.\", \"parameters\": {\"properties\": {\"python_code\": {\"description\": \"Valid python command.\", \"type\": \"string\"}}, \"required\": [\"python_code\"], \"type\": \"object\"}} </tools>Use the following pydantic model json schema for each tool call you will make: {\"properties\": {\"name\": {\"title\": \"Name\", \"type\": \"string\"}, \"arguments\": {\"title\": \"Arguments\", \"type\": \"object\"}}, \"required\": [\"name\", \"arguments\"], \"title\": \"FunctionCall\", \"type\": \"object\"}}\n",
"For each function call return a json object with function name and arguments within <tool_call></tool_call> XML tags as follows:\n",
"<tool_call>\n",
"{\"name\": <function-name>, \"arguments\": <args-dict>}\n",
"</tool_call><|im_end|>\n",
"<|im_start|>system\n",
"You are a helpful assistant.<|im_end|>\n",
"<|im_start|>user\n",
"What's 2 to the power of 5?<|im_end|>\n",
"<|im_start|>assistant\n",
"\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"I don't like this chat template. It appears that our messages have **two different and consecutive** system prompts. I prefer much more the previous chat template of the model \"Qwen/Qwen2.5-0.5B-Instruct\"."
],
"metadata": {
"id": "FtmWalgFn_PY"
}
},
{
"cell_type": "markdown",
"source": [
"## Thinking mode\n",
"\n",
"A recent addition is the `enable_thinking` in some new reasoning models where the model will \"think\" between some tags, e.g. `<think>...</think>`. For example in `Qwen/Qwen3-0.6B` the model has the possibility to reason (which is *the default*), but you can turn this option off if you want to.\n",
"\n",
"The parameter is `enable_thinking=False` so the whole instruction is:\n",
"```\n",
"tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False)\n",
"```"
],
"metadata": {
"id": "aMBoq7mroJPk"
}
},
{
"cell_type": "code",
"source": [
"MODEL_ID = 'Qwen/Qwen3-0.6B'\n",
"reasoning_tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)\n",
"\n",
"messages = [\n",
" {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n",
" {\"role\": \"user\", \"content\": \"How many r's in strawberry?\"},\n",
"]\n",
"\n",
"print(reasoning_tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=False))"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "K5EK0BpWn4Bv",
"outputId": "aea520ba-fe11-413e-99b9-6a792032d1ee"
},
"execution_count": 15,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"<|im_start|>system\n",
"You are a helpful assistant.<|im_end|>\n",
"<|im_start|>user\n",
"How many r's in strawberry?<|im_end|>\n",
"<|im_start|>assistant\n",
"<think>\n",
"\n",
"</think>\n",
"\n",
"\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"I saw some influencers (with multiple reposts) claiming they found a 'hack' to make the model not think, and it consisted of appending `<think>\\n\\n</think>`.\n",
"\n",
"This is exactly what the chat template does!!!"
],
"metadata": {
"id": "2yNvDvbapCFi"
}
},
{
"cell_type": "markdown",
"source": [
"## How does the chat template handles this?\n",
"\n",
"The chat template has been programmed in [Jinja](https://jinja.palletsprojects.com/en/stable/) which is usually used in web development.\n",
"You can see the chat template with the following command:"
],
"metadata": {
"id": "rAGwxCNpqZkU"
}
},
{
"cell_type": "code",
"source": [
"print(tokenizer.chat_template)"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "UGxBgyNlo1PW",
"outputId": "5c743c68-c256-4c24-8e6b-505a29f4c399"
},
"execution_count": 16,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"{%- if tools %}\n",
" {{- '<|im_start|>system\\n' }}\n",
" {%- if messages[0]['role'] == 'system' %}\n",
" {{- messages[0]['content'] }}\n",
" {%- else %}\n",
" {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n",
" {%- endif %}\n",
" {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n",
" {%- for tool in tools %}\n",
" {{- \"\\n\" }}\n",
" {{- tool | tojson }}\n",
" {%- endfor %}\n",
" {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n",
"{%- else %}\n",
" {%- if messages[0]['role'] == 'system' %}\n",
" {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n",
" {%- else %}\n",
" {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n",
" {%- endif %}\n",
"{%- endif %}\n",
"{%- for message in messages %}\n",
" {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n",
" {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n",
" {%- elif message.role == \"assistant\" %}\n",
" {{- '<|im_start|>' + message.role }}\n",
" {%- if message.content %}\n",
" {{- '\\n' + message.content }}\n",
" {%- endif %}\n",
" {%- for tool_call in message.tool_calls %}\n",
" {%- if tool_call.function is defined %}\n",
" {%- set tool_call = tool_call.function %}\n",
" {%- endif %}\n",
" {{- '\\n<tool_call>\\n{\"name\": \"' }}\n",
" {{- tool_call.name }}\n",
" {{- '\", \"arguments\": ' }}\n",
" {{- tool_call.arguments | tojson }}\n",
" {{- '}\\n</tool_call>' }}\n",
" {%- endfor %}\n",
" {{- '<|im_end|>\\n' }}\n",
" {%- elif message.role == \"tool\" %}\n",
" {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n",
" {{- '<|im_start|>user' }}\n",
" {%- endif %}\n",
" {{- '\\n<tool_response>\\n' }}\n",
" {{- message.content }}\n",
" {{- '\\n</tool_response>' }}\n",
" {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n",
" {{- '<|im_end|>\\n' }}\n",
" {%- endif %}\n",
" {%- endif %}\n",
"{%- endfor %}\n",
"{%- if add_generation_prompt %}\n",
" {{- '<|im_start|>assistant\\n' }}\n",
"{%- endif %}\n",
"\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"After spending some time understanding what's going on, you can create your own if you want to change its behavior."
],
"metadata": {
"id": "0NnGgEm0qp9D"
}
},
{
"cell_type": "code",
"source": [
"old_template = tokenizer.chat_template\n",
"print(old_template)"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "qiVZ_iEpqyiQ",
"outputId": "73a1246c-86d3-4c85-e435-78ef425f095c"
},
"execution_count": 17,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"{%- if tools %}\n",
" {{- '<|im_start|>system\\n' }}\n",
" {%- if messages[0]['role'] == 'system' %}\n",
" {{- messages[0]['content'] }}\n",
" {%- else %}\n",
" {{- 'You are Qwen, created by Alibaba Cloud. You are a helpful assistant.' }}\n",
" {%- endif %}\n",
" {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n",
" {%- for tool in tools %}\n",
" {{- \"\\n\" }}\n",
" {{- tool | tojson }}\n",
" {%- endfor %}\n",
" {{- \"\\n</tools>\\n\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\n<tool_call>\\n{\\\"name\\\": <function-name>, \\\"arguments\\\": <args-json-object>}\\n</tool_call><|im_end|>\\n\" }}\n",
"{%- else %}\n",
" {%- if messages[0]['role'] == 'system' %}\n",
" {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n",
" {%- else %}\n",
" {{- '<|im_start|>system\\nYou are Qwen, created by Alibaba Cloud. You are a helpful assistant.<|im_end|>\\n' }}\n",
" {%- endif %}\n",
"{%- endif %}\n",
"{%- for message in messages %}\n",
" {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n",
" {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n",
" {%- elif message.role == \"assistant\" %}\n",
" {{- '<|im_start|>' + message.role }}\n",
" {%- if message.content %}\n",
" {{- '\\n' + message.content }}\n",
" {%- endif %}\n",
" {%- for tool_call in message.tool_calls %}\n",
" {%- if tool_call.function is defined %}\n",
" {%- set tool_call = tool_call.function %}\n",
" {%- endif %}\n",
" {{- '\\n<tool_call>\\n{\"name\": \"' }}\n",
" {{- tool_call.name }}\n",
" {{- '\", \"arguments\": ' }}\n",
" {{- tool_call.arguments | tojson }}\n",
" {{- '}\\n</tool_call>' }}\n",
" {%- endfor %}\n",
" {{- '<|im_end|>\\n' }}\n",
" {%- elif message.role == \"tool\" %}\n",
" {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n",
" {{- '<|im_start|>user' }}\n",
" {%- endif %}\n",
" {{- '\\n<tool_response>\\n' }}\n",
" {{- message.content }}\n",
" {{- '\\n</tool_response>' }}\n",
" {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n",
" {{- '<|im_end|>\\n' }}\n",
" {%- endif %}\n",
" {%- endif %}\n",
"{%- endfor %}\n",
"{%- if add_generation_prompt %}\n",
" {{- '<|im_start|>assistant\\n' }}\n",
"{%- endif %}\n",
"\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"new_template = \"\"\"{%- if tools %}\n",
" {{- '<|im_start|>system\\n' }}\n",
" {%- if messages[0]['role'] == 'system' %}\n",
" {{- messages[0]['content'] }}\n",
" {%- else %}\n",
" {{- 'You are The Wizard of Oz. You are a helpful assistant.' }}\n",
" {%- endif %}\n",
" {{- \"\\n\\n# Tools\\n\\nYou may call one or more functions to assist with the user query.\\n\\nYou are provided with function signatures within <tools></tools> XML tags:\\n<tools>\" }}\n",
" {%- for tool in tools %}\n",
" {{- \"\\n\" }}\n",
" {{- tool | tojson }}\n",
" {%- endfor %}\n",
" {{- \"\\\\n</tools>\\\\n\\\\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\\\\n<tool_call>\\\\n{\\\\\"name\\\\\": <function-name>, \\\\\"arguments\\\\\": <args-json-object>}\\\\n</tool_call><|im_end|>\\\\n\" }}\n",
"{%- else %}\n",
" {%- if messages[0]['role'] == 'system' %}\n",
" {{- '<|im_start|>system\\n' + messages[0]['content'] + '<|im_end|>\\n' }}\n",
" {%- else %}\n",
" {{- '<|im_start|>system\\nYou are The Wizard of Oz. You are a helpful assistant.<|im_end|>\\n' }}\n",
" {%- endif %}\n",
"{%- endif %}\n",
"{%- for message in messages %}\n",
" {%- if (message.role == \"user\") or (message.role == \"system\" and not loop.first) or (message.role == \"assistant\" and not message.tool_calls) %}\n",
" {{- '<|im_start|>' + message.role + '\\n' + message.content + '<|im_end|>' + '\\n' }}\n",
" {%- elif message.role == \"assistant\" %}\n",
" {{- '<|im_start|>' + message.role }}\n",
" {%- if message.content %}\n",
" {{- '\\n' + message.content }}\n",
" {%- endif %}\n",
" {%- for tool_call in message.tool_calls %}\n",
" {%- if tool_call.function is defined %}\n",
" {%- set tool_call = tool_call.function %}\n",
" {%- endif %}\n",
" {{- '\\n<tool_call>\\n{\"name\": \"' }}\n",
" {{- tool_call.name }}\n",
" {{- '\", \"arguments\": ' }}\n",
" {{- tool_call.arguments | tojson }}\n",
" {{- '}\\n</tool_call>' }}\n",
" {%- endfor %}\n",
" {{- '<|im_end|>\\n' }}\n",
" {%- elif message.role == \"tool\" %}\n",
" {%- if (loop.index0 == 0) or (messages[loop.index0 - 1].role != \"tool\") %}\n",
" {{- '<|im_start|>user' }}\n",
" {%- endif %}\n",
" {{- '\\n<tool_response>\\n' }}\n",
" {{- message.content }}\n",
" {{- '\\n</tool_response>' }}\n",
" {%- if loop.last or (messages[loop.index0 + 1].role != \"tool\") %}\n",
" {{- '<|im_end|>\\n' }}\n",
" {%- endif %}\n",
" {%- endif %}\n",
"{%- endfor %}\n",
"{%- if add_generation_prompt %}\n",
" {{- '<|im_start|>assistant\\n' }}\n",
"{%- endif %}\"\"\""
],
"metadata": {
"id": "_zkkQGQorH0_"
},
"execution_count": 18,
"outputs": []
},
{
"cell_type": "code",
"source": [
"tokenizer.chat_template = new_template"
],
"metadata": {
"id": "u83Elw5WqhfF"
},
"execution_count": 19,
"outputs": []
},
{
"cell_type": "code",
"source": [
"messages = [\n",
" {\"role\": \"user\", \"content\": \"Hi!\"},\n",
"]"
],
"metadata": {
"id": "7pRcu4jSrpKl"
},
"execution_count": 20,
"outputs": []
},
{
"cell_type": "code",
"source": [
"print(tokenizer.apply_chat_template(messages, tokenize=False))"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "1VXJluOwr04d",
"outputId": "4af4e616-709a-4248-cbd9-539b22e203ed"
},
"execution_count": 21,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"<|im_start|>system\n",
"You are The Wizard of Oz. You are a helpful assistant.<|im_end|>\n",
"<|im_start|>user\n",
"Hi!<|im_end|>\n",
"\n"
]
}
]
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment