Luke Hinds lukehinds

Execution-Based Tool Tracing: Building Better Agentic Training Data

Training agentic models that can effectively use tools remains one of the harder problems in applied ML. Models trained on purely synthetic data - where tool calls and their responses are both generated by an LLM - consistently underperform when deployed against real systems. They struggle with error recovery, mishandle state dependencies, and often exhibit what we call "time travel" errors: acting on information they haven't actually received yet.

This post introduces DeepFabric's execution-based tool tracing system, which replaces simulated tool outputs with real execution inside WebAssembly sandboxes. The result is training data grounded in actual system behavior, including the messy parts that make real-world tool use challenging.

The Problem with Simulated Tool Calls

Consider a typical synthetic data generation pipeline for tool-using agents. An LLM generates a user request, then generates an assistant response with tool calls,

The "Academic Diagram" Prompt Template

Replace the [bracketed text] with your specific topic.

Create a professional, high-fidelity academic diagram illustrating [insert concept or summary of text here].

Visual Style: Clean vector-style graphics, flat design, technical schematic. Similar to a figure found in a high-end computer science textbook or an IEEE research paper.

Composition: Organized flow-chart or system architecture layout. Use distinct geometric nodes (rectangles, cylinders for databases) connected by clear directional arrows to show process flow.

	#####################################################################
	# Spin Blender Tools Dataset Configuration
	#####################################################################
	# This configuration demonstrates using Blender MCP tools via Spin
	# for generating synthetic 3D design assistant training data.
	#
	# Prerequisites:
	# 1. Start the Spin service:
	# cd tools-sdk
	# spin build && spin up

	#!/bin/bash
	# Load comprehensive Blender MCP mock data into the mock tools server
	#
	# Usage: ./load-blender-mock-data.sh [base_url]
	# Default base_url: http://localhost:3000

	set -e

	BASE_URL="${1:-http://localhost:3000}"
	SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"

	{
	"description": "Comprehensive Blender MCP mock data for testing tool execution with 3D design assistant scenarios",
	"version": "1.0.0",
	"mockResponses": {
	"get_scene_info": {
	"defaultResponse": {
	"name": "Untitled",
	"objects": [],
	"activeObject": null,
	"renderEngine": "CYCLES",

	{
	"description": "Comprehensive Google Workspace mock data for testing tool execution with productivity assistant scenarios",
	"version": "1.0.0",
	"mockResponses": {
	"search_gmail_messages": {
	"defaultResponse": {
	"messages": [
	{
	"id": "msg_001",
	"threadId": "thread_001",

	#####################################################################
	# Spin Google Workspace Tools Dataset Configuration
	#####################################################################
	# This configuration demonstrates using Google Workspace MCP tools via Spin
	# for generating synthetic productivity assistant training data.
	#
	# Prerequisites:
	# 1. Start the Spin service:
	# cd tools-sdk
	# spin build && spin up

	#!/bin/bash
	# Load comprehensive Google Workspace mock data into the mock tools server
	#
	# Usage: ./load-google-workspace-mock-data.sh [base_url]
	# Default base_url: http://localhost:3000

	set -e

	BASE_URL="${1:-http://localhost:3000}"
	SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"