{
 "date": "2026-10-02 13:01",
 "capabilities": 10,
 "machine": "<laptop>",
 "python": "3.12.9",
 "repeat_calls_per_path": 10,
 "notes": [
  "One laptop, one account, one day; ten trivial functions; ten first calls and ten repeat calls per path. Small sample, no statistics beyond a median.",
  "Path 'fixed' restarts the server for every capability because this baseline is built that way. MCP does not require it: a server may change its tool list at run time and notify the client (tools/list_changed), and path 'generic' adds capabilities with no edit and no restart.",
  "Path 'generic' is a local child process over stdio: no network, no authentication, no second machine, no sandbox, Python only, word-match search. It is a minimal general executor with storage written for this measurement; it is NOT Anthropic's code execution with MCP and NOT Cloudflare Code Mode, and neither of those was measured.",
  "Path 'extella-mcp' is a remote service plus a device listener, so its seconds include the network and the relay to the device; the 'local' device is this same laptop. A general executor placed behind a network would also be slower than the local one measured here; by how much was not measured.",
  "Seconds of the local paths and of the Extella path are therefore not a like-for-like latency comparison; they show the order of magnitude each setup has today.",
  "tool definitions bytes = length of the JSON of the tool list as the official Python SDK parses it. Bytes, not model tokens. The Extella count is what this account and agent were served at the time of the run; the server also offers enable_group/disable_group tools, so the list may differ in another configuration.",
  "Second machine: measured only for Extella (3 of the saved experts, a device prepared in advance). For 'fixed' and 'generic' it was not measured because the server would have to be deployed there first; that deployment cost is real but is not quantified here.",
  "Extella experts are created on the first run of this script in an account and overwritten on later runs (the script deletes nothing).",
  "Not measured at all: security isolation, versioning, search quality, cost, concurrency, failure behaviour, token usage of a model driving these tools.",
  "Second run of the day: all ten Extella saves in this run overwrite experts created by the first run about 20 minutes earlier (kept as result3-2026-10-02-run1.json, where nine of ten saves were creations)."
 ],
 "paths": {
  "fixed": {
   "transport": "MCP stdio, own server (official Python SDK, FastMCP), one tool per capability",
   "rows": [
    {
     "n": 1,
     "capability": "word_count",
     "server_edits": 1,
     "server_restarts": 1,
     "tools_listed": 1,
     "tool_definitions_bytes": 193,
     "seconds_to_first_call": 0.313,
     "seconds_call_only": 0.002,
     "ok": true
    },
    {
     "n": 2,
     "capability": "total",
     "server_edits": 1,
     "server_restarts": 1,
     "tools_listed": 2,
     "tool_definitions_bytes": 397,
     "seconds_to_first_call": 0.296,
     "seconds_call_only": 0.002,
     "ok": true
    },
    {
     "n": 3,
     "capability": "reverse_text",
     "server_edits": 1,
     "server_restarts": 1,
     "tools_listed": 3,
     "tool_definitions_bytes": 594,
     "seconds_to_first_call": 0.311,
     "seconds_call_only": 0.002,
     "ok": true
    },
    {
     "n": 4,
     "capability": "is_prime",
     "server_edits": 1,
     "server_restarts": 1,
     "tools_listed": 4,
     "tool_definitions_bytes": 775,
     "seconds_to_first_call": 0.32,
     "seconds_call_only": 0.002,
     "ok": true
    },
    {
     "n": 5,
     "capability": "fibonacci",
     "server_edits": 1,
     "server_restarts": 1,
     "tools_listed": 5,
     "tool_definitions_bytes": 958,
     "seconds_to_first_call": 0.302,
     "seconds_call_only": 0.002,
     "ok": true
    },
    {
     "n": 6,
     "capability": "to_fahrenheit",
     "server_edits": 1,
     "server_restarts": 1,
     "tools_listed": 6,
     "tool_definitions_bytes": 1166,
     "seconds_to_first_call": 0.295,
     "seconds_call_only": 0.003,
     "ok": true
    },
    {
     "n": 7,
     "capability": "slug",
     "server_edits": 1,
     "server_restarts": 1,
     "tools_listed": 7,
     "tool_definitions_bytes": 1350,
     "seconds_to_first_call": 0.298,
     "seconds_call_only": 0.002,
     "ok": true
    },
    {
     "n": 8,
     "capability": "median",
     "server_edits": 1,
     "server_restarts": 1,
     "tools_listed": 8,
     "tool_definitions_bytes": 1556,
     "seconds_to_first_call": 0.311,
     "seconds_call_only": 0.002,
     "ok": true
    },
    {
     "n": 9,
     "capability": "vowels",
     "server_edits": 1,
     "server_restarts": 1,
     "tools_listed": 9,
     "tool_definitions_bytes": 1741,
     "seconds_to_first_call": 0.315,
     "seconds_call_only": 0.002,
     "ok": true
    },
    {
     "n": 10,
     "capability": "longest_word",
     "server_edits": 1,
     "server_restarts": 1,
     "tools_listed": 10,
     "tool_definitions_bytes": 1938,
     "seconds_to_first_call": 0.298,
     "seconds_call_only": 0.002,
     "ok": true
    }
   ],
   "repeat_calls": [
    {
     "capability": "word_count",
     "seconds": 0.0017,
     "ok": true
    },
    {
     "capability": "total",
     "seconds": 0.0014,
     "ok": true
    },
    {
     "capability": "reverse_text",
     "seconds": 0.0014,
     "ok": true
    },
    {
     "capability": "is_prime",
     "seconds": 0.0014,
     "ok": true
    },
    {
     "capability": "fibonacci",
     "seconds": 0.0014,
     "ok": true
    },
    {
     "capability": "to_fahrenheit",
     "seconds": 0.0015,
     "ok": true
    },
    {
     "capability": "slug",
     "seconds": 0.0014,
     "ok": true
    },
    {
     "capability": "median",
     "seconds": 0.0013,
     "ok": true
    },
    {
     "capability": "vowels",
     "seconds": 0.0014,
     "ok": true
    },
    {
     "capability": "longest_word",
     "seconds": 0.0013,
     "ok": true
    }
   ],
   "tools_listed_before_first": {
    "count": 0,
    "definitions_bytes": 0,
    "note": "the server does not exist before the first capability"
   },
   "tools_listed_after_tenth": {
    "count": 10,
    "definitions_bytes": 1938
   },
   "second_machine": "not measured: the server would have to be deployed there",
   "summary": {
    "correct_of_10": 10,
    "server_edits": 10,
    "server_restarts": 10,
    "median_seconds_to_first_call": 0.306,
    "repeat_calls": 10,
    "repeat_calls_correct": 10,
    "median_seconds_repeat_call": 0.001
   }
  },
  "generic": {
   "transport": "MCP stdio, generic_server.py (official Python SDK, FastMCP), 4 fixed tools",
   "rows": [
    {
     "n": 1,
     "capability": "word_count",
     "server_edits": 0,
     "server_restarts": 0,
     "seconds_save": 0.002,
     "seconds_run": 0.024,
     "seconds_to_first_call": 0.026,
     "ok": true,
     "save_status": "success"
    },
    {
     "n": 2,
     "capability": "total",
     "server_edits": 0,
     "server_restarts": 0,
     "seconds_save": 0.003,
     "seconds_run": 0.024,
     "seconds_to_first_call": 0.027,
     "ok": true,
     "save_status": "success"
    },
    {
     "n": 3,
     "capability": "reverse_text",
     "server_edits": 0,
     "server_restarts": 0,
     "seconds_save": 0.003,
     "seconds_run": 0.025,
     "seconds_to_first_call": 0.027,
     "ok": true,
     "save_status": "success"
    },
    {
     "n": 4,
     "capability": "is_prime",
     "server_edits": 0,
     "server_restarts": 0,
     "seconds_save": 0.003,
     "seconds_run": 0.023,
     "seconds_to_first_call": 0.026,
     "ok": true,
     "save_status": "success"
    },
    {
     "n": 5,
     "capability": "fibonacci",
     "server_edits": 0,
     "server_restarts": 0,
     "seconds_save": 0.003,
     "seconds_run": 0.024,
     "seconds_to_first_call": 0.026,
     "ok": true,
     "save_status": "success"
    },
    {
     "n": 6,
     "capability": "to_fahrenheit",
     "server_edits": 0,
     "server_restarts": 0,
     "seconds_save": 0.003,
     "seconds_run": 0.022,
     "seconds_to_first_call": 0.025,
     "ok": true,
     "save_status": "success"
    },
    {
     "n": 7,
     "capability": "slug",
     "server_edits": 0,
     "server_restarts": 0,
     "seconds_save": 0.003,
     "seconds_run": 0.024,
     "seconds_to_first_call": 0.026,
     "ok": true,
     "save_status": "success"
    },
    {
     "n": 8,
     "capability": "median",
     "server_edits": 0,
     "server_restarts": 0,
     "seconds_save": 0.003,
     "seconds_run": 0.024,
     "seconds_to_first_call": 0.026,
     "ok": true,
     "save_status": "success"
    },
    {
     "n": 9,
     "capability": "vowels",
     "server_edits": 0,
     "server_restarts": 0,
     "seconds_save": 0.003,
     "seconds_run": 0.023,
     "seconds_to_first_call": 0.026,
     "ok": true,
     "save_status": "success"
    },
    {
     "n": 10,
     "capability": "longest_word",
     "server_edits": 0,
     "server_restarts": 0,
     "seconds_save": 0.003,
     "seconds_run": 0.023,
     "seconds_to_first_call": 0.025,
     "ok": true,
     "save_status": "success"
    }
   ],
   "repeat_calls": [
    {
     "capability": "word_count",
     "seconds": 0.0234,
     "ok": true
    },
    {
     "capability": "total",
     "seconds": 0.0239,
     "ok": true
    },
    {
     "capability": "reverse_text",
     "seconds": 0.0242,
     "ok": true
    },
    {
     "capability": "is_prime",
     "seconds": 0.0236,
     "ok": true
    },
    {
     "capability": "fibonacci",
     "seconds": 0.0244,
     "ok": true
    },
    {
     "capability": "to_fahrenheit",
     "seconds": 0.0241,
     "ok": true
    },
    {
     "capability": "slug",
     "seconds": 0.0253,
     "ok": true
    },
    {
     "capability": "median",
     "seconds": 0.0225,
     "ok": true
    },
    {
     "capability": "vowels",
     "seconds": 0.0226,
     "ok": true
    },
    {
     "capability": "longest_word",
     "seconds": 0.0238,
     "ok": true
    }
   ],
   "session_start_seconds": 0.299,
   "tools_listed_before_first": {
    "count": 4,
    "definitions_bytes": 1866
   },
   "tools_listed_after_tenth": {
    "count": 4,
    "definitions_bytes": 1866
   },
   "tool_names": [
    "get_program",
    "run_program",
    "save_program",
    "search_programs"
   ],
   "tool_list_unchanged": true,
   "server_file_unchanged_sha256": true,
   "search_programs_found": 10,
   "survives_restart": {
    "program": "fibonacci",
    "ok_without_saving_again": true
   },
   "second_machine": "not measured: the server would have to be deployed there",
   "summary": {
    "correct_of_10": 10,
    "server_edits": 0,
    "server_restarts": 0,
    "median_seconds_to_first_call": 0.026,
    "repeat_calls": 10,
    "repeat_calls_correct": 10,
    "median_seconds_repeat_call": 0.024,
    "seconds_first_capability_including_server_start": 0.325
   }
  },
  "extella-mcp": {
   "transport": "MCP streamable HTTP at https://api.extella.ai/mcp/ (the real Extella MCP server; official Python SDK client)",
   "server": {
    "name": "extella-mcp",
    "version": "4.0.0",
    "declares_tools_listChanged": true
   },
   "session_start_seconds": 2.06,
   "tools_listed_before_first": {
    "count": 44,
    "definitions_bytes": 53044
   },
   "tools_listed_after_tenth": {
    "count": 44,
    "definitions_bytes": 53044
   },
   "tool_list_unchanged": true,
   "tool_names": [
    "list_rules",
    "add_rule",
    "delete_rule",
    "update_rule",
    "search_concepts",
    "add_concept",
    "delete_concept",
    "update_concept",
    "run_expert",
    "search_experts",
    "get_expert",
    "save_expert",
    "delete_expert",
    "check_task",
    "list_tokens",
    "create_token",
    "revoke_token",
    "search_kv",
    "get_kv",
    "set_kv",
    "delete_kv",
    "search_targets",
    "add_target",
    "delete_target",
    "set_default_target",
    "get_default_target",
    "get_current_profile_and_agent",
    "add_agent_to_profile",
    "export_chats",
    "export_calls",
    "run_agent",
    "get_agent_run_status",
    "list_agents",
    "get_agent",
    "create_agent",
    "update_agent",
    "delete_agent",
    "list_profiles",
    "create_profile",
    "delete_profile",
    "enable_group",
    "disable_group",
    "duckduckgo_search",
    "report_issue"
   ],
   "largest_tool_definitions_bytes": [
    {
     "name": "update_agent",
     "bytes": 8951
    },
    {
     "name": "create_agent",
     "bytes": 8850
    },
    {
     "name": "run_agent",
     "bytes": 2166
    },
    {
     "name": "run_expert",
     "bytes": 1954
    },
    {
     "name": "search_concepts",
     "bytes": 1504
    }
   ],
   "expert_tools_only": {
    "names": [
     "run_expert",
     "search_experts",
     "get_expert",
     "save_expert",
     "check_task"
    ],
    "count": 5,
    "definitions_bytes": 6611,
    "note": "what a client would carry if it could filter the listing down to these five; the server lists all"
   },
   "rows": [
    {
     "n": 1,
     "capability": "word_count",
     "expert": "mcpm3_word_count",
     "server_edits": 0,
     "server_restarts": 0,
     "seconds_save": 2.56,
     "seconds_run": 7.63,
     "platform_run_time_ms": 6688,
     "seconds_to_first_call": 10.19,
     "ok": true,
     "save_is_error": false
    },
    {
     "n": 2,
     "capability": "total",
     "expert": "mcpm3_total",
     "server_edits": 0,
     "server_restarts": 0,
     "seconds_save": 2.82,
     "seconds_run": 8.46,
     "platform_run_time_ms": 7522,
     "seconds_to_first_call": 11.29,
     "ok": true,
     "save_is_error": false
    },
    {
     "n": 3,
     "capability": "reverse_text",
     "expert": "mcpm3_reverse_text",
     "server_edits": 0,
     "server_restarts": 0,
     "seconds_save": 2.12,
     "seconds_run": 8.53,
     "platform_run_time_ms": 7552,
     "seconds_to_first_call": 10.65,
     "ok": true,
     "save_is_error": false
    },
    {
     "n": 4,
     "capability": "is_prime",
     "expert": "mcpm3_is_prime",
     "server_edits": 0,
     "server_restarts": 0,
     "seconds_save": 1.86,
     "seconds_run": 9.36,
     "platform_run_time_ms": 8390,
     "seconds_to_first_call": 11.22,
     "ok": true,
     "save_is_error": false
    },
    {
     "n": 5,
     "capability": "fibonacci",
     "expert": "mcpm3_fibonacci",
     "server_edits": 0,
     "server_restarts": 0,
     "seconds_save": 2.02,
     "seconds_run": 8.54,
     "platform_run_time_ms": 7556,
     "seconds_to_first_call": 10.56,
     "ok": true,
     "save_is_error": false
    },
    {
     "n": 6,
     "capability": "to_fahrenheit",
     "expert": "mcpm3_to_fahrenheit",
     "server_edits": 0,
     "server_restarts": 0,
     "seconds_save": 1.92,
     "seconds_run": 9.47,
     "platform_run_time_ms": 8389,
     "seconds_to_first_call": 11.39,
     "ok": true,
     "save_is_error": false
    },
    {
     "n": 7,
     "capability": "slug",
     "expert": "mcpm3_slug",
     "server_edits": 0,
     "server_restarts": 0,
     "seconds_save": 2.05,
     "seconds_run": 8.54,
     "platform_run_time_ms": 7561,
     "seconds_to_first_call": 10.59,
     "ok": true,
     "save_is_error": false
    },
    {
     "n": 8,
     "capability": "median",
     "expert": "mcpm3_median",
     "server_edits": 0,
     "server_restarts": 0,
     "seconds_save": 1.94,
     "seconds_run": 9.3,
     "platform_run_time_ms": 8369,
     "seconds_to_first_call": 11.24,
     "ok": true,
     "save_is_error": false
    },
    {
     "n": 9,
     "capability": "vowels",
     "expert": "mcpm3_vowels",
     "server_edits": 0,
     "server_restarts": 0,
     "seconds_save": 2.42,
     "seconds_run": 8.52,
     "platform_run_time_ms": 7547,
     "seconds_to_first_call": 10.95,
     "ok": true,
     "save_is_error": false
    },
    {
     "n": 10,
     "capability": "longest_word",
     "expert": "mcpm3_longest_word",
     "server_edits": 0,
     "server_restarts": 0,
     "seconds_save": 2.19,
     "seconds_run": 9.32,
     "platform_run_time_ms": 8372,
     "seconds_to_first_call": 11.51,
     "ok": true,
     "save_is_error": false
    }
   ],
   "repeat_calls": [
    {
     "capability": "word_count",
     "seconds": 10.23,
     "platform_run_time_ms": 9235,
     "ok": true
    },
    {
     "capability": "total",
     "seconds": 10.26,
     "platform_run_time_ms": 9260,
     "ok": true
    },
    {
     "capability": "reverse_text",
     "seconds": 9.4,
     "platform_run_time_ms": 8407,
     "ok": true
    },
    {
     "capability": "is_prime",
     "seconds": 10.26,
     "platform_run_time_ms": 9270,
     "ok": true
    },
    {
     "capability": "fibonacci",
     "seconds": 11.04,
     "platform_run_time_ms": 10057,
     "ok": true
    },
    {
     "capability": "to_fahrenheit",
     "seconds": 11.21,
     "platform_run_time_ms": 10211,
     "ok": true
    },
    {
     "capability": "slug",
     "seconds": 11.09,
     "platform_run_time_ms": 10126,
     "ok": true
    },
    {
     "capability": "median",
     "seconds": 10.23,
     "platform_run_time_ms": 9268,
     "ok": true
    },
    {
     "capability": "vowels",
     "seconds": 7.6,
     "platform_run_time_ms": 6362,
     "ok": true
    },
    {
     "capability": "longest_word",
     "seconds": 9.29,
     "platform_run_time_ms": 8333,
     "ok": true
    }
   ],
   "second_machine": [
    {
     "n": 1,
     "capability": "word_count",
     "seconds": 5.75,
     "platform_run_time_ms": 4781,
     "ok": true
    },
    {
     "n": 5,
     "capability": "fibonacci",
     "seconds": 7.68,
     "platform_run_time_ms": 6704,
     "ok": true
    },
    {
     "n": 10,
     "capability": "longest_word",
     "seconds": 7.28,
     "platform_run_time_ms": 6282,
     "ok": true
    }
   ],
   "summary": {
    "correct_of_10": 10,
    "server_edits": 0,
    "server_restarts": 0,
    "median_seconds_to_first_call": 11.085,
    "repeat_calls": 10,
    "repeat_calls_correct": 10,
    "median_seconds_repeat_call": 10.245,
    "seconds_first_capability_including_session_start": 12.25,
    "median_seconds_save": 2.08,
    "median_seconds_first_run": 8.54,
    "median_platform_run_time_ms_repeat": 9264.0,
    "second_device_correct": "3 of 3",
    "second_device_seconds": [
     5.75,
     7.68,
     7.28
    ]
   }
  }
 },
 "what_failed": [],
 "mcp_sdk": "1.26.0"
}