One handler, many programs: what changes on an update

Ledger: four short programs and a replaced handler

Question

If the handler of a language is replaced, do the saved programs of that language run under the new one without being saved again?

Conditions

A toy language ledger of two statements; four programs in it and one text that is not a ledger program. A laptop and a Linux server. Every step was a call of the MCP tools save_expert and run_expert, made by a script.

Result

• Under handler 1 the four programs gave the expected totals (10, 99, 42, 21); the text with import os was refused by the handler.
• After handler 2 was registered, the same four programs, not saved again, ran under it with the same totals and the new fields it adds.
• At the same time the server, with handler 1, ran two of the programs the old way.

Limits

One day, one laptop and one server, a toy language. No sandbox: the handler runs with the rights of the user who runs Extella. The platform keeps no versions of handlers.

Ten programs through the platform and a plain MCP server

Question

What does it take to add a new program as a tool: fixed tools of an MCP server, a general executor with program storage, or Extella?

Conditions

The same ten small Python functions added three ways. A script made every call; no model drove the servers. The two local ways ran as local processes; Extella dispatched to a device, which was the same laptop. Only Extella was also run on a second machine.

Result

• Fixed tools: 10 edits and 10 restarts. General executor and Extella: 0 and 0.
• A repeat call: 0.0014 s, 0.025 s, and 6 to 14 s through Extella (median about 9 s, 30 calls).
• On the second machine Extella answered 30 of 30 calls correctly without a second save, in 5 to 9 s.

Limits

The general executor was not run on a second machine, so the run shows no advantage over it there. The seconds compare local processes with a remote service. It says nothing about the products of Anthropic or Cloudflare.

Restyling one report

Question

Where does a new look of a report go when the report is a program in a presentation language, and what does the change cost?

Conditions

The same change in two builds of one application: A on Extella (the look is set by the handler of the presentation language), B as a plain Python program. Each change was made by a new AI assistant that had not seen the code.

Result

• Both builds changed the look in one place and kept the content of the report: in A one file, the handler (+21/−9 lines); in B two files (+22/−11).
• On this one report Extella saved nothing: 93,734 tokens and 2.3 minutes against 81,191 tokens and 1.7 minutes; rebuilding the report took 10.3 s against 0.23 s.

Limits

One run per build: an observation, not an average. The gain from one handler shows when many programs share it, see the next experiment.

Updating 40 reports

Question

One handler serves 40 saved programs. What happens when it gets a new version, including a broken one, and can the update be checked before it goes in?

Conditions

The expected total of each program was computed beforehand by a script straight from the program lines, independently of the handler. Version 3 had a planted defect: it dropped the kopecks of every amount, without raising any error.

Result

• A new rule: one registration, 8.5 s; all 40 programs answered by the new version without being saved again.
• The broken version registered without a check silently changed the totals of 35 programs out of 40, with no error on the screen.
• A check run before the replacement ran all 40 programs on the candidate in 10 s, found 35 mismatches and did not register it. Restoring the previous version through the same check: 10 s, 40 of 40 correct.

Limits

The check catches only what the expected results cover. Most of the 10 s is the trip through the platform. The same scheme works with a shared library and tests outside Extella; it is not a property only Extella has.

Fund value on a laptop and a server

Question

One operation has three users: two on a laptop, one on a server. What happens when it is replaced from Python to Go while a call is running?

Conditions

The same contract, a 30-second call in flight during the replacement, then a rollback. Three ways: an Extella program with a Go language handler, a plain program copied to both machines, and Windmill CE v1.825.0 in local Docker on one machine.

Result

• The replacement: one action in 2.4 s on Extella, eight actions in 15 s for the plain program, one action in 0.54 s on Windmill. The call in flight finished on the old implementation in all three.
• On Extella the replacement also reached the server, which has no Go compiler, and the third user failed there. The plain program shipped a built binary and kept working; Windmill has Go in its worker image.
• A call took 9.5–11 s on Extella, 0.4–0.9 s for the plain program and 0.57–0.69 s on Windmill. Rollback on Extella worked only because the old text was kept locally: the platform keeps no versions. Windmill keeps versions by hash.

Limits

One run per configuration. Windmill was not run on a second machine: in the free edition a remote worker without access to the server database did not connect ("Agent mode is only available in the EE"). Extella ran on a second machine behind a router.

A site made of blocks

Question

Does one saved program run unchanged under different handlers?

Conditions

One expert in the language block (a JSON document describing a quote on a web page). Three handlers on two Linux servers, six runs. Two handlers produce HTML, the third is separately written code that renders plain text.

Result

• The same program ran under all three handlers without being edited. Registering a handler took 6.8 to 7.9 s.
• The model-call counter was 0 in every run; the handler hash returned by the device matched the published file in all six.

Limits

One program and six runs. That the same holds for every program of a language is the stated design of the runtime; we did not measure it on a larger set here.

The live example is updated over time; the results describe the run on the date above.

Shift schedules and a small model

Question

Does a small model build valid shift schedules more often in a short checked language than in Python with the same checks?

Conditions

Six tasks. A: Python, one attempt, no checks. B: Python, the rules checked after each attempt and errors returned, up to 3 attempts. C: a small schedule language, the rules in the handler on a device through Extella, up to 3 attempts. Small model qwen-flash, three runs; large model Qwen max, one run. Repeated on 15 October after Ella's review: the coverage check was made exact and the instructions of B and C were aligned.

Result

• Small model: A 2 of 18 (47 thousand tokens), B 6 of 18 (154 thousand), C 10 of 18 (27 thousand). B used 5.7 times more tokens than C.
• Large model: 6 of 6 in every way, with no difference in tokens.

Limits

Tasks and handlers were written by us, and the grader checks the same rules as the handler. Six tasks, two models of one family. The run does not separate the effect of the language from the place of the check. In one run B solved a task C did not. Every step through the platform is 7 to 17 s slower.