VerifiedMeasured 2026-08-01 · task suite v1.1.0 · claude-sonnet-5
NativeScript vs. Expo
One iOS app with two deep native features — HealthKit and Speech — built from a byte-identical spec by isolated AI agents in NativeScript and in Expo (React Native), under a spec that forbids third-party wrappers for the platform capability under test.
8 trials per framework · 78/78 phases build-green across two studies · every number below is generated from the published raw data — nstudio/token-economics ↗
Framework effect on token cost
none
With verification effort held equal on both arms, NativeScript and Expo are statistically indistinguishable (Mann-Whitney U=13/25, exact p=1.00; the null expectation is 12.5).
What actually predicts cost
84%
Variance in token cost explained by how much the agent verified its own work. Framework choice explains 6%.
Price of one verification step
~1,160
Output tokens per interactive UI check (t=8.04). The most portable number in this study.
Native-language code written
209 vs 0
Lines of Swift per Expo app across two local modules; NativeScript wrote none, reaching the same APIs from TypeScript — at the same token cost.
01Every trial — total output tokens
Each dot is one complete app build (three phased agent sessions, summed). The two arms’ ranges overlap heavily — this pairing is close enough that agent behaviour, not framework choice, drives the spread.
NativeScriptExpo
each dot = one complete app build · output tokens NativeScriptmedian 84,366 037.5K75K112.5K150K
02The mechanism — verification effort, not interop architecture
Agents chose how much to drive the simulator, and that choice correlates with total token cost at r = 0.89 — within each arm and pooled. NativeScript’s agents verified in every trial (20–37 UI calls); Expo’s ranged from zero to 37. Controlling for it flips the sign of the framework coefficient and lifts explained variance from 6% to 84%. A companion study pinned verification off on both arms and found no difference at all.
NativeScriptExpo
output tokens · median of 8 trialsUI shellExpo 0.7× NativeScript
HealthKitExpo 0.7× NativeScript
SpeechExpo 1.0× NativeScript
03Where the tokens go
The four usage buckets are reported separately: cache reads dominate raw counts in multi-turn agentic sessions, which is why a single collapsed “total tokens” would drown the signal. Output tokens — the agent’s actual work — are the headline metric.
NativeScript21,278,729 all buckets
Input · 0.00%Output · 0.40%Cache read · 98%Cache write · 1%
Expo11,362,374 all buckets
Input · 0.00%Output · 0.63%Cache read · 97%Cache write · 2%
04What you get — runtime performance of the finished apps
App size on disk
−44%
Expo 28.7 MB vs NativeScript 51.1 MB as an unsigned arm64 device archive, with an 89% smaller main executable.
Cold launch
tie
302 vs 301 ms to foreground-active; both idle at 0.0% CPU.
Idle memory
mixed
RSS favours NativeScript (216 vs 228 MB); physical footprint favours Expo (34 vs 40 MB). Neither gap is decisive.
Functional acceptance
8/8 · 8/8
Both arms pass every spec flow when agents could verify freely. With verification forbidden, Expo drops to 4/5 — one app whose HealthKit authorization never resolves at runtime.
Corrected 2026-08-02: App size was measured on simulator builds, which are not app sizes. What changed →
05The two interop models
Runtime reflection over native metadata
NativeScript
Direct reflection bridge over the native runtime
NativeScript runs JS/TS directly against the native platform runtimes and exposes native classes to JavaScript through metadata generated at build time from the platform SDKs. Any public iOS API is reachable from TypeScript by referencing its native name — there is no wrapping step and no native-language source to author.
Across both platform features in this study that meant a measured zero lines of native-language code; all the native-side work was Info.plist and entitlement configuration. Whether that translates into a token saving against Expo is the question this comparison asks.
Strengths
- +No bridge-authoring step for capabilities already in the native SDK — call it directly
- +Complete, generated TypeScript typings for the native surface
- +Zero lines of native-language code for both platform features — all native work was configuration
Friction points
- −Verbose, fully-qualified native symbol names inflate output tokens per call site
- −Smaller ecosystem and far less training-data representation than React Native
- −Correctness depends on native marshalling/threading rules the type system does not enforce
Authored Expo Modules with a generated scaffold
Expo
Autolinked native modules over JSI
Expo extends into native platform code through the Expo Modules API: `npx create-expo-module --local` scaffolds a module, a Swift `ModuleDefinition` declares what it exposes, and autolinking wires it up with no manual registration. Calls cross over JSI rather than an async bridge.
Native configuration is declarative — entitlements and Info.plist keys live in app.json or a config plugin, because Continuous Native Generation regenerates the iOS project. That layer may well prove cheaper than editing native files directly; the Swift authorship is where the cost should land.
Strengths
- +Largest training-data footprint of any mobile stack, and the most mature ecosystem
- +Module scaffolding is generated, and autolinking removes the registration step entirely
- +Declarative native configuration through app.json and config plugins
Friction points
- −A capability with no module still means authoring Swift, per capability
- −Continuous Native Generation regenerates ios/, so native edits must go through config plugins or be lost
- −The official docs MCP exposes 27 tools including package install and EAS build control — broad surface, and its schemas ride in context every turn
06Held constant across both frameworks
- –Model (claude-sonnet-5) and agent harness (headless Claude Code 2.1.220), pinned for every session on both arms
- –Byte-identical SPEC.md v1.1.0 and audio fixture in both repos, enforced by harness/sync-spec.sh --check, not by convention
- –Structurally identical CLAUDE.md files — same sections, same level of help, no implementation hints
- –One official docs MCP per framework and nothing else, verified per trial by an invocation probe that aborts the trial if the server fails to load
- –Expo’s template-shipped agent onboarding (AGENTS.md, CLAUDE.md, and an enabled official Claude Code plugin) removed at baseline so both arms get the same level of help — recorded verbatim and disclosed as a real advantage held constant
- –Expo MCP local tier (simulator tap/screenshot) left off: no other arm has a visual verification loop
- –Fresh isolated session per phase (throwaway config dir: no memory, no personal settings)
- –Per-phase turn cap (160), build gates run by the harness — never self-reported by the agent
- –Interleaved trial order (NS, Expo, NS, …), strictly sequential, same machine and Xcode 26.5
- –Both arms measured fresh inside a single window — no numbers carried over from any earlier run
- –No hidden retries: every started trial is in the published record
07The task suite
01Phase 1 — App shell
Home screen plus two placeholder feature screens with working navigation, exact labels pinned by spec. No native calls. Expo writes React where NativeScript writes Vue — the one unavoidable asymmetry, and this phase is the read on what it is worth.
02Phase 2 — HealthKit (first native feature)
Step-count read/write authorization, logging quantity samples, and a 7-day statistics query rendered as a list. Under v1.1 no third-party HealthKit wrapper may be installed, so Expo must author a local native module.
03Phase 3 — Speech (second native feature)
Transcribe a bundled audio clip with streaming partial results. The decisive phase: whether a second native capability reuses the first one’s scaffolding (as direct access did) or costs full price again (as bridge authoring did).
04Acceptance & runtime performance
Automated functional acceptance driving every Release app through the spec flows, then size, memory, launch and tap-to-result latency on one fixed simulator. Both arms pass 8/8 with free-choice verification; with verification forbidden Expo drops to 4/5.
08Raw data — all 24 measured phases
Per trial, per phase — the rows behind every chart above. Full transcripts, diffs, and manifests are in the repository.
| Trial | Phase | Output | Est. cost | Turns | Wall (s) | Docs | JS LOC | Native LOC |
|---|
| main-ns-1 | 1 | 10,674 | $0.99 | 47 | 187 | 0 | 57 | 0 |
| 2 | 42,884 | $4.27 | 114 | 802 | 8 | 157 | 14 |
| 3 | 12,523 | $0.78 | 30 | 218 | 4 | 100 | 2 |
| main-ns-2 | 1 | 16,377 | $1.87 | 63 | 272 | 8 | 65 | 0 |
| 2 | 28,063 | $2.11 | 64 | 375 | 9 | 160 | 14 |
| 3 | 50,321 | $10.26 | 199 | 2,225 | 4 | 103 | 2 |
| main-ns-3 | 1 | 17,157 | $1.37 | 53 | 391 | 7 | 34 | 0 |
| 2 | 52,071 | $5.90 | 156 | 1,072 | 12 | 161 | 13 |
| 3 | 42,447 | $5.55 | 149 | 868 | 2 | 112 | 2 |
| main-ns-4 | 1 | 16,531 | $1.43 | 56 | 273 | 8 | 39 | 0 |
| 2 | 47,201 | $6.32 | 137 | 1,002 | 3 | 232 | 12 |
| 3 | 46,658 | $5.71 | 139 | 1,012 | 6 | 171 | 2 |
| main-ns-5 | 1 | 12,531 | $1.40 | 56 | 234 | 7 | 57 | 0 |
| 2 | 57,208 | $6.95 | 167 | 1,097 | 6 | 204 | 12 |
| 3 | 14,627 | $0.93 | 33 | 168 | 2 | 146 | 2 |
| main-ns-6 | 1 | 9,253 | $0.80 | 40 | 170 | 2 | 51 | 0 |
| 2 | 32,565 | $2.95 | 82 | 555 | 3 | 155 | 12 |
| 3 | 36,521 | $3.87 | 97 | 623 | 5 | 106 | 2 |
| main-ns-7 | 1 | 13,423 | $1.44 | 61 | 381 | 4 | 43 | 0 |
| 2 | 30,162 | $1.90 | 62 | 479 | 8 | 146 | 12 |
| 3 | 13,575 | $0.99 | 32 | 185 | 3 | 83 | 2 |
| main-ns-8 | 1 | 12,134 | $1.30 | 54 | 216 | 6 | 53 | 0 |
| 2 | 52,032 | $4.89 | 125 | 839 | 7 | 148 | 12 |
| 3 | 18,175 | $1.12 | 37 | 221 | 3 | 87 | 2 |
| main-expo-1 | 1 | 12,225 | $1.40 | 56 | 604 | 0 | 216 | 0 |
| 2 | 24,442 | $1.12 | 33 | 329 | 0 | 182 | 122 |
| 3 | 22,641 | $1.27 | 43 | 307 | 0 | 158 | 117 |
| main-expo-2 | 1 | 5,917 | $0.47 | 17 | 142 | 0 | 157 | 0 |
| 2 | 26,538 | $1.53 | 56 | 377 | 1 | 180 | 154 |
| 3 | 26,052 | $1.84 | 52 | 429 | 2 | 123 | 122 |
| main-expo-3 | 1 | 14,937 | $0.00 | 1 | 626 | 0 | 246 | 0 |
| 2 | 29,729 | $1.68 | 51 | 376 | 2 | 184 | 167 |
| 3 | 19,172 | $1.05 | 35 | 230 | 1 | 152 | 125 |
| main-expo-4 | 1 | 9,948 | $0.00 | 3 | 364 | 0 | 249 | 0 |
| 2 | 39,872 | $3.95 | 114 | 728 | 1 | 181 | 138 |
| 3 | 44,368 | $3.47 | 100 | 743 | 0 | 140 | 158 |
| main-expo-5 | 1 | 6,610 | $0.50 | 21 | 149 | 0 | 198 | 0 |
| 2 | 35,253 | $3.34 | 113 | 698 | 0 | 175 | 134 |
| 3 | 30,275 | $2.05 | 52 | 465 | 3 | 119 | 141 |
| main-expo-6 | 1 | 10,811 | $1.12 | 50 | 517 | 0 | 230 | 0 |
| 2 | 32,836 | $1.99 | 55 | 510 | 1 | 139 | 147 |
| 3 | 21,171 | $1.35 | 40 | 419 | 3 | 123 | 124 |
| main-expo-7 | 1 | 7,621 | $0.51 | 18 | 130 | 0 | 265 | 0 |
| 2 | 27,453 | $1.79 | 42 | 441 | 0 | 157 | 143 |
| 3 | 52,758 | $4.58 | 112 | 1,006 | 0 | 124 | 146 |
| main-expo-8 | 1 | 7,398 | $0.52 | 16 | 137 | 0 | 260 | 0 |
| 2 | 42,930 | $4.47 | 126 | 817 | 0 | 156 | 138 |
| 3 | 61,143 | $7.48 | 170 | 1,262 | 0 | 145 | 126 |
09Go deeper
Deep diveMethodology →
Trial protocol, isolation controls, fairness design, measurement instrumentation, and the disclosed confounders.
Deep divePerformance →
Release-build size composition, memory, launch, idle CPU, and tap-to-result platform-API latency, measured across every agent-built app.
Deep diveReproduce it →
Run the benchmark yourself: harness commands, requirements, pinned versions, and what gets archived per trial.
Replicate itEvery artifact behind these numbers is published.
Baselines, per-trial diffs and transcripts, build logs, perf JSON, and the harness that produced them. Run it against a different model, a new framework version, or your own framework pair — and submit the results.