VerifiedMeasured 2026-07-25 · task suite v1.0.0 · claude-sonnet-5
NativeScript vs. LynxJS
The same iOS app — same spec, same UI, two deep native features — built by isolated AI agents in both frameworks, with every token metered and the finished products performance-profiled.
5 trials per framework · 30/30 phases build-green · every number below is generated from the published raw data — nstudio/token-economics ↗
Output tokens per app
1.9×
LynxJS 153.1K vs NativeScript 81.7K median — and the per-trial ranges never overlap.
Estimated cost per app
2.4×
LynxJS $18.29 vs NativeScript $7.52 median, list-price estimate.
Native-language code written
226 vs 0
Lines of Swift per app. Both agents wrote the same 16 lines of plist/entitlement config; only LynxJS also authored a native module. JS/Vue volume was equal.
Second native feature
4.0×
Phase 3 is the mechanism: bridge cost recurs per feature; direct-access cost amortizes after the first.
01Every trial, no overlap — total output tokens
Each dot is one complete app build (three phased agent sessions, summed). The most expensive NativeScript trial used fewer output tokens than the cheapest LynxJS trial — clean separation at n=5 per side.
NativeScriptLynxJS
each dot = one complete app build · output tokens NativeScriptmedian 81,697 050K100K150K200K
02The mechanism — bridge cost recurs, direct access amortizes
After learning the direct-TypeScript platform pattern in phase 2, the NativeScript agent’s second native feature cost 71% less — Speech needed 2 native lines, both config, and zero Swift. The Lynx agent wrote the same 2 config lines plus a second complete Swift module (106 lines) with bridge registration and streaming-callback plumbing, so its phase 3 stayed as expensive as its phase 2 (−9%).
NativeScriptLynxJS
output tokens · median of 5 trialsUI shellLynxJS 1.7× NativeScript
HealthKitLynxJS 1.3× NativeScript
SpeechLynxJS 4.0× NativeScript
03Where the tokens go
The four usage buckets are always reported separately: cache reads dominate raw counts in multi-turn agentic sessions (the Lynx side re-read vendored Pods/Sparkling sources heavily in place of docs lookups), which is why a single collapsed “total tokens” would drown the signal. Output tokens — the agent’s actual work — are the headline metric.
NativeScript15,830,028 all buckets
Input · 0.00%Output · 0.52%Cache read · 98%Cache write · 2%
LynxJS42,163,122 all buckets
Input · 0.00%Output · 0.36%Cache read · 98%Cache write · 1%
04What you get — runtime performance of the finished apps
App size on disk
−49%
LynxJS 26.1 MB vs NativeScript 51.1 MB as an unsigned arm64 device archive — and still 5.3 MB of that is the DevTool stack Sparkling ships in Release.
Idle memory (RSS)
+34%
LynxJS 288 MB vs NativeScript 215 MB on the same screen; physical footprint agrees (+26%).
Cold launch
tie
316 vs 317 ms to foreground-active; both idle at 0.0% CPU.
API-path latency
wash
HealthKit query is a dead tie; Lynx wins the write→refresh round trip by ~30%.
Corrected 2026-08-02: The app-size finding was wrong, and reversed on re-measurement. What changed →
05The two interop models
Runtime reflection over native metadata
NativeScript
Direct reflection bridge over the native runtime
NativeScript runs JS/TS directly against the native platform runtimes — V8 on iOS and Android — and exposes native classes to JavaScript through metadata generated at build time from the platform SDKs. There is no fixed allow-list of wrapped components: in principle, any public Android or iOS API is reachable directly from JS by referencing its fully-qualified native name (for example, constructing an android.widget.TextView or calling into UIKit types directly).
Full TypeScript declarations for the native API surface are generated ahead of time (via @nativescript/types), so an agent working with those types in context has precise signatures available. The tradeoff is vocabulary size: native namespaces are long, platform-specific, and often diverge between iOS and Android, so a cross-platform call typically means writing and branching two native code paths rather than one shared call.
Strengths
- +No bridge-authoring step for capabilities already in the native SDK — call it directly
- +Complete, generated TypeScript typings for the native surface when included in context
- +Single-file reach: a novel native call can usually be added without touching native-language source
Friction points
- −Verbose, fully-qualified native symbol names inflate output tokens per call site
- −iOS and Android APIs diverge, so cross-platform code often means two branches, not one
- −Correctness depends on knowing native marshalling/threading rules (e.g. main-thread dispatch) that aren’t enforced by the type system
Authored Native Modules with a defined JS-facing API
LynxJS
Explicit native module bridge
LynxJS renders through its own engine and dual-thread architecture rather than a WebView, and extends into native platform code through Native Modules: a native class (Kotlin/Java, Swift/Obj-C) explicitly registers the methods it wants to expose, and JS calls into it through a defined module API. There is no arbitrary reflection over the native SDK — if a capability hasn’t been wrapped in a module, it isn’t callable from JS yet.
This narrows the vocabulary an agent needs on the JS side to whatever a module’s author decided to expose, which should make call sites for already-wrapped capabilities short and framework-idiomatic rather than platform-specific. The cost moves elsewhere: reaching a capability with no existing module means writing genuine native-language bridge code and registering it, spanning multiple files and at least two languages before any JS call can be made.
Strengths
- +Terse, curated JS-facing API for already-wrapped capabilities — no native vocabulary required at the call site
- +A narrower interop surface is easier to keep consistent across iOS and Android from the JS side
- +Native module boundaries are explicit, which may reduce misuse of platform threading/marshalling rules
Friction points
- −No existing module means writing and registering native bridge code before any JS call is possible
- −Bridge authoring spans multiple files and languages, which likely costs more total tokens and more agentic tool-call round-trips for genuinely new capabilities
- −Younger ecosystem (open-sourced 2025) means fewer pre-built modules to fall back on today
06Held constant across both frameworks
- –Model (claude-sonnet-5) and agent harness (headless Claude Code 2.1.220), pinned for every session
- –Byte-identical SPEC.md and audio fixture committed to both repos; every user-visible label pinned exactly
- –Structurally identical CLAUDE.md files — same sections, same level of help, no implementation hints
- –One official docs MCP server per framework, and nothing else, via strict MCP config
- –Fresh isolated session per phase (throwaway config dir: no memory, no personal settings)
- –Per-phase turn cap (160), build gates run by the harness — never self-reported by the agent
- –Interleaved trial order (NS, Lynx, NS, …), strictly sequential, same machine and Xcode 26.5
- –No hidden retries: every started trial is in the published record
07The task suite
01Phase 1 — App shell (shared Vue 3 UI)
Home screen plus two placeholder feature screens with working navigation, exact labels pinned by spec. No native calls — both frameworks write Vue 3, isolating baseline framework fluency.
02Phase 2 — HealthKit (first native feature)
Step-count read/write authorization, logging quantity samples, and a 7-day statistics query rendered as a list. Entitlements, permission flows, and async native data marshaling.
03Phase 3 — Speech (second native feature)
Transcribe a bundled audio clip with streaming partial results into the UI. The second native feature reveals whether platform-access cost amortizes (direct access) or recurs (per-feature bridge authoring).
04Acceptance & runtime performance
Operator checklist against the pinned spec, then Release rebuilds of every implementation profiled for size, memory, launch, and tap-to-result platform-API latency on one fixed simulator.
08Raw data — all 30 measured phases
Per trial, per phase — the rows behind every chart above. Full transcripts, diffs, and manifests are in the repository.
| Trial | Phase | Output | Est. cost | Turns | Wall (s) | Docs | JS LOC | Native LOC |
|---|
| main-ns-1 | 1 | 11,466 | $1.17 | 54 | 218 | 7 | 51 | 0 |
| 2 | 51,535 | $5.14 | 124 | 1,007 | 7 | 200 | 15 |
| 3 | 15,403 | $1.21 | 40 | 293 | 5 | 125 | 2 |
| main-ns-2 | 1 | 8,979 | $0.95 | 49 | 169 | 3 | 52 | 0 |
| 2 | 53,760 | $3.45 | 80 | 799 | 15 | 184 | 14 |
| 3 | 14,747 | $0.87 | 33 | 172 | 4 | 103 | 2 |
| main-ns-3 | 1 | 12,932 | $1.29 | 52 | 224 | 4 | 59 | 0 |
| 2 | 59,840 | $6.56 | 152 | 972 | 10 | 169 | 15 |
| 3 | 12,158 | $0.82 | 33 | 153 | 0 | 99 | 2 |
| main-ns-4 | 1 | 16,571 | $1.92 | 77 | 421 | 7 | 58 | 0 |
| 2 | 52,794 | $5.85 | 147 | 884 | 13 | 203 | 13 |
| 3 | 26,538 | $2.74 | 71 | 487 | 7 | 135 | 3 |
| main-ns-5 | 1 | 16,198 | $1.62 | 58 | 256 | 8 | 49 | 0 |
| 2 | 47,439 | $2.47 | 60 | 516 | 12 | 248 | 13 |
| 3 | 18,060 | $1.25 | 39 | 249 | 5 | 93 | 2 |
| main-lynx-1 | 1 | 19,850 | $1.61 | 48 | 240 | 1 | 137 | 0 |
| 2 | 67,499 | $6.63 | 133 | 863 | 3 | 124 | 127 |
| 3 | 65,761 | $9.61 | 169 | 1,172 | 7 | 84 | 108 |
| main-lynx-2 | 1 | 22,525 | $2.56 | 78 | 334 | 2 | 231 | 0 |
| 2 | 55,474 | $4.87 | 88 | 849 | 3 | 85 | 148 |
| 3 | 61,660 | $10.86 | 184 | 1,163 | 6 | 77 | 94 |
| main-lynx-3 | 1 | 26,324 | $3.39 | 90 | 456 | 3 | 155 | 0 |
| 2 | 113,124 | $15.61 | 161 | 4,134 | 5 | 101 | 136 |
| 3 | 54,119 | $6.93 | 154 | 1,029 | 2 | 79 | 100 |
| main-lynx-4 | 1 | 18,372 | $1.60 | 51 | 252 | 1 | 182 | 0 |
| 2 | 111,714 | $12.80 | 209 | 1,637 | 3 | 184 | 241 |
| 3 | 46,301 | $6.13 | 112 | 758 | 3 | 137 | 180 |
| main-lynx-5 | 1 | 24,710 | $2.59 | 76 | 377 | 1 | 173 | 0 |
| 2 | 57,221 | $4.96 | 95 | 654 | 3 | 123 | 218 |
| 3 | 64,186 | $7.86 | 149 | 1,049 | 2 | 96 | 165 |
09Go deeper
Deep diveMethodology →
Trial protocol, isolation controls, fairness design, measurement instrumentation, and the disclosed confounders.
Deep divePerformance →
Release-build size composition, memory, launch, idle CPU, and tap-to-result platform-API latency, measured across every agent-built app.
Deep diveReproduce it →
Run the benchmark yourself: harness commands, requirements, pinned versions, and what gets archived per trial.
Replicate itEvery artifact behind these numbers is published.
Baselines, per-trial diffs and transcripts, build logs, perf JSON, and the harness that produced them. Run it against a different model, a new framework version, or your own framework pair — and submit the results.