Porting a real CLI to 0.0.17: unlowered surface, and no dynamic escape hatch for host globals

[ryoppippi]

I ported ax — a ~2000-line HTML/HTTP CLI that depends on linkedom — to scriptc 0.0.17. It works: --dynamic produces a 2.5 MB binary whose output is byte-identical to the Bun build across 32 differential cases (fetch, CSS extraction, markdown, table parsing with colspan/rowspan, a --where expression language, stdin, -o, -I, -u, -H, -d, error paths). Streaming response bodies and a non-standard fetch init field both survived. That is impressive for an experimental compiler, so thank you.

Filing the unlowered surface I hit as one list rather than a dozen issues — happy to split any of these out if that is more useful. Everything below is from an actual diagnostic, not speculation.

Unlowered surface (SC2020 / SC1090)

Node builtins

• util.parseArgs — the whole CLI front end depended on it; replaced with ~120 lines of hand-rolled parsing.
• fs.rename / fs.renameSync — absent entirely. This removes write-to-temp-then-atomically-rename, which is how a cache entry or a download avoids being observed half-written. No substitute exists (copyFileSync is not atomic).
• fs.writeSync — with no rename either, incremental file output is unavailable; a download has to be buffered whole and written once.
• fs/promises.open — no FileHandle, same consequence.
• fs/promises.writeFile with 3 arguments — writeFileSync(path, text, { mode }) lowers, so mode is reachable only through the sync form.
• fs.writeFileSync with 3 arguments when the payload is bytes — mode works for strings, not for Uint8Array.
• writeFileSync(1, …) / readFileSync(0) asymmetry — reading fd 0 lowers (great, that is how stdin works), writing fd 1 does not.

Globals / stdlib

• Object.keys, Object.values, Object.entries, Object.fromEntries — for...in covers keys well enough; entries/fromEntries need explicit loops at every site.
• Array.prototype.unshift, Array.prototype.reverse.
• WeakMap — the hint (use Map) is good, but Map keys are limited to string/number, so keying a memo on object identity needs a hand-rolled serial-number tag on each object.
• new TextDecoder(label) beyond the default utf-8. This is the one real functional loss in the port: charset-aware decoding falls back to Buffer.from(bytes).toString(enc), which covers utf8/utf16le/latin1 but not shift_jis, euc-jp, windows-125x or gbk.
• Buffer.prototype.toString(enc) requires a literal encoding — a variable holding 'utf16le' is refused, so the dispatch has to be unrolled into one branch per literal.
• process.stdout.write — only a single string argument. No completion callback (so no process.exit drain barrier) and no byte writes (so binary stdout has to round-trip through a decode).
• String.prototype.split(sep, limit) and String.prototype.startsWith(s, pos) — the 2-argument forms.
• Response.body.getReader() statically (see below — reachable dynamically).
• Promise.race over a non-Promise-typed entry.
• Number(x) and Array.isArray(x) where x is any.
• Uint8Array.prototype.set from an any source.
• instanceof against a built-in class (x instanceof RegExp).
• globalThis.

Typing constraints that shaped the port

Not bugs, but the rules that drove the most rewriting — a "porting real code" guide mentioning them would have saved me hours:

• A function value whose parameter is unknown or any cannot be compiled, so any callback reaching .filter()/.map() needs a concrete parameter type. This propagated back through several public signatures.
• Indexing Record<string, unknown> has no lowering, and indexing a concrete index-signature type is typed | undefined under noUncheckedIndexedAccess, which the keyed-read lowering also rejects. Both had to route through a dynamic receiver.
• An optional parameter of type Map<…> makes a union with no runtime narrowing test, so a memoized/unmemoized function pair cannot share one entry point.
• Spreads must come first in an object literal, and a spread of a computed source must be bound to a const first. Conditional-tail spreads (...(cond ? { k: v } : {})) — a common way to build optional JSON fields — need rewriting to explicit k: cond ? v : undefined.

The dynamic escape hatch is inconsistent

This is the item I would most like to see addressed, because it is what makes the gaps above unworkable rather than merely inconvenient.

Routing a value through any does reach the embedded engine's full surface — this is how the port kept streaming response bodies and passed a non-standard fetch init field:

// works: the direct call lowers, and the result widens to dyn
const res: any = await fetch(url, init)
const reader = res.body.getReader()      // unlowered statically, fine here

const init: any = { tls: { rejectUnauthorized: false } }
await fetch(url, init)                   // field outside RequestInit, fine

But the same trick is refused for host globals:

const g: any = globalThis                // SC1090: the reference to 'g' (a binding form with no lowering)
new (globalThis as any).TextDecoder(l)   // SC2020: 'globalThis' has no scriptc lowering
const p: any = process                   // SC1090: the reference to 'p'
await (fetch as any)(url)                // SC2020: 'fetch' has no scriptc lowering

So TextDecoder('shift_jis') and process.stdout.write(bytes, cb) are unreachable by any route, static or dynamic. Because the refusal is at compile time, a runtime capability check cannot avoid it either — I had to split the two affected functions into separate Bun and scriptc source files and substitute one at build time.

If --dynamic builds allowed globalThis (or an explicit opt-in like scriptc.host) to yield a dyn value, every gap in this issue would become a one-line workaround instead of a build-system change.

Environment

• scriptc 0.0.17, @scriptc/compiler 0.0.17
• macOS 26.5.2, arm64, Node v24.18.0
• --dynamic (backend fell back to C: llvm refused: npmEmbedding)