Skip to content

Generate ABI 3 codecs for external model mappings - #48

Merged
danwalmsley merged 4 commits into
mainfrom
codex/external-model-abi3-codecs
Sep 9, 2026
Merged

danwalmsley merged 4 commits into
mainfrom
codex/external-model-abi3-codecs

Conversation

@danwalmsley

@danwalmsley danwalmsley commented Sep 9, 2026 •

Copy link
Copy Markdown
Collaborator

External model type mappings currently generate signatures that compile but fall back to JSON invocation, which NativeJavaScriptInvoker rejects. This change generates ABI 3 readers and writers in the integration assembly for compatible external classes, structs, and records, so the contracts assembly needs no WebScene or backend dependency.

  • Support direct objects, arrays, nullable values, nested external contracts, and object aliases without intermediate DTOs or projected arrays on writes. Struct writes use typed access without per-value boxing, including nullable structs and struct array elements.
  • Support external long and long? properties for TypeScript number and nullable number. Both directions enforce the JavaScript safe-integer interval ±(2^53 − 1). Reads reject fractional/non-finite values, and writes reject integers outside the safe interval; invalid conversions throw OverflowException instead of rounding or truncating.
  • Support explicit per-property optionalProperties policies for neutral CLR contracts: always emit on writes, and reject absent/undefined reads or substitute null/a configured scalar default. Explicit null follows the declared value type. Unconfigured fields retain JavaScriptOptional<T> semantics.
  • Read adapter-valued instance/global properties through ABI 3 retained handles, including optional/nullable and promise-valued properties. These getters now return JavaScriptObjectReference rather than an abstract adapter; callers release each returned handle. Callback registration is unchanged.
  • Validate CLR property types and construction using Roslyn symbols. Match properties by PascalCase or explicit model propertyMappings, and read through constructors or writable/init properties. Struct readers prefer a matching constructor.
  • Report WEBSCENEJS004 with an actionable reason for unsupported contracts, including recursive graphs, ref structs, generics, index signatures, and incompatible properties or constructors. Optional JavaScriptOptional<long> property conversion remains unsupported.
  • Document the supported shapes, integer precision rules, and extend the policy schema for property renames.

Validation: all 104 interop tests pass, including separate-assembly contracts through NativeJavaScriptInvoker with an in-memory ABI 3 transport, the ChartHost array/void pattern, the requested TradingViewBar record struct with a renamed long timestamp, nullable structs, numeric boundaries and invalid reads/writes, byte-for-byte equivalence with generated DTO codecs, pooled-write allocation checks, optional scalar/object read semantics (absent, undefined, null, and value), preservation of wrapper behavior, and struct arrays with optional volume members. An isolated probe with all 13 actual chart contracts and the unchanged API manifest compiles with zero warnings/errors after adding explicit optional-property policies; all three bridge getters use ABI 3. The policy schema parses successfully and git diff --check passes.

Fixes #47.

@danwalmsley

Copy link
Copy Markdown
Collaborator Author

Thanks—this addresses the missing ABI 3 codec generation reported in #47 and keeps the codecs in the integration assembly, which is what we need for the MVVM split.

Reviewing this against SandwichTradingDevOps/StackWich#1060, two additional capabilities would let us consume our existing neutral chart contracts directly:

  1. Numeric conversion between TypeScript number and CLR long properties. Our spreader points and candle records use long Time, and broker records also use integer timestamps. The current exact-type validation rejects these because number maps to double. I added a local diagnostic test using record Sample(long Timestamp, double? Price) and confirmed it produces WEBSCENEJS004 (expected 'double'). Could the generated codecs support this conversion with explicit read-side range/fraction handling and documented JavaScript number precision limits? This would avoid changing application timestamp contracts or retaining per-point DTO conversions.

  2. External record structs / value-type contracts. Our main candle contract is public readonly record struct TradingViewBar(long TimeMilliseconds, double Open, double High, double Low, double Close, double Volume). Property renaming can handle time → TimeMilliseconds, but the class-only validation currently rejects the contract. Supporting constructor-based record structs, including arrays and nullable values, would let us pass these bars directly without introducing reference-type DTO allocations. Allocation coverage should also verify that generated writes do not box each value.

I ran the interop suite at fb7958ca4a094c20d0cf7d296370eec578d54157: all 65 tests passed, including my additional local long rejection case. The existing separate-assembly, native-invoker, wire-equivalence, and pooled-write coverage is useful; extending it with these two production contract shapes would close the remaining gaps for our chart integration.

@danwalmsley

Copy link
Copy Markdown
Collaborator Author

The update at adab0e5528dbf7cef5ede2a3aa7fe365c2bf730a addresses both requested capabilities: checked long timestamp conversions and external record structs. I ran the updated interop suite locally: all 73 tests pass.

I also compiled an integration probe using the actual contracts assembly and API/policy manifests from SandwichTradingDevOps/StackWich#1060, mapping all 13 chart models to their existing neutral CLR contracts. This exposed one remaining compatibility gap: optional TypeScript properties currently require JavaScriptOptional<T> in the external contract.

Examples from the real manifests:

  • spreadBidPriceInBps?: number | null maps to our double? SpreadBidPriceInBps, but the generator requires JavaScriptOptional<double?>.
  • volume?: number maps to TradingViewBar.Volume (double), but requires JavaScriptOptional<double>.
  • Optional nullable account, message, price and quantity properties have the same mismatch.

With WEBSCENEJS004 promoted to an error, the unchanged manifests correctly fail compilation. In a second, isolated probe, changing the optional model properties to required properties and removing their undefined union member—while retaining null where declared—allowed all 13 mapped models to compile with zero warnings/errors. This was a diagnostic experiment, not a proposed silent change to the wire contract.

Could external model policy explicitly support ordinary CLR properties for these optional members, without requiring the contracts assembly to reference WebScene? The policy should define the semantics rather than implicitly discard the absent/null distinction: for example, always emit the property on writes, and explicitly choose whether an absent property on reads is rejected or mapped to null/a configured default. Existing JavaScriptOptional<T> behavior should remain available when consumers need to preserve absence.

Regression coverage using these production shapes would be useful: optional nullable scalar, optional non-nullable scalar, optional nullable external object, and a record-struct array containing such a member. Please cover absent versus explicit null/value on reads, the selected write semantics, and retention of the direct-write/no-boxing allocation behavior.

Alternatively, we can tighten our application-owned TypeScript declarations if we verify these fields are always present, but that is an application contract change rather than a drop-in external mapping. We want to keep WebScene-specific optional wrappers out of the neutral contracts assembly.

@danwalmsley
danwalmsley merged commit b5cc761 into main Sep 9, 2026
4 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Generate ABI 3 codecs for external model typeMappings (or diagnose unsupported mappings)

1 participant