fix(cli): support JSON-LD ingestion - #2506
branarakic wants to merge 8 commits into
Conversation
60eab8e to
8d0617b
Compare
|
Rebased the existing JSON-LD ingestion fix onto current New head: Validation on the rebased tree:
The dependency and lockfile follow-up is limited to the CLI's JSON-LD and shared RDF utility dependencies. |
otReviewAgent
left a comment
There was a problem hiding this comment.
Operational Notice: Review Agent could not complete this review.
Business logic reviewer failed: You've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Sep 20th, 2026 9:00 AM.
8d0617b to
50ecdb0
Compare
otReviewAgent
left a comment
There was a problem hiding this comment.
Operational Notice: Review Agent could not complete this review.
Business logic reviewer failed: You've hit your usage limit. Visit https://chatgpt.com/codex/settings/usage to purchase more credits or try again at Sep 20th, 2026 9:00 AM.
50ecdb0 to
81c4e8f
Compare
| } | ||
|
|
||
| describe('JSON-LD CLI writes through the real daemon', () => { | ||
| it.each(['list', 'nested'] as const)('writes and finalizes linked %s RDF without losing blank-node edges', async (kind) => { |
There was a problem hiding this comment.
🟡 Issue: Blank-node collisions across separate JSON-LD writes are never exercised by a test
What's wrong
The change newly allows JSON-LD anonymous nodes (_:bN) through validation and storage, and jsonld.js assigns those labels per document, not per write. Because empty-graph quads all land in one per-KA WM graph, two separate ka write -f invocations on the same KA with different JSON-LD files silently merge their anonymous nodes. The only end-to-end test writes a single document per KA, so a conflated result (wrong graph structure at finalize/skolemization) still passes green and the suite gives the impression that JSON-LD blank-node writes are fully validated.
Example
File A: {"@id":"urn:docA","https://example.org/child":{"https://example.org/name":"A"}} → <urn:docA> <.../child> _:b0 and _:b0 <.../name> "A". File B: {"@id":"urn:docB","https://example.org/child":{"https://example.org/name":"B"}} → also _:b0. After two separate ka write calls both nodes are _:b0 in the single per-KA graph, so urn:docA and urn:docB point at one node carrying both "A" and "B". Expected: the two anonymous nodes stay distinct. The existing test only ever runs ka write once per KA, so this conflation is not detected.
Suggested direction
Add a live-daemon regression that creates one KA, runs two separate ka write -f (or ka create) invocations with different JSON-LD files that each contain an anonymous nested node, then reads /wm/quads and (after finalize) asserts the two documents' anonymous nodes are distinct — e.g. each document's child carries only its own literal. If the intended design is to disambiguate on ingest, add the relabeling in rdf-parser.ts/cli-helpers.ts and keep the current single-write test.
For Agents
Look at packages/cli/test/jsonld-knowledge-asset-lifecycle.test.ts and extend it: after the single-write cases, add a case that writes two different JSON-LD documents (each with an anonymous nested node) to the same KA via two separate CLI ka write -f invocations, then assert wm/quads contains two distinct blank-node subjects/objects and that finalization preserves both edges. If the fix is to relabel blank nodes at ingest, implement it in rdf-parser.ts/cli-helpers.ts (where quads are produced) and keep the existing single-write assertions unchanged.
| } | ||
|
|
||
| /** Array-only compatibility facade for callers that do not need source provenance. */ | ||
| export async function parseRdf( |
There was a problem hiding this comment.
💡 Suggestion: parseRdf is now a pass-through facade whose only callers are tests.
Why it matters
The PR renamed the real parser to parseRdfInput and reintroduced parseRdf as an array-only wrapper, but every production call site goes through loadRdfFromInput -> parseRdfInput; only test files import parseRdf. An exported entry point kept alive solely by tests makes the module look like it has two APIs and invites the two signatures to drift.
Suggestion
Remove the test-only parseRdf wrapper so the module exposes a single parse entry point.
The CLI advertises
.jsonldinput but rejects documents that use@context. Convert JSON-LD through jsonld.js and the shared private N3 quad parser, preserving named graphs, blank nodes, lists, language tags and datatypes. JSON and legacy JSON-LD quad arrays share one typed decoder. The neutral parser returns source provenance alongside quads; its existing array facade remains available.File input supplies its file URL as the document base. Safe conversion rejects expansion that would discard statements. Contexts must be inline: an offline loader rejects external
@contextand@importreferences without making requests. The CLI README documents the input and context policy.Use the canonical RDF literal formatter so quotes, backslashes and control characters survive ingestion without producing invalid RDF.
Default-finalizing
ka createrejects JSON-LD named-graph documents before connecting to the daemon, with actionable--no-finalizeguidance. WM-only imports preserve their named graphs; existing quad-array inputs retain their prior contract. The Knowledge Asset command owns this policy and uses parser provenance to distinguish new JSON-LD document conversion from legacy arrays; no lifecycle flags pass through shared parsing helpers.Working-memory writes accept validated RDF blank-node terms so JSON-LD lists and nested objects survive ingestion and the existing canonical finalization step. Malformed or injectable blank-node labels fail at the HTTP boundary. Inline
--triplesuses the same legacy quad-array decoder as JSON files.Validation: all 60 parser and actual CLI smoke tests pass on this revision, covering syntax provenance, named/default graphs, lists/nested objects, literal escaping, file bases, legacy arrays and offline contexts. CLI/dependency builds, public types/package-boundary checks, repository lint, the 1,801-file test inventory, SPARQL and merge-tree checks pass. Earlier revisions additionally exercised blank-node writes and canonical finalization through a local daemon backed by the test chain.
The parser refactor keeps the array compatibility adapter one-way: parseRdf calls parseRdfInput, whose RDF and converted JSON-LD branches both call private parseN3Quads directly. No branch calls back through the compatibility adapter.
Closes #15.