feat(gateway): typed DTO contract for HTTP responses and OpenAPI schema

README.md

@@ -98,7 +98,7 @@ Beyond faults, medkit exposes the full ROS 2 graph through REST:
 | **Software Updates** | Async prepare/execute lifecycle with pluggable backends |
 | **Authentication** | JWT-based RBAC (viewer, operator, configurator, admin) |
 | **Logs** | Log entries and configuration |
-| **Docs** | OpenAPI 3.1.0 spec and Swagger UI at `/api/v1/docs` |
+| **Docs** | OpenAPI 3.1.0 spec and Swagger UI at `/api/v1/docs` - schemas are generated from typed C++ structs so the spec always matches the wire format |
 
 On the [roadmap](https://selfpatch.github.io/ros2_medkit/roadmap.html): entity lifecycle control, mode management, communication logs.
 

docs/tutorials/openapi.rst

@@ -77,6 +77,37 @@ When disabled, all ``/docs`` endpoints return HTTP 501.
 
 See :doc:`/config/server` for the full parameter reference.
 
+How Schemas Are Generated
+--------------------------
+
+The ``components/schemas`` object in every ``/docs`` response is generated
+automatically from the DTO registry. Most response and request types are
+declared as a plain C++ struct with a ``constexpr dto_fields<T>`` descriptor
+tuple; the ``SchemaWriter<T>`` visitor folds over this tuple at compile time to
+produce the OpenAPI JSON Schema entry. The ``AllDtos`` registry in
+``dto/registry.hpp`` lists every named type so that
+``collect_component_schemas()`` populates the entire ``components/schemas`` map
+with no hand-written schema factories.
+
+For a field-walking DTO the same descriptor also drives serialization
+(``JsonWriter<T>``) and request-body validation (``JsonReader<T>``), so the wire
+shape and the published schema are always derived from the same source.
+
+A few envelope types whose payload shape is decided at runtime by a plugin or by
+a live ROS 2 type - the ``Data*Result`` / ``Fault*Result`` /
+``OperationExecutionResult`` *opaque* DTOs - have no ``dto_fields``. They carry a
+hand-written ``JsonWriter`` / ``JsonReader`` / ``SchemaWriter`` trio instead, are
+still listed in ``AllDtos``, and publish an opaque object schema
+(``{type: object, additionalProperties: true, x-medkit-opaque: true}``). Inside
+an ordinary DTO, a free-form member typed as a bare ``nlohmann::json`` (for
+example the fault environment records) appears as an unconstrained object
+(``{}``). The per-topic / per-service / per-action routes for live ROS 2 data do
+not use ``components/schemas`` at all: their request and response bodies carry an
+inline schema derived from the ROS 2 type on the route itself.
+
+For the full design of the DTO contract layer, see
+:doc:`/design/ros2_medkit_gateway/dto_contract`.
+
 See Also
 --------
 

docs/tutorials/plugin-system.rst

@@ -98,7 +98,7 @@ Writing a Plugin
      tl::expected<std::vector<std::string>, UpdateBackendErrorInfo>
        list_updates(const UpdateFilter& filter) override { /* ... */ }
 
-     tl::expected<nlohmann::json, UpdateBackendErrorInfo>
+     tl::expected<dto::UpdateDetail, UpdateBackendErrorInfo>
        get_update(const std::string& id) override { /* ... */ }
 
      tl::expected<void, UpdateBackendErrorInfo>
@@ -201,7 +201,7 @@ A self-contained plugin implementing UpdateProvider (copy-paste starting point):
        return std::vector<std::string>{};
      }
 
-     tl::expected<nlohmann::json, UpdateBackendErrorInfo>
+     tl::expected<dto::UpdateDetail, UpdateBackendErrorInfo>
      get_update(const std::string& id) override {
        return tl::make_unexpected(
          UpdateBackendErrorInfo{UpdateBackendError::NotFound, "not found: " + id});

src/ros2_medkit_gateway/CHANGELOG.rst

@@ -7,12 +7,94 @@ Unreleased
 
 **Breaking Changes:**
 
+* Typed router refactor. ``HandlerContext`` no longer carries
+  ``send_json`` / ``send_error`` / ``send_plugin_error`` / ``send_dto`` /
+  ``parse_body``: handlers return ``http::Result<TResponse>`` and the
+  framework owns response writing through ``RouteRegistry``. The raw
+  ``void(httplib::Request, httplib::Response)`` ``RouteRegistry`` lambda
+  overloads are removed - call sites must use the typed
+  ``reg.get<T>`` / ``reg.post<TBody, T>`` / ``reg.del<T>`` overloads, the
+  multi-shape ``reg.post_alternates<TBody, TAlt...>`` /
+  ``reg.del_alternates<TAlt...>``, or one of the named escape hatches
+  (``reg.sse`` / ``reg.binary_download`` / ``reg.multipart_upload<T>`` /
+  ``reg.static_asset`` / ``reg.docs_endpoint`` / ``reg.docs_subtree``).
+  ``static_assert(dto::has_dto_shape_v<T>)`` gates every typed overload, so
+  non-DTO return types fail at compile time. The plugin ABI is unaffected:
+  ``PluginResponse`` keeps its ``send_json`` / ``send_error`` surface and
+  now routes through the same internal ``http::detail::write_json_body``
+  primitive as the framework, so plugin wire format is unchanged
+  (`#403 <https://github.com/selfpatch/ros2_medkit/issues/403>`_)
+* Provider ABI typed. ``FaultProvider``, ``DataProvider``,
+  ``OperationProvider``, and ``UpdateProvider::get_update`` return typed
+  DTO envelopes (``FaultListResult`` / ``FaultDetailResult`` /
+  ``FaultClearResult`` / the matching ``Data*Result`` and
+  ``Operation*Result`` shapes / ``UpdateStatusResult``) instead of raw
+  ``tl::expected<nlohmann::json, ErrorInfo>``. The wire bytes are
+  byte-identical because each envelope wraps an opaque ``content`` object
+  emitted verbatim by ``JsonWriter``; commercial and out-of-tree plugins
+  must wrap their existing JSON in the matching envelope type
+  (mechanical: ``Result.content = std::move(json_payload)``). The plugin
+  ABI itself (``PluginRoute`` shape, ``PluginResponse`` ctor, plugin api
+  version) is locked by ``test_plugin_abi_conformance`` and is unchanged
+  (`#403 <https://github.com/selfpatch/ros2_medkit/issues/403>`_)
+* ``SchemaWriter`` emits optional DTO fields as
+  ``anyOf: [<inner>, {type: "null"}]`` (the OpenAPI 3.1 idiom) instead of
+  ``nullable: true``. Generated clients see ``T | null`` for every optional
+  field rather than ``T | undefined``. Wire format is unchanged - the
+  gateway still omits absent optional fields, and ``JsonReader`` continues
+  to accept absent fields; the schema change only opts the published spec
+  into round-tripping a literal ``null`` value cleanly for clients that
+  prefer to send one. As part of this, a handful of fields that were
+  previously emitted as an explicit JSON ``null`` when absent are now omitted
+  entirely (consistent with the optional-omission policy): the script
+  execution fields ``progress`` / ``started_at`` / ``completed_at`` /
+  ``parameters`` / ``error`` (``GET .../scripts/{id}/executions/{eid}``) and
+  the script ``parameters_schema`` field (``GET .../scripts/{id}``). Clients
+  that tested ``field === null`` or relied on the key always being present must
+  treat an absent key the same as ``null``. The ``rtmaps_medkit`` variant is
+  explicitly NOT covered by this PR - its handlers continue to run on the
+  pre-typed HandlerContext surface and will be migrated separately
+  (`#403 <https://github.com/selfpatch/ros2_medkit/issues/403>`_)
+* Synchronous operation-execution service-call failures
+  (``POST /api/v1/{entity-path}/operations/{id}/executions`` when the underlying
+  ROS 2 service call fails) now return the standard SOVD ``GenericError`` envelope
+  (``{"error_code": "vendor-error", "vendor_code":
+  "x-medkit-ros2-service-unavailable", "message": "Service call failed", ...}``,
+  HTTP status 500 unchanged) instead of the previous bespoke nested
+  ``{"error": {"code", "message", "details"}}`` object. This aligns the one
+  remaining non-standard error path with every other gateway error; clients that
+  parsed ``error.code`` / ``error.details`` for this specific failure must read
+  ``vendor_code`` / ``parameters`` instead
+  (`#403 <https://github.com/selfpatch/ros2_medkit/issues/403>`_)
+* ``GET /api/v1/{entity-path}/data`` now publishes the opaque ``DataListResult``
+  schema (``{type: object, additionalProperties: true, x-medkit-opaque: true}``),
+  matching how ``GET .../faults`` already publishes ``FaultListResult``. The wire
+  payload is unchanged for runtime (ROS 2) entities - it is still
+  ``{"items": [...], "x-medkit": {...}}`` built from the typed
+  ``Collection<DataItem, DataListXMedkit>`` - but for plugin-owned entities the
+  provider's free-form per-item shape now passes through verbatim instead of
+  being re-parsed into ``Collection<DataItem>``. This fixes a regression in which
+  plugin per-item fields (the OPC-UA plugin's ``value`` / ``unit`` / ``data_type``
+  / ``writable``) were silently dropped by the typed re-parse. Clients that
+  generated a typed ``DataItem`` model from the previous spec for this route now
+  see an opaque object instead
+  (`#403 <https://github.com/selfpatch/ros2_medkit/issues/403>`_)
+* Entity responses (areas, components, apps, functions - list items and detail)
+  now always carry a top-level ``type`` discriminator (an enum of
+  ``area`` / ``component`` / ``app`` / ``function``). Previously list items had no
+  ``type`` and detail responses exposed it only inside ``x-medkit.entityType``.
+  Additive and tolerant-client-safe; consumers keying on the entity kind can now
+  read the top-level ``type``
+  (`#403 <https://github.com/selfpatch/ros2_medkit/issues/403>`_)
 * ``ros2_medkit_msgs/srv/ClearFault`` request gains a ``bool skip_correlation_auto_clear`` field (see the per-entity fault scope entry below for the in-tree motivation). Adding a request field changes the service type hash, so out-of-tree callers that invoke the service directly (for example ``ros2 service call /fault_manager/clear_fault ros2_medkit_msgs/srv/ClearFault ...`` as documented in the ``ros2_medkit_fault_manager`` README) must rebuild against the new ``ros2_medkit_msgs`` to keep talking to ``fault_manager``. The in-tree gateway client and server are updated together (`#395 <https://github.com/selfpatch/ros2_medkit/issues/395>`_)
 * Per-entity fault routes are now correctly scoped to the entity's hosted apps. ``GET /api/v1/{entity-path}/faults/{fault_code}``, ``DELETE /api/v1/{entity-path}/faults/{fault_code}``, ``GET /api/v1/{entity-path}/faults``, and ``DELETE /api/v1/{entity-path}/faults`` previously fell back to a prefix match against the entity's ``namespace_path``; when that was empty (host-derived / synthetic components, manifest components without a ``namespace`` field, Areas, Functions, and Apps with a wildcard ``ros_binding.namespace_pattern``) the scope filter was silently disabled and the routes exposed - and on ``DELETE``, cleared - faults reported by apps that belonged to entirely different entities. All four handlers now resolve the addressed entity to its hosted-app FQN set (via the new ``HandlerContext::resolve_entity_source_fqns`` helper) and apply a strict all-sources scope check: a fault counts as in scope only when **every** entry in its ``reporting_sources`` is owned by the entity (exact FQN match, or strict path-child via ``<fqn>/...``). Per-fault routes return ``404 Resource Not Found`` for any fault that fails the check; collection routes return an empty ``items`` array. The underlying ``GetFault.srv`` contract is unchanged; ``ClearFault.srv`` gains a new ``skip_correlation_auto_clear`` request flag so per-entity DELETE can opt out of cascade-clearing correlated symptom fault codes that may live in other entities. Per-entity collection responses no longer include the global ``muted_count`` / ``cluster_count`` / ``muted_faults`` / ``clusters`` correlation metadata; those remain on the global ``GET /api/v1/faults`` route. Behavior changes visible to clients: (a) faults reported by apps outside the addressed entity are no longer returned or cleared via that entity's route, (b) **mixed-source** faults that include at least one out-of-entity reporter are likewise rejected with ``404`` on per-fault routes and excluded from per-entity collection responses (use the global ``GET /api/v1/faults`` to see them), (c) per-entity DELETE no longer cascade-clears correlated symptoms outside the entity (`#395 <https://github.com/selfpatch/ros2_medkit/issues/395>`_)
 * ``GET /api/v1/updates/{id}/status`` no longer returns ``404`` for a registered-but-idle package; ``POST /api/v1/updates`` now seeds a ``pending`` status, so the endpoint returns ``200 {"status": "pending"}`` immediately after registration. ``404`` is reserved for packages that are not registered. Clients that used ``404`` as a signal for "registered but nothing started yet" must adapt (`#378 <https://github.com/selfpatch/ros2_medkit/issues/378>`_)
 
 **Features:**
 
+* Typed ``fan_out_collection<T>`` aggregating helper replaces raw-JSON ``merge_peer_items`` on the typed collection routes (data, operations, config, logs). Peer items are decoded via ``dto::JsonReader<T>``; items that fail validation are removed from the merged ``items`` array, recorded in ``x-medkit.peer_dropped_items`` with the JsonReader error plus a best-effort ``source_id``, and logged at ``WARN``. Items that parse successfully are re-serialized through the local ``dto::JsonWriter<T>``, so any peer-supplied fields outside the local DTO schema are dropped from the merged response (the previous raw passthrough preserved unknown peer fields verbatim). Previously, malformed peer items silently disappeared into the merged response; fleet operators can now detect inter-gateway schema drift directly on the wire (`#403 <https://github.com/selfpatch/ros2_medkit/issues/403>`_)
+* ``Collection<T, XMedkitT>`` is now a 2-parameter template. Domain list endpoints (faults, config, logs) reference their richer per-domain collection x-medkit struct (``FaultListXMedkit``, ``ConfigListXMedkit``, ``LogListXMedkit``) directly in the published schema instead of the generic ``XMedkitCollection``, so generated clients see aggregation counts, peer provenance, and ``peer_dropped_items`` from the schema. The data list builds the same typed ``Collection<DataItem, DataListXMedkit>`` internally (so the wire still carries those fields) but publishes the opaque ``DataListResult`` envelope, because plugin-owned data entities can return vendor per-item fields the typed item schema cannot describe (see the data-list breaking-change entry above) (`#403 <https://github.com/selfpatch/ros2_medkit/issues/403>`_)
+* New ``opaque_object("key", &T::field)`` DTO field descriptor in ``dto/contract.hpp``. Binds a ``nlohmann::json`` member as a typed "any JSON object" field: ``JsonWriter`` emits it verbatim, ``JsonReader`` rejects scalars / arrays / null, ``SchemaWriter`` emits ``{type: object, additionalProperties: true, x-medkit-opaque: true}``. Used for fields whose runtime shape is decided by an upstream component the gateway cannot introspect (live ROS message payloads, plugin-defined fault envelopes, action results) (`#403 <https://github.com/selfpatch/ros2_medkit/issues/403>`_)
 * ``GET /api/v1/faults/stream`` event payloads now carry an optional ``x-medkit`` SOVD payload-extension object with ``entity_type`` and ``entity_id`` fields. When the gateway can resolve the fault's first reporting source back to a SOVD entity (via the manifest-mode linking index, or a runtime-mode last-segment match against an existing App), consumers can hit ``/{entity_type}/{entity_id}/bulk-data/rosbags/{fault_code}`` directly instead of HEAD-probing every entity. Resolution is snapshotted at event arrival, so a discovery refresh between enqueue and stream-out cannot retroactively change the entity reported to consumers. The ``x-medkit`` object is omitted entirely when no entity can be resolved, so existing SSE consumers ignore the addition (`#380 <https://github.com/selfpatch/ros2_medkit/issues/380>`_)
 * Plugin API version bumped to v7. Adds ``PluginContext::notify_entities_changed(EntityChangeScope)`` lifecycle hook for plugins that mutate the entity surface at runtime; default no-op keeps v6 source code compiling unchanged against v7 headers. Binary compatibility is not provided: the plugin loader uses a strict equality check on ``plugin_api_version()``, so out-of-tree plugins must be recompiled (`#376 <https://github.com/selfpatch/ros2_medkit/issues/376>`_)
 * New ``discovery.manifest.fragments_dir`` parameter: gateway scans the directory for ``*.yaml`` / ``*.yml`` fragment files on every manifest load / reload and merges apps, components, and functions on top of the base manifest. Fragments are forbidden from declaring top-level ``areas``, ``metadata``, ``discovery``, ``scripts``, ``capabilities``, or ``lock_overrides`` - those stay in the base manifest. Presence of any forbidden key (including empty-valued ones like ``areas: []``) is reported as a ``FRAGMENT_FORBIDDEN_FIELD`` validation error that fails the load / reload. Unknown top-level keys (typos such as ``app:`` vs ``apps:``) are ignored with a warning log. Files merged in alphabetical order for deterministic duplicate-id errors (`#376 <https://github.com/selfpatch/ros2_medkit/issues/376>`_)