[miyoungc]

Description

The API reference currently depends on a handwritten OpenAPI file that can drift from the FastAPI server. Generate the contract during documentation builds, alongside the Python SDK reference, so a source change updates the built API reference automatically. Contributors commit the source and curated documentation; fern/openapi.yml becomes an ignored build artifact.

• Add an offline, deterministic FastAPI exporter that does not start the application lifespan or call providers. Preserve descriptions, examples, response documentation, and endpoint identities in fern/openapi-overlay.yml, and add /v1/checks to navigation.
• Generate OpenAPI before SDK generation, validation, local serving, previews, and staging/public publishing. Generation failure blocks publishing, including parallel Make runs. Preview watching regenerates references when source or dependencies change and ignores generated-file writes to avoid rebuild loops.
• Add the OpenAPI generation CI workflow to exercise generation and exporter tests. Keep both existing API Conformance workflows: Implementation remains unchanged; Docs now checks the generated, overlay-applied contract and compares it with the target revision for breaking changes. Upstream OpenAI compatibility gaps remain informational.
• Generate both comparison inputs from their own source revisions using native Fern export. Include overlay-only changes in the workflow triggers, fail on missing expected inputs, and use GitHub's checked-out test merge's first parent as the actual target baseline. The initial migration exception applies only when the target lacks the exporter and runtime/dependency sources are unchanged.
• Generate historical OpenAPI from each snapshot's own source, while retaining the handwritten spec for snapshots that predate the exporter. Give historical SDK generation its own Fern working directory and version-redirection context so cold-cache builds write pages into the correct checkout.
• Correct three existing action hash/version-comment mismatches in the test workflows.
• Keep all eight conformance-baseline regression cases running on Windows by using Git Bash explicitly, writing LF-only command stubs, and passing a shell-compatible temporary path. Include captured subprocess output in assertion failures.
• Remove seven optional image titles that duplicate their unchanged alt text. Fern CLI 5.167.0 treated those titles as part of the image filename and blocked preview publishing; the image paths and descriptions are preserved. Apply the same cleanup to temporary historical checkouts before SDK cache restoration or generation, preserving the immutable snapshot commits and tags.

Related Issue(s)

Tracking: SDR-1591. The request originated in Slack and is tracked on the SDR documentation board, as authorized by the directing contributor.

Verification

• Remote CI is clean on 22f56ec027fde7b9503ef17340d9097ed2f683e8, with no pending or failed PR checks. Successful checks include Python 3.10–3.13 on Ubuntu, coverage, OpenAPI generation, both API conformance jobs, Fern validation and preview publishing, lint, zizmor, CodeQL, and Greptile. The separately dispatched Full Tests matrix also passes all eight Windows/macOS combinations across Python 3.10–3.13 on this exact commit. Staging publishing was intentionally skipped.
• Focused locked Python 3.13 tests: 18 passed. Covers deterministic offline export, ignored/untracked output, generated comparison inputs, missing-baseline failure, HTTP examples, and migration guards, including an upstream change after the PR event.
• npm run test:docs-scripts with Node.js 22: 18 passed. Covers missing/stale artifacts, failure blocking under make -j2, historical SDK caching and inherited Fern context, source-change watching, and native Fern compilation/export.
• Pre-commit on changed files: passed, including YAML, Ruff, formatting, license checks, zizmor, and ty.
• make docs-fern with Node.js 22 and the organization-required Fern CLI 5.167.0: passed, regenerating OpenAPI and 406 SDK pages with 0 errors and 566 existing warnings. The generated artifacts remain ignored and untracked.
• The Windows matrix reproduced the shell-harness failure before its fix. A real cold historical build of v0.24.1 with Fern 5.167.0 generated pages in the snapshot checkout, cached 361 normalized SDK pages, and passed native Fern validation without publishing. Cache-hit/cache-miss tests verify image normalization, preservation of distinct titles, and unchanged snapshot commit contents. All four published snapshot refs contain the seven affected duplicate titles covered by this cleanup.
• The effective-contract comparison found no breaking changes. Temporarily removing overlay-defined RailsConfigInfo.id failed the pinned oasdiff check with response-required-property-removed, confirming overlay response changes are covered.
• Greptile's latest review confirms the earlier comparison gap is fixed and reports no new actionable issues. Structured local source/security review also completed; the separate Codex CLI review was blocked by automatic approval review and was not run.

The PR preview was published successfully. No staging or production publication was performed. The earlier Windows failures were reproduced by the full platform matrix, and the Fern 5.167.0 image-path failure was reproduced locally before the fix (7 errors) and cleared afterward (0 errors).

AI Assistance

• No AI tools were used.
• AI tools were used; a human reviewed and can explain every change (tool: OpenAI Codex).

OpenAI Codex implemented the exporter, overlay, build/workflow wiring, documentation, and regression tests, and ran local validation. Human review is pending.

Checklist

• I've read the CONTRIBUTING guidelines.
• This PR links to the Jira tracking ticket approved by the directing contributor.
• My PR title follows the project commit convention.
• I've updated the documentation if applicable.
• I've added tests if applicable.
• I've noted verification beyond CI and checks that could not run.
• I did not update generated changelog files manually.
• All automated and human review comments are addressed or have a reply explaining why no change is needed.
• @mentions of the person or team responsible for reviewing proposed changes.

Summary by CodeRabbit

• New Features

  • Added documentation for the Guardrails API’s POST /v1/checks endpoint.
  • Expanded API documentation to cover health, challenges, models, rail configurations, chat completions, content checks, streaming events, validation errors, and guardrail-specific responses.
  • Added support for richer request and response schemas, including audio, annotations, tool calls, moderation, and token usage.

• Documentation

  • API references are now generated and validated for freshness to keep published documentation aligned with the server contract.
  • Documentation publishing now verifies reference generation before release.

[github-actions]

Staged Fern docs preview: https://nvidia-preview-pr-2475.docs.buildwithfern.com/nemo/guardrails

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Repository: NVIDIA-NeMo/Guardrails/.coderabbit.yaml
• Review profile: CHILL
• Plan: Enterprise
• Run ID: c4706e48-6bde-4cb0-8b4f-1641e4ba4e3b

📥 Commits

Reviewing files that changed from the base of the PR and between 3a6e84f and 659412d.

📒 Files selected for processing: 17

• .github/workflows/_test.yml
• .github/workflows/api-conformance-docs.yml
• .github/workflows/docs-build.yaml
• .github/workflows/openapi.yml
• .github/workflows/test-docker.yml
• Makefile
• docs/README.mdx
• docs/index.yml
• fern/generators.yml
• fern/openapi-overlay.yml
• fern/openapi.yml
• scripts/export_openapi.py
• scripts/openai_coverage.py
• scripts/tests/docs-publish.test.mjs
• scripts/tests/fixtures/openapi-reference.json
• scripts/tests/openapi-reference.test.mjs
• tests/server/test_export_openapi.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

📝 Walkthrough

The server OpenAPI document is now generated and used as the Fern base specification. Make targets and documentation workflows regenerate API references before publishing. CI checks specification freshness and compatibility, with tests covering export behavior, reference output, and publishing.

Changes: API reference generation

Layer / File(s)	Summary
Generate and document the server contract scripts/export_openapi.py, fern/openapi.yml, fern/openapi-overlay.yml, fern/generators.yml, tests/server/test_export_openapi.py	The exporter writes or checks the generated OpenAPI document. The Fern overlay and generator settings define the documented API reference. Tests check the generated contract, freshness, deterministic output, and /v1/checks behavior.
Generate and publish Fern references Makefile, .github/workflows/docs-build.yaml, docs/README.mdx, docs/index.yml, scripts/tests/*	Make targets and documentation jobs generate OpenAPI and SDK references. Publishing stops if generation fails. Documentation and tests cover the commands, endpoint navigation, generated reference output, and publishing behavior.
Validate freshness and compatibility .github/workflows/api-conformance-docs.yml, .github/workflows/openapi.yml, scripts/openai_coverage.py, tests/server/test_export_openapi.py	Workflows check generated OpenAPI freshness and run compatibility checks. Baseline initialization depends on whether the base revision has an exporter and whether runtime sources changed.

Workflow action annotations

Layer / File(s)	Summary
Update action version comments .github/workflows/_test.yml, .github/workflows/test-docker.yml	Action version comments are updated. Pinned action references and workflow behavior remain unchanged.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Other

Suggested reviewers: m-misiura, pouyanpi

Merge Risk: ⚪ Minimal · up to 65941

This change generates the API specification from the server and regenerates the API and SDK references before documentation is previewed or published. Publication stops if generation fails, and CI checks that the committed specification is current. No concrete defects were identified, so the change appears ready to merge after normal CI passes. The only minor caveat is that output equality was checked only on Python 3.12 and 3.13, although Python 3.10 and 3.11 are also listed as supported for local generation.

Pre-merge checks: 5 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 13 functions across 5 files. (12 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 5 passed

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Test Results For Major Changes		The PR contains major CI, OpenAPI, and documentation-generation changes, and the description documents testing results. It reports 11 exporter tests passed, 15 documentation-script tests passed, pre-c…
Title check		The title clearly and concisely describes the main change: generating the OpenAPI specification during documentation CI workflows.

Full details: Test Results For Major Changes

Explanation

The PR contains major CI, OpenAPI, and documentation-generation changes, and the description documents testing results. It reports 11 exporter tests passed, 15 documentation-script tests passed, pre-commit checks passed, Fern generation completed with 0 errors, and generated YAML matched under Python 3.12 and 3.13. The changes do not affect application numerics, convergence, or runtime performance.

Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 13 functions across 5 files. (12 skipped: 12 unsupported.)

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

• Fix all pre-merge checks with AI

✨ Finishing Touches: 💡 1

📝 Generate docstrings: 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

_{Comment @coderabbitai help to get the list of available commands.}

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge; the previous comparison gap is fixed and no new actionable issues were found.

Summary

The PR generates OpenAPI from FastAPI during documentation builds and keeps curated API details in a Fern overlay.

• The API reference now follows the FastAPI source and Fern overlay.
• Docs builds and publishing now generate API references from their source.
• API conformance checks now compare generated contracts from both revisions.
• Historical SDK pages now build with each snapshot's own API reference context.
• Workflow comments now show the exact pinned action versions.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[FastAPI source] --> B[Generate OpenAPI]
  B --> C[Apply Fern overlay]
  C --> D[Validate or publish docs]
  B --> E[Generate SDK reference]
  E --> D

Loading

_{Reviews (7) · Last reviewed commit: "fix(docs): read historical pages without..." · Reviewed by Greptile}

[greptile-apps]

Response changes escape checks

The breaking-change check still reads only fern/openapi.yml, but this PR moves response shapes such as ErrorResponse and RailsConfigInfo into fern/openapi-overlay.yml. Changes to those shapes now escape the comparison, and overlay-only changes do not trigger this workflow.

Compare the overlay-applied specifications from both revisions, and trigger the workflow on overlay changes.

Prompt To Fix With AI

This is a comment left during a code review.
Path: .github/workflows/api-conformance-docs.yml
Line: 74-76

Comment:
**Response changes escape checks**

The breaking-change check still reads only `fern/openapi.yml`, but this PR moves response shapes such as `ErrorResponse` and `RailsConfigInfo` into `fern/openapi-overlay.yml`. Changes to those shapes now escape the comparison, and overlay-only changes do not trigger this workflow.

Compare the overlay-applied specifications from both revisions, and trigger the workflow on overlay changes.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[Pouyanpi]

I'm wondering why it's being called overlay? I think the agent is conflating some concepts from the new experimental server

[Pouyanpi]

better to rename this, one can confuse it with openai's API spec