Skip to content

Add PostgreSQL partial-column inserts with composable CTE support - #827

Draft
kushagarr wants to merge 12 commits into
haskell-beam:masterfrom
kushagarr:cte_support_postgres
Draft

Add PostgreSQL partial-column inserts with composable CTE support#827
kushagarr wants to merge 12 commits into
haskell-beam:masterfrom
kushagarr:cte_support_postgres

Conversation

@kushagarr

@kushagarr kushagarr commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds PostgreSQL-specific partial-column inserts that preserve ON CONFLICT support and compose cleanly with RETURNING and data-modifying CTEs.

Motivation

The existing PostgreSQL insert helpers support insertValues, insertExpressions, and insertFrom, but require values for every table column. This prevents CTE inserts from omitting generated, identity, or database-defaulted columns.

Beam’s backend-independent insertOnly supports partial-column inserts, but cannot express PostgreSQL’s ON CONFLICT behavior.

Changes

  • Add pgInsertOnly for partial-column PostgreSQL inserts with PgInsertOnConflict.
  • Add pgCteInsert to lift any SqlInsert Postgres table into a side-effect-only CTE.
  • Add pgCteInsertReturning to attach a RETURNING projection and expose the inserted rows as a reusable CTE.
  • Refactor the existing full-row insert helpers through the same implementation.
  • Preserve the existing insert, insertReturning, cteInsert, and cteInsertReturning APIs.
  • Document the new composition model and record the APIs for beam-postgres-0.6.3.1.
  • Allow the integration suite to use an externally managed PostgreSQL instance through BEAM_POSTGRES_TEST_CONNSTR.

Design

Partial-column selection is treated as a property of insert construction, while conflict handling, RETURNING, and CTE placement remain independently composable:

command =
  Pg.pgInsertOnly table selectedFields values onConflict

runInsert command
returning command resultProjection
Pg.pgCteInsert command
Pg.pgCteInsertReturning command resultProjection

pgInsertOnly delegates target-column projection and base SQL generation to Beam’s existing insertOnly implementation, then adds PostgreSQL’s conflict clause. This avoids maintaining a second insert renderer and keeps column ordering, quoting, empty-input behavior, and type checking aligned with the backend-independent implementation.

The command-level CTE adapters also avoid introducing narrowly specialized APIs such as pgCteInsertOnly.

Compatibility

This is additive and does not require changes to existing user code. Existing PostgreSQL insert and CTE helpers retain their signatures and behavior.

pgCteInsertReturning returns Nothing only for SqlInsertNoRows. A valid ON CONFLICT DO NOTHING command still returns a reusable relation, which may contain no rows after execution.

Validation

  • Full beam-postgres test suite: 149 tests passed against PostgreSQL 18.4.
  • Added rendering and integration coverage for:
    • selected-column ordering;
    • insertData and insertFrom;
    • generated identities and database defaults;
    • ON CONFLICT DO NOTHING;
    • ON CONFLICT DO UPDATE and excluded values;
    • SQL parameter ordering;
    • side-effect-only and returning CTEs;
    • empty inserts and zero-column projections;
    • compatibility between existing helpers and the composable command path.
  • Added negative type tests for invalid CTE placement and mismatched insert projections.
  • Haddock generation completed successfully.
  • git diff --check passes.

kushagarr added 12 commits July 11, 2026 05:19
- Introduced `pgWithSyntax` to prefix PostgreSQL statements with CTEs.
- Added new types for data-modifying CTEs: `PgDataModifyingCommonTableExpressionSyntax` and `PgCommonTableExpressionSyntax`.
- Implemented type classes for handling CTEs in SQL queries, ensuring correct placement of data-modifying CTEs.
- Created unit and integration tests for rendering and type safety of CTEs, including negative tests for invalid placements.
- Updated documentation to reflect new CTE capabilities and usage examples.
- Modified test setup to use PostgreSQL version 18.4 for better compatibility.
- Updated CTE handling in tests to reflect PostgreSQL's behavior with zero-column CTEs and side-effect-only CTEs.
- Added new tests for materialization execution and rendering, ensuring compliance with PostgreSQL 12+ features.
- Refined error handling for invalid CTE placements, particularly for side-effect-only operations.
- Improved documentation on PostgreSQL-specific CTE usage, including materialization options and nested CTEs.
- Adjusted dependencies in the SQLite library for compatibility.
- Ensured reproducibility of CI runs by pinning PostgreSQL server version.
@kushagarr

Copy link
Copy Markdown
Contributor Author

@LaurentRDC could you check this please.

@LaurentRDC

Copy link
Copy Markdown
Member

I haven't forgotten about this; I'm just swamped at work right now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants