Skip to content

Commit d64b8d7

Browse files
theodorejbclaude
theodorejb
and
claude
committed
Add idColumn option to return actual insert IDs
insertRows() computes IDs from the last insert ID, assuming each batch received a contiguous range. With concurrent inserts, SQL Server and PostgreSQL can interleave identity values between sessions, so returned IDs could belong to other connections' rows. insertRow() and insertRows() now accept an optional idColumn argument to return the IDs from the insert query itself. SQL Server uses MERGE ... OUTPUT INTO a table variable, which outputs each identity value with the index of its row, and selects the IDs as a single JSON row (fetching one row per ID is slow with pdo_sqlsrv when there are many bound parameters). PostgreSQL uses RETURNING. Without idColumn, SQL Server now gets the last insert ID from SCOPE_IDENTITY() rather than @@IDENTITY (as before v7.0), so it's no longer wrong when a trigger inserts into another identity table, and no IDs are returned for tables with an INSTEAD OF INSERT trigger. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent cd1afb5 commit d64b8d7

11 files changed

Lines changed: 549 additions & 45 deletions

File tree

‎CHANGELOG.md‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,25 @@
11
# Changelog
22

3+
## [Unreleased]
4+
### Added
5+
- Optional `idColumn` parameter for `insertRow()` and `insertRows()`. With SQL Server and PostgreSQL,
6+
this returns the IDs from the insert query itself (using `MERGE ... OUTPUT` or `RETURNING`), so they
7+
are correct even when other connections insert into the same table at the same time, when inserting
8+
explicit identity values, and with any identity increment. With SQL Server, this requires 2016+.
9+
10+
### Fixed
11+
- With SQL Server, `insertRow()` and `insertRows()` returned the wrong ID when an insert trigger
12+
inserted into another table with an identity column. The IDs now come from `SCOPE_IDENTITY()`
13+
rather than `@@IDENTITY` (like before v7.0).
14+
- With SQL Server, the affected count from `insertRow()` and `insertRows()` no longer includes
15+
rows inserted by triggers.
16+
17+
### Changed
18+
- With SQL Server, `insertRow()` and `insertRows()` no longer return IDs for tables and views with an
19+
`INSTEAD OF INSERT` trigger, as before v7.0. Since v7.0 these were computed from `@@IDENTITY`,
20+
which isn't reliable with concurrent inserts.
21+
22+
323
## [7.0.2] - 2026-08-06
424
### Changed
525
- `Statement::getAll()` is now annotated as returning `list<mixed[]>` rather than an

‎README.md‎

Lines changed: 25 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
# PeachySQL
22

33
PeachySQL is a high-performance query builder and runner which streamlines prepared statements
4-
and working with large datasets. It is officially tested with MySQL, PostgreSQL, and SQL Server,
5-
but it should also work with any standards-compliant database which has a driver for PDO.
4+
and working with large datasets. It is officially tested with MySQL, PostgreSQL, and SQL Server
5+
(2016 or later), but it should also work with any standards-compliant database which has a driver for PDO.
66

77
## Install via Composer
88

@@ -194,8 +194,29 @@ $affected = $result->affected; // 3
194194
$queries = $result->queryCount; // 1
195195
```
196196

197-
An optional third parameter can be passed to `insertRows()` to override the default
198-
identity increment value:
197+
The `ids` are returned in the same order as the inserted rows. By default, they are computed from
198+
the last insert ID. With SQL Server and PostgreSQL, this can be incorrect if other connections are
199+
inserting into the same table at the same time, so you can instead pass the table's identity column
200+
as the `idColumn` argument to return the IDs from the insert query itself (via `MERGE ... OUTPUT`
201+
or `RETURNING`):
202+
203+
```php
204+
$result = $db->insertRows('Users', $userData, idColumn: 'user_id');
205+
```
206+
207+
This is also correct when inserting explicit identity values, and with any identity increment.
208+
Only pass `idColumn` for tables with an identity column, since the query will otherwise fail
209+
(or return the wrong IDs, if it's a different column). With SQL Server, don't pass it
210+
for a table or view with an `INSTEAD OF INSERT` trigger either, since the IDs of rows inserted by the
211+
trigger can't be output (without `idColumn`, no IDs are returned for these).
212+
213+
`insertRow()` also accepts an `idColumn` argument, but only needs it with PostgreSQL, where the ID
214+
otherwise comes from `lastval()`. This returns the last value from any sequence in the session,
215+
e.g. one used by an insert trigger, or from a previous insert into a different table.
216+
217+
With MySQL, `idColumn` isn't used, since InnoDB assigns consecutive IDs for multi-row inserts.
218+
When the IDs are computed from the last insert ID and the identity increment isn't 1 (e.g. if your
219+
MySQL server uses a different `auto_increment_increment`), pass it as the optional third parameter:
199220

200221
```php
201222
$result = $db->insertRows('Users', $userData, 2);

‎src/PeachySql.php‎

Lines changed: 57 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -140,15 +140,36 @@ public function query(string $sql, array $params = []): Statement
140140
/**
141141
* @param list<ColValues> $colVals
142142
*/
143-
private function insertBatch(string $table, array $colVals, int $identityIncrement = 1): BulkInsertResult
143+
private function insertBatch(string $table, array $colVals, int $identityIncrement, ?string $idColumn): BulkInsertResult
144144
{
145-
$sqlParams = (new Insert($this->options))->buildQuery($table, $colVals);
146-
$result = $this->query($sqlParams->sql, $sqlParams->params);
145+
$insert = new Insert($this->options);
146+
$driver = $this->options->driver;
147147

148-
try {
149-
$lastId = (int) $this->conn->lastInsertId();
150-
} catch (\PDOException $e) {
151-
$lastId = 0;
148+
// Unlike computing IDs from the last insert ID, returning the ID of each row from the
149+
// insert query itself is correct if other sessions insert into the same table concurrently.
150+
// With SQL Server, SCOPE_IDENTITY() is already correct for a single row.
151+
if ($idColumn !== null && $driver === 'sqlsrv' && count($colVals) > 1) {
152+
return $this->insertMergeOutput($insert->buildMergeOutputQuery($table, $colVals, $idColumn));
153+
} elseif ($idColumn !== null && $driver === 'pgsql') {
154+
return $this->insertReturningIds($insert->buildReturningQuery($table, $colVals, $idColumn));
155+
}
156+
157+
if ($driver === 'sqlsrv') {
158+
// unlike lastInsertId() (which uses @@IDENTITY), SCOPE_IDENTITY() excludes IDs generated by triggers
159+
$sqlParams = $insert->buildScopeIdentityQuery($table, $colVals);
160+
/** @var array{id: int|string|null, affected: int|string} $row */
161+
$row = $this->query($sqlParams->sql, $sqlParams->params)->getFirst();
162+
$lastId = (int) $row['id'];
163+
$affected = (int) $row['affected'];
164+
} else {
165+
$sqlParams = $insert->buildQuery($table, $colVals);
166+
$affected = $this->query($sqlParams->sql, $sqlParams->params)->getAffected();
167+
168+
try {
169+
$lastId = (int) $this->conn->lastInsertId();
170+
} catch (\PDOException $e) {
171+
$lastId = 0;
172+
}
152173
}
153174

154175
if ($lastId) {
@@ -164,7 +185,25 @@ private function insertBatch(string $table, array $colVals, int $identityIncreme
164185
$ids = [];
165186
}
166187

167-
return new BulkInsertResult($ids, $result->getAffected());
188+
return new BulkInsertResult($ids, $affected);
189+
}
190+
191+
private function insertReturningIds(SqlParams $sqlParams): BulkInsertResult
192+
{
193+
/** @var list<array{id: int|string}> $rows */
194+
$rows = $this->query($sqlParams->sql, $sqlParams->params)->getAll();
195+
$ids = array_map(fn(array $row) => (int) $row['id'], $rows);
196+
return new BulkInsertResult($ids, count($ids));
197+
}
198+
199+
private function insertMergeOutput(SqlParams $sqlParams): BulkInsertResult
200+
{
201+
/** @var array{ids: string|null} $row */
202+
$row = $this->query($sqlParams->sql, $sqlParams->params)->getFirst();
203+
/** @var list<array{id: int}> $idRows */
204+
$idRows = $row['ids'] === null ? [] : json_decode($row['ids'], true, flags: JSON_THROW_ON_ERROR);
205+
$ids = array_column($idRows, 'id');
206+
return new BulkInsertResult($ids, count($ids));
168207
}
169208

170209
public function selectFrom(string $query): QueryableSelector
@@ -180,27 +219,33 @@ public function select(SqlParams $query): QueryableSelector
180219
/**
181220
* Inserts one row
182221
* @param ColValues $colVals
222+
* @param string|null $idColumn The identity column, to return the ID from the insert query with PostgreSQL
223+
* (the ID otherwise comes from lastval(), which may be for a different sequence)
183224
*/
184-
public function insertRow(string $table, array $colVals): InsertResult
225+
public function insertRow(string $table, array $colVals, ?string $idColumn = null): InsertResult
185226
{
186-
$result = $this->insertBatch($table, [$colVals]);
227+
$result = $this->insertBatch($table, [$colVals], 1, $idColumn);
187228
$ids = $result->ids;
188229
return new InsertResult($ids ? $ids[0] : 0, $result->affected);
189230
}
190231

191232
/**
192233
* Insert multiple rows
193234
* @param list<ColValues> $colVals
235+
* @param int $identityIncrement Used to compute the IDs when they aren't returned by the insert query
236+
* @param string|null $idColumn The identity column, to return the IDs from the insert query with SQL Server
237+
* and PostgreSQL. Unlike computing the IDs from the last insert ID, this is
238+
* correct if other sessions insert concurrently.
194239
*/
195-
public function insertRows(string $table, array $colVals, int $identityIncrement = 1): BulkInsertResult
240+
public function insertRows(string $table, array $colVals, int $identityIncrement = 1, ?string $idColumn = null): BulkInsertResult
196241
{
197242
// check whether the query needs to be split into multiple batches
198243
$batches = Insert::batchRows($colVals, $this->options->maxBoundParams, $this->options->maxInsertRows);
199244
$ids = [];
200245
$affected = 0;
201246

202247
foreach ($batches as $batch) {
203-
$result = $this->insertBatch($table, $batch, $identityIncrement);
248+
$result = $this->insertBatch($table, $batch, $identityIncrement, $idColumn);
204249
$ids = array_merge($ids, $result->ids);
205250
$affected += $result->affected;
206251
}

‎src/QueryBuilder/Insert.php‎

Lines changed: 81 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -39,17 +39,91 @@ public static function batchRows(array $colVals, int $maxBoundParams, int $maxRo
3939
*/
4040
public function buildQuery(string $table, array $colVals): SqlParams
4141
{
42-
if (!$colVals || empty($colVals[0])) {
43-
throw new \Exception('A valid array of columns/values to insert must be specified');
44-
}
45-
46-
$columns = $this->escapeColumns(array_keys($colVals[0]));
42+
$columns = $this->getEscapedColumns($colVals);
4743
$insert = "INSERT INTO {$table} (" . implode(', ', $columns) . ')';
4844

4945
$valSetStr = ' (' . str_repeat('?,', count($columns) - 1) . '?),';
5046
$valStr = ' VALUES' . substr_replace(str_repeat($valSetStr, count($colVals)), '', -1); // remove trailing comma
51-
$params = array_merge(...array_map(array_values(...), $colVals));
5247

53-
return new SqlParams($insert . $valStr, $params);
48+
return new SqlParams($insert . $valStr, $this->getParams($colVals));
49+
}
50+
51+
/**
52+
* Generates a PostgreSQL INSERT query which returns the ID of each row, in the order of the VALUES list.
53+
* An INSERT ... SELECT with ORDER BY would formally guarantee the order that serial values are assigned
54+
* (as with SQL Server), but parameters in a VALUES subquery are typed as text rather than being
55+
* coerced to the column types, which would break inserts into non-text columns.
56+
* @param list<ColValues> $colVals
57+
*/
58+
public function buildReturningQuery(string $table, array $colVals, string $idColumn): SqlParams
59+
{
60+
$query = $this->buildQuery($table, $colVals);
61+
$sql = $query->sql . ' RETURNING ' . $this->quoteIdentifier($idColumn) . ' AS id';
62+
63+
return new SqlParams($sql, $query->params);
64+
}
65+
66+
/**
67+
* Generates a SQL Server batch which inserts the rows and selects the last identity value and affected count.
68+
* Unlike @@IDENTITY, SCOPE_IDENTITY() excludes identity values generated by triggers, and is
69+
* null rather than an error if the table has no identity column.
70+
* @param list<ColValues> $colVals
71+
*/
72+
public function buildScopeIdentityQuery(string $table, array $colVals): SqlParams
73+
{
74+
$query = $this->buildQuery($table, $colVals);
75+
$sql = $query->sql . '; SELECT CAST(SCOPE_IDENTITY() AS bigint) AS id, @@ROWCOUNT AS affected';
76+
77+
return new SqlParams($sql, $query->params);
78+
}
79+
80+
/**
81+
* Generates a SQL Server batch which inserts the rows and selects their identity values in row order.
82+
* Unlike INSERT, MERGE can output columns from the source rows, so each identity value is output
83+
* along with the index of its row. This doesn't depend on the order that identity values are
84+
* assigned (which can differ from row order, e.g. with IDENTITY_INSERT), or the order of OUTPUT rows.
85+
* OUTPUT INTO is used since OUTPUT without INTO isn't allowed on tables with triggers.
86+
* The IDs are selected as a single JSON array, since with pdo_sqlsrv each fetched row
87+
* is slow when the statement has many bound parameters.
88+
* @param list<ColValues> $colVals
89+
*/
90+
public function buildMergeOutputQuery(string $table, array $colVals, string $idColumn): SqlParams
91+
{
92+
$columns = $this->getEscapedColumns($colVals);
93+
$aliases = array_map(fn($i) => "p{$i}", array_keys($columns));
94+
$placeholders = str_repeat('?,', count($columns));
95+
$valSets = array_map(fn($i) => "({$placeholders}{$i})", array_keys($colVals));
96+
97+
$sql = 'DECLARE @ids TABLE (id bigint, _rn int); '
98+
. "MERGE INTO {$table} USING (VALUES " . implode(', ', $valSets) . ')'
99+
. ' AS v (' . implode(', ', $aliases) . ', _rn) ON 1 = 0'
100+
. ' WHEN NOT MATCHED THEN INSERT (' . implode(', ', $columns) . ')'
101+
. ' VALUES (' . implode(', ', array_map(fn($a) => "v.{$a}", $aliases)) . ')'
102+
. ' OUTPUT inserted.' . $this->quoteIdentifier($idColumn) . ', v._rn INTO @ids;'
103+
. ' SELECT (SELECT id FROM @ids ORDER BY _rn FOR JSON PATH) AS ids';
104+
105+
return new SqlParams($sql, $this->getParams($colVals));
106+
}
107+
108+
/**
109+
* @param list<ColValues> $colVals
110+
* @return string[]
111+
*/
112+
private function getEscapedColumns(array $colVals): array
113+
{
114+
if (!$colVals || empty($colVals[0])) {
115+
throw new \Exception('A valid array of columns/values to insert must be specified');
116+
}
117+
118+
return $this->escapeColumns(array_keys($colVals[0]));
119+
}
120+
121+
/**
122+
* @param list<ColValues> $colVals
123+
* @return list<mixed>
124+
*/
125+
private function getParams(array $colVals): array
126+
{
127+
return array_merge(...array_map(array_values(...), $colVals));
54128
}
55129
}

‎src/QueryBuilder/Query.php‎

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -50,15 +50,19 @@ public function escapeIdentifier(string $identifier): string
5050
throw new \InvalidArgumentException('Identifier cannot be blank');
5151
}
5252

53-
$escaper = function (string $identifier): string {
54-
$c = $this->options->identifierQuote;
55-
return $c . str_replace($c, $c . $c, $identifier) . $c;
56-
};
57-
58-
$qualifiedIdentifiers = array_map($escaper, explode('.', $identifier));
53+
$qualifiedIdentifiers = array_map($this->quoteIdentifier(...), explode('.', $identifier));
5954
return implode('.', $qualifiedIdentifiers);
6055
}
6156

57+
/**
58+
* Quotes a single identifier (which may contain periods)
59+
*/
60+
protected function quoteIdentifier(string $identifier): string
61+
{
62+
$c = $this->options->identifierQuote;
63+
return $c . str_replace($c, $c . $c, $identifier) . $c;
64+
}
65+
6266
/**
6367
* @param WhereClause $columnVals
6468
* @throws \Exception if a column filter is empty

0 commit comments

Comments
 (0)