Repository navigation
Commit a5faa32
PERF: Optimize Row construction and repeated cursor bookkeeping (#558)
### Work Item / Issue Reference
<!--
IMPORTANT: Please follow the PR template guidelines below.
For mssql-python maintainers: Insert your ADO Work Item ID below
For external contributors: Insert Github Issue number below
Only one reference is required - either GitHub issue OR ADO Work Item.
-->
<!-- mssql-python maintainers: ADO Work Item -->
>
[AB#44921](https://sqlclientdrivers.visualstudio.com/c6d89619-62de-46a0-8b46-70b92a84d85e/_workitems/edit/44921)
<!-- External contributors: GitHub Issue -->
### Summary
Reduce Python-side overhead in `fetchone()`, `fetchmany()`, and
`fetchall()` while preserving decoding, converter, diagnostic, and
Row-access behavior. The branch incorporates main through `e279a4f6` and
includes the review fixes in `56c3d3b5`.
#### Fetch optimizations
- Construct eligible `fetchmany()`/`fetchall()` rows in native
`construct_rows`, bypassing the Python per-row initialization loop. Use
`Row._fast_create()` for the corresponding `fetchone()` path.
- Add `Row.__slots__` and retain zero-copy storage of fetched values
when no conversion is required. Initialize the shared lowercase column
map on both Python and native construction paths.
- Cache CHAR/WCHAR encoding strings and the CHAR C type. Refresh them
only when connection decoding settings change, including for existing
cursors.
- Reuse the SQL-to-C type mapping and use `str.isascii()` for Unicode
detection.
- Keep the no-conversion fast path available when converters are
registered but none applies to the current result set. An explicit empty
mapping avoids unnecessary row copies or per-row fallback lookups.
#### Correctness and compatibility
- Track converter-generation changes so adding, replacing, removing, or
clearing converters after `execute()` updates subsequent fetches without
rebuilding mappings on unchanged cache hits.
- Preserve current-main dispatch precedence: raw ODBC SQL type, Python
type, then the string/bytes-only WVARCHAR fallback. Unrelated integer
and UUID columns are not passed to the string fallback.
- Preserve direct `Row` construction's connection-converter fallback
when no precomputed mapping is supplied, including UUID stringification.
- Retain main's lowercase/string-key column access, profiling scopes,
and CHAR decoding C-type forwarding. Native WCHAR decoding semantics are
unchanged.
- Validate `row_class` with `PyType_Check` before casting in the
Python-callable native helper. Invalid arguments raise `TypeError`
rather than crashing the interpreter.
- Restore unconditional diagnostic retrieval after non-error fetch
returns. The native wrappers can overwrite intermediate statuses, so
checking only the final `SQL_SUCCESS_WITH_INFO` is unsafe. The earlier
diagnostic-skip optimization has been removed; this restores prior
behavior rather than claiming to repair pre-existing native
diagnostic-record overwrites.
- Check errors in `fetchone()` and `fetchmany()` before updating row
positions or constructing rows, consistent with `fetchall()`.
### Performance: current main versus this PR
| Workload (rows x columns) | API | Main median | PR median | Time
reduction |
|---|---|---:|---:|---:|
| Narrow: 10,000 x 3 | `fetchmany(1000)` | 9.36 ms | 6.54 ms |
**30.18%** |
| Narrow: 10,000 x 3 | `fetchall()` | 8.67 ms | 6.99 ms | **19.30%** |
| Wide: 10,000 x 24 | `fetchmany(1000)` | 27.57 ms | 24.72 ms |
**10.34%** |
| Wide: 10,000 x 24 | `fetchall()` | 30.59 ms | 26.07 ms | **14.79%** |
Positive reduction means less time, calculated from unrounded medians.
Narrow/wide batch cases improved in 10/10 and 9/10 pairs respectively.
`fetchmany(1)` medians were only 1.15-1.80% lower, with narrow results
inconclusive. Narrow `fetchone()` was 5.02% lower; wide `fetchone()`.
These are workload-specific results, not a universal or cross-platform
speedup. Full results for all 11 fetch cases, control, dispersion,
paired intervals, build identities, and raw repetitions are retained
locally; older pre-merge numbers are not mixed in.
---------
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Gaurav Sharma <sharmag@microsoft.com>1 parent f3e34e6 commit a5faa32
6 files changed
Lines changed: 1188 additions & 85 deletions
File tree
- mssql_python
- pybind
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
540 | 540 | | |
541 | 541 | | |
542 | 542 | | |
| 543 | + | |
543 | 544 | | |
544 | 545 | | |
545 | 546 | | |
| |||
782 | 783 | | |
783 | 784 | | |
784 | 785 | | |
| 786 | + | |
785 | 787 | | |
786 | 788 | | |
787 | 789 | | |
| |||
1320 | 1322 | | |
1321 | 1323 | | |
1322 | 1324 | | |
| 1325 | + | |
| 1326 | + | |
| 1327 | + | |
1323 | 1328 | | |
1324 | 1329 | | |
1325 | 1330 | | |
| |||
1443 | 1448 | | |
1444 | 1449 | | |
1445 | 1450 | | |
| 1451 | + | |
1446 | 1452 | | |
1447 | 1453 | | |
1448 | 1454 | | |
| |||
1647 | 1653 | | |
1648 | 1654 | | |
1649 | 1655 | | |
| 1656 | + | |
| 1657 | + | |
1650 | 1658 | | |
1651 | 1659 | | |
1652 | 1660 | | |
| |||
1687 | 1695 | | |
1688 | 1696 | | |
1689 | 1697 | | |
| 1698 | + | |
1690 | 1699 | | |
1691 | 1700 | | |
1692 | 1701 | | |
| |||
1717 | 1726 | | |
1718 | 1727 | | |
1719 | 1728 | | |
| 1729 | + | |
| 1730 | + | |
1720 | 1731 | | |
1721 | 1732 | | |
1722 | 1733 | | |
| |||
1726 | 1737 | | |
1727 | 1738 | | |
1728 | 1739 | | |
| 1740 | + | |
1729 | 1741 | | |
1730 | 1742 | | |
1731 | 1743 | | |
| |||
1737 | 1749 | | |
1738 | 1750 | | |
1739 | 1751 | | |
| 1752 | + | |
| 1753 | + | |
1740 | 1754 | | |
1741 | 1755 | | |
1742 | 1756 | | |
1743 | 1757 | | |
1744 | 1758 | | |
| 1759 | + | |
1745 | 1760 | | |
1746 | 1761 | | |
1747 | 1762 | | |
| |||
0 commit comments