Skip to content

Build: Bump gcsfs from 2026.6.0 to 2026.8.1 - #4053

Merged
Fokko merged 1 commit into
mainfrom
dependabot/uv/gcsfs-2026.8.1
Oct 2, 2026
Merged

Fokko merged 1 commit into
mainfrom
dependabot/uv/gcsfs-2026.8.1

Conversation

@dependabot

@dependabot dependabot Bot commented on behalf of github Oct 1, 2026

Copy link
Copy Markdown
Contributor

Bumps gcsfs from 2026.6.0 to 2026.8.1.

Release notes

Sourced from gcsfs's releases.

2026.8.1

What's Changed

Bug Fixes Fixes fsspec/gcsfs#1048: In release 2026.8.0, cat_file began using a default concurrency of 4 instead of 1. This caused a performance regression for small cat_file operations with unknown sizes, as an unnecessary _info() request was being triggered to determine the size for concurrent reads.

Full Changelog: fsspec/gcsfs@2026.8.0...2026.8.1

2026.8.0

What's Changed

Adaptive Concurrent Prefetching is now the default read path

Enhanced read path via adaptive concurrent prefetching is now the default in GCSFS. Starting with this version, GCSFS predicts the next byte range an application will read, fetches it in the background across several concurrent HTTP requests, and keeps the bytes in memory before next read() is called. Network round-trips overlap with application compute instead of blocking calls where compute has to wait for data to be fetched. We are also enabling read concurrency, so a single reader is no longer limited by the bandwidth of a single HTTP connection.

GCSFS prefetcher adapts to workload read IO patterns. It tracks the rolling average of recent read sizes and scales the prefetch window linearly with the detected sequential streak, rather than using a fixed block size or exponential doubling. This is inline to what modern Linux kernels will do to balance prefetch and memory footprint. When the pattern turns random read i.e. we are not able to leverage the prefetched buffer to answer the next read() call, it drains the buffer to zero, so that random-access workloads pay no bandwidth or memory penalty.

Why this matters for AI/ML workloads

  • Model loading and checkpoint restore are typically large sequential reads - and prefetcher shines there. In our benchmarking, a single-stream sequential throughput improved from 23.69 MB/s to 658.71 MB/s for 1 MB I/O, and from 156 MB/s to 736 MB/s for 16 MB I/O.
  • Training data pipelines stay fed. Parquet and sharded dataset reads issue small-to-medium sequential ranges that previously suffered from low throughput, but now achieve significantly more; at 16 MB I/O throughput rises from 150 MB/s to 730 MB/s. Reducing the wait time for data loading improves accelerator goodput(amount of time accelerator is utilised for training than waiting).
  • Multi-worker dataloader scaling. The prefetcher manufactures its own parallelism per worker instead of relying on process count alone.
  • Accelerate the throughput even further with Rapid Buckets. With Rapid Buckets single node throughput reaches 21 GiB/s with 16-process sequentially reading at 16 MiB I/O compared to standard buckets with 48processes.

Adaptive prefetcher is enabled by default when cache_type is not explicitly set and concurrency value is set at 4(DEFAULT_GCSFS_CONCURRENCY=4) for both Standard and Rapid buckets. You can disable adaptive prefetcher by setting an explicit cache_type, or by setting USE_EXPERIMENTAL_ADAPTIVE_PREFETCHING='false', or by passing use_experimental_adaptive_prefetching=False to open() call.

(Warning) Impact on memory: Prefetching trades memory for throughput. Peak memory rises from ~170 MB to 600 MB on single-stream reads for 16 MB IO size and varies with requested IO sizes, and would be materially more under high process counts. Please ensure that application memory limits accordingly to use prefetcher without any Out of Memory(OOM) issues. To put hard limit, you can also use user_max_prefetch_size

For details on architecture, tuning, full benchmark tables, along with known limitations please refer to : https://github.com/fsspec/gcsfs/blob/main/docs/source/prefetcher.rst

(#795, #805, #818, #877)

Bug Fixes & Improvements

  • Zero-cost local backward seeks in PrefetchConsumer - Parquet footer and ZIP directory reads are served from the existing buffer instead of re-issuing a network request. (#930)
  • Concurrent downloads cap task count against a minimum chunk size, removing per-task overhead on small ranges. (#926)
  • Generation consistency across parallel fetches, guaranteeing every chunk comes from the same object version. (#921)
  • Fixed silent truncation on short reads in zonal bucket downloads. (#920)
  • Zero-copy read and write paths via memoryview, cutting CPU and transient memory in the hot path. (#840, #907, #928)
  • Graceful fallback where ctypes.pythonapi is unavailable. (#938)

New Contributors

Full Changelog: fsspec/gcsfs@2026.7.0...2026.8.0

2026.7.0

... (truncated)

Commits
  • 73929a1 fix: reverting default cat_file concurrency back to 1 (#1051) (#1055)
  • dc33f23 Merge branch 'main' into release-2026.8.0
  • e88cd8b lint fixes
  • d98d04b Update changelog.rst
  • d5c45dd Update changelog.rst
  • 158badb feat(subsystembenchmarks): add webdataset image dataloading read benchmark (#...
  • 2089c10 feat(subsystembenchmarks): generalize dataloading seam for multiple loaders (...
  • 158c361 Update pyproject.toml
  • c99137d subsystembenchmarks: support model id substitution in checkpointing save (#1003)
  • 9d38875 Track whether cache_type is explicitly set or defaulting in User-Agent header...
  • Additional commits viewable in compare view

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot show <dependency name> ignore conditions will show all of the ignore conditions of the specified dependency
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)

Bumps [gcsfs](https://github.com/fsspec/gcsfs) from 2026.6.0 to 2026.8.1.
- [Release notes](https://github.com/fsspec/gcsfs/releases)
- [Commits](fsspec/gcsfs@2026.6.0...2026.8.1)

---
updated-dependencies:
- dependency-name: gcsfs
  dependency-version: 2026.8.1
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
@dependabot dependabot Bot added dependencies Pull requests that update a dependency file python:uv Pull requests that update python:uv code labels Oct 1, 2026
@Fokko
Fokko added this pull request to the merge queue Oct 2, 2026
@Fokko

Fokko commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Thanks @ebyhr for the review 🚀

@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 2, 2026
@Fokko
Fokko added this pull request to the merge queue Oct 2, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 2, 2026
@Fokko
Fokko added this pull request to the merge queue Oct 2, 2026
Merged via the queue into main with commit d51065f Oct 2, 2026
21 checks passed
@Fokko
Fokko deleted the dependabot/uv/gcsfs-2026.8.1 branch October 2, 2026 21:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file python:uv Pull requests that update python:uv code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants