RFC: split the interface and chunk types into a dependency-free DiskArraysCore

[mkitti]

Stacked on #345 (the first commit, "Reduce method invalidations on load", is that PR; review only the second commit, "Split the interface and chunk types out into DiskArraysCore"). RFC / draft: I'd like feedback on the direction before polishing (docs, registration, naming).

Motivation

Packages that want to be a disk array (e.g. HDF5.Dataset, JuliaIO/HDF5.jl#1246, Zarr, NetCDF, ...) currently have to depend on all of DiskArrays, which brings in OffsetArrays, LRUCache, ConstructionBase and, more importantly, the methods that DiskArrays adds to Base functions for AbstractDiskArray (indexing, broadcast, mapreduce, zip, Generator, ...). Those invalidate compiled code in every session that loads the package, whether or not the user ever touches a disk array through the generic path (916 invalidated MethodInstances on main, 191 after #345).

The interface itself (the abstract type, readblock!/writeblock!/eachchunk/haschunks/chunkexists and the chunk types) needs none of that. Splitting it out lets implementers depend only on the interface, and lets the generic implementations be opt-in for the user (using DiskArrays).

What this PR does

Adds a dependency-free subpackage lib/DiskArraysCore (version 0.1.0) containing, moved verbatim from src/:

• diskarray.jl: AbstractDiskArray, isdisk, readblock!, writeblock!, eachchunk, haschunks, chunkexists
• chunks.jl: RegularChunks, IrregularChunks, GridChunks, Chunked/Unchunked, ChunkIndex, ChunkIndices, estimate_chunksize, ...
• the BatchStrategy types (NoBatch, SubRanges, ChunkRead, AllowStepRange, ...), the first ~70 lines of batchgetindex.jl, because Chunked()/Unchunked() default to them

DiskArrays depends on DiskArraysCore and imports every one of those names, so DiskArrays.AbstractDiskArray === DiskArraysCore.AbstractDiskArray etc. Nothing changes for current users or for packages that extend DiskArrays.readblock!, etc. (tested). The only code edit besides the move: wrapchunk (the one use of OffsetArrays in the moved files) now lives in DiskArrays.

Measured (SnoopCompile @snoop_invalidations, Julia 1.13):

	invalidated MethodInstances
using DiskArraysCore	0
using DiskArrays (with #345)	191

Things to decide / known limitations

• What a core-only subtype gets. A type that subtypes DiskArraysCore.AbstractDiskArray gets the interface but no generic getindex, broadcast, mapreduce, ... until DiskArrays is loaded; at that point the existing @implement_diskarray AbstractDiskArray methods apply to it. So a package depending only on Core needs a fallback for basic indexing (or declares that DiskArrays is needed) and should be designed so both states work. This is the main design question: is this layering acceptable or should Core also carry minimal scalar/block getindex?
• Naming / location. DiskArraysCore in lib/ (monorepo, registered with subdir = "lib/DiskArraysCore") vs. a separate repository; happy to change.
• Registration. DiskArraysCore has to be registered before this can be released. [sources] in Project.toml handles dev/test on Julia >= 1.11; CI runs Pkg.develop first so 1.10 works too, and runs the Core tests.
• Docs are untouched (docstrings moved with their bindings, @refs to them still resolve to the same objects) but I haven't built the docs locally.
• OffsetArrays (10 invalidations: in(::Any, ::OffsetArray)) is only a dependency of DiskArrays, not Core.

Testing

• DiskArrays: full suite passes (including Aqua).
• DiskArraysCore: new suite (a minimal interface-only disk array, chunk types, a guard that Core defines no methods on Base.Generator/zip/collect/map).
• Added a test that the names re-exported by DiskArrays are identical to the Core ones.

[meggart]

To me this sounds like a very good approach, in particular with respect to bringing DiskArrays into HDF5 in a very unintrusive way. I am still not entirely sure if this plan would work out without extensive type piracy between DiskArraysCore.jl and DiskArrays.jl. In the end the problem remains that there will be non-local behavior changes in the code depending on whether or not a user has loaded a package that requires DiskArrays in their environment. So, the whole indexing machinery in a HDF5 dataset would change if somewhere else in the code someone is doing using NCDatasets, this would be a bit surprising.

Maybe the better approach for integrating into HDF5 would be to have a DiskArrays extension and add an option to allow e.g. h5open(filename; diskarray=false/true)or similar so users can explicitly opt-in into the diskarray behavior through kwarg instead of through ausing DiskArrays`.