Skip to content

Add Zstandard (Zstd) Compress and Decompress operations#2373

Open
lzandman wants to merge 14 commits into
gchq:masterfrom
lzandman:leon/add-zstd-compress-decompress
Open

Add Zstandard (Zstd) Compress and Decompress operations#2373
lzandman wants to merge 14 commits into
gchq:masterfrom
lzandman:leon/add-zstd-compress-decompress

Conversation

@lzandman
Copy link
Copy Markdown

Description
Zstandard is a modern lossless compression algorithm developed by Meta. It is widely used in Linux kernels, databases, container images, and network protocols due to its high compression ratio at fast speeds.

This PR contains the following:

  • Zstd Compress operation — compresses input data using the Zstandard algorithm, with a configurable compression level (1–22, default: 3).
  • Zstd Decompress operation — decompresses Zstd-compressed data, with auto-detection support via the magic bytes 28 B5 2F FD.
  • Zstd file type detection in FileSignatures.mjs — recognises both standard Zstd frames and skippable frames (.zst, application/zstd).

Existing Issue
This previously was requested in #1470, though I decided to add this feature based on my own requirements.

Test Coverage

  • Compress arbitrary input and verify the output starts with 28 B5 2F FD (Zstd magic bytes).
  • Round-trip: compress then decompress, confirm output matches original input.
  • Verify auto-detection triggers Zstd Decompress when pasting Zstd-compressed data.
  • Confirm compression levels 1 and 22 both produce valid output.
  • Run the existing unit tests (tests/operations/tests/Zstd.mjs).

Notes
This PR relies on the @bokuweb/zstd-wasm npm package, which provides a WebAssembly build of the Zstd library.

AI disclosure
Claude Code used to create tests.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant