Conversation
… types Adds prost-generated protobuf types from proto_database.proto matching the Blizzard Agent product.db binary format. Messages: Database, ProductInstall, ProductConfig, ProductLanguage, ProductSettings, PatchManifest, Uid. Integration tests against 3 real WoW Classic .product.db fixtures (1.13.2, 1.14.0, 1.15.2 macOS).
…e storage Stores content as loose files with Turso-backed encrypted SQLite metadata. Components: ContainerlessDb, loose, sparse, bgdl, block_mover, eheader, product_db, residency, ContainerlessStorage.
…ence data Providers: BlizzTrack API, wago.tools TACT keys, WoWDev/TACTKeys text format, community FileDataID listfiles. ImportManager coordinates all providers with configurable refresh schedules.
…pipeline Pipeline stages: classify, download, install, metadata, manifests, loose, update (with leeching, patch resolution). Patch application: ZBSDIFF1, block-patch, BLTE re-encode. Supporting: CDN source, checkpoint, endpoint scorer, layout parsers, community mirror list.
…intenance GC, compaction (defrag, fill-holes, merger, mover), preservation, and repair (data, loose files, KMT markers).
FileDataIdResolver, ContentCategorizer, MetadataOrchestrator for the installation pipeline.
Compatible with the Blizzard Agent API. Full axum route table covering all known endpoints. State: InstallationDb (SQLite), OperationQueue, ProductRegistry, SizeCache. Binary: cascette-agent with clap CLI.
…KeyProvider - murmur3_fmix64: residency fast-path hashing - armadillo: read Armadillo encryption key files - generate: generate_tact_key(), generate_salsa20_iv() via OS CSPRNG - ChainedKeyProvider: chain-of-responsibility lookup across multiple TactKeyProvider backends - examples: crypto_primitives, salsa20_encryption, tact_keyring, armadillo_keys
- watch: file watcher for hot-reload of builds.json without server restart - start_tls_server(): HTTPS via axum_server with ring-backed rustls - BPSV: seqn line now between header and data rows (was at end) - AppState::database() is now async with Arc<BuildDatabase> snapshot - current_seqn() takes product: &str, returns per-product AtomicU64 - Benchmark snapshots Arc<BuildDatabase> before bench loops
…ent updates - download_from_endpoints(): CDN endpoint failover with retry on failure - download_range(): RetryPolicy handling 416, 429, 5xx responses - ArchiveReader::with_keys, ArchiveFile::read_blte_at_offset_with_keys accept Arc<dyn TactKeyProvider> instead of concrete TactKeyStore - examples: cdn_protocol_verification, cdn_chain_verification
…, examples - UpdateEntry (24-byte), UpdatePage (512-byte, 21 entries), UpdateSection with merge-sort flush to sorted section via atomic file replacement - KMT v8 hash guard: Jenkins hashlittle on entry[4..37], seed 0, OR 0x80000000 - Reconstruction header checksum_a: Jenkins hashlittle on first 22 bytes, seed 0x3D6BE971 - Reconstruction header checksum_b: XOR over first 26 bytes with rotating index - example: local_verification
Added: - patch_chain module and patch_archive/location module - KeyringConfig: parse/build/validate keyring config files - PatchIndex: block-based format with header, type-2/type-8 entries, builder - PatchArchive: header hash verification, block-based format rewrite, extended header support (flags bit 1), block sort validation - BuildConfig typed accessors: build_file_db, client_version, chunk_entries, feature_use_hardlinks, feature_placeholder, install_high_ver, key_layout_index_bits, key_layout_entries, no_frame_encoding, vfs_espec, vfs_root_espec - 7 format examples: parse_build_config, blte_encoding, manifests, encoding_table, root_file, archive_index, binary_patches - CDN test fixtures and integration tests for: ESpec (50 strings, 14 tests), download manifest (100 entries, 7 tests), install manifest (2 products, 10 tests), root file (V1/V2, 11 tests), ZBSDIFF1 (5 triplets, round-trip), CDN archive index (3 files, 9 tests), patch archive (3 files, 9 tests), TVFS (3 products, 10 tests), encoding (2 files, 14 tests), build config (3 files, 14 tests), keyring config (3 files, 16 tests), patch index (3 files, 12 tests) - ZbsdiffBuilder: suffix array bsdiff using divsufsort, near-optimal patches Fixed: - BLTE: reject single-chunk encrypted, nested encryption; force multi-chunk header; accept 4 or 8 byte IVs; prepend inner compression mode byte - ESpec: accept 1-8 byte IVs (was exactly 4); reject multiple variable blocks; parse shorthand block table form b:256K*=z - Encoding: reject zero page sizes; padding sentinel espec_index 0xFFFFFFFF - Root: content/locale flags corrected (10 of 12 wrong values fixed); default magic changed to TSFM; extended header size handling fixed; version detection accepts V1; name hash normalized to backslashes; word swap removed - Download: flag_size validation added (reject > 4) - Install: V2 extended header support (content_key_size, entry_count_v2) - ZBSDIFF1: endianness corrected to LE; control block sign-magnitude encoding - CDN archive index: ekey_length range relaxed to 1-16; element_count doc corrected; build() fixed to use entries not element_count; TYPICAL_TRUNCATED_KEY_SIZE - PatchArchive: block-based format replaces incorrect flat-entry model; extended header parsing; block sort validation; key size range 1-16 - BuildConfig: build_partial_priority split on spaces not commas; validate() skips feature flags and espec strings; removed typo key build-playbuild-installer - TVFS: rewritten to match real CDN data; recursive prefix tree; span-based VFS; byte-offset CFT; header error shows actual vs expected; new methods cft_offs_size, est_offs_size, cft_entry_size
- Members: cascette-agent, cascette-containerless, cascette-import, cascette-installation, cascette-maintenance, cascette-metadata, cascette-proto - Dependencies: uuid, chrono, sysinfo, tokio-util, turso, prost, prost-types, tower, http-body-util, libc - .mise.toml: add protobuf = "latest" for protoc compiler - README.md: document new crate capabilities; note protoc as dev dependency
- Agent HTTP API reference: full endpoint documentation with request/response shapes (was a stub) - Encoding format: updated to use actual cascette-formats API; header flags field; builder example corrected; padding sentinel language updated - Config formats: missing build config keys added; key-value separator corrected - TVFS: rewritten to match real format (path table, VFS, CFT); analysis section simplified - Patches: ZBSDIFF1 header endianness note corrected; format descriptions updated - BLTE: LZ4 codec note updated - operations/references.md: BlizzTrack API documented - operations/mirroring.md: community mirrors documented - Summary and README updated for new crates and docs sections
… paste advisory - ci: add arduino/setup-protoc@v3 to quick-checks, test, docs, and coverage jobs so cascette-proto build.rs finds protoc - cascette-crypto: gate generate module and getrandom dep behind cfg(not(target_arch = "wasm32")) so cascette-crypto compiles for WASM - deny.toml: allow CC0-1.0 (notify via cascette-ribbit) and MPL-2.0 (option-ext via dirs via cascette-crypto) - deny.toml: ignore RUSTSEC-2024-0436 (paste unmaintained, pulled in by turso_core — no upgrade available upstream)
…ions - endpoint_scorer: use powi(5) and powi(3) instead of powf for integer exponents (clippy::suboptimal_flops) - handler_tests: fix test_get_hardware assertions to match the flat wire format (cpu_num_cores, memory, cpu_arch, gpu_1) instead of a nested structure that was never implemented
… validation - Fix test_get_hardware to use flat field names matching handler output - Fix test_post_and_get_session to not send a PID (no live game in CI) - Fix clippy: redundant_clone + useless_vec in coordinator update pipeline
- Add #[allow(unwrap_used, expect_used)] to test modules in markers.rs, generate.rs, and hardware.rs - Add #[allow(unwrap_used, expect_used, uninlined_format_args)] to armadillo_keys example - Suppress unnecessary_box_returns in store_trait.rs test helper - Fix redundant_closure and bool_to_int_with_if in hardware handler - Fix significant_drop_tightening in override_config handler - Fix too_long_first_doc_paragraph in admin handler - Fix manual_string_new in containerless product_db test - Fix needless_raw_string_hashes in handler_tests
Documents the exact CI commands to run locally before submitting changes. Covers the RUSTFLAGS="-D warnings" requirement and common pitfalls around test module allow attributes.
Adds 89 new integration tests across five crates, modeled on the bootstrap and extraction workflows used by TACTSharp, wow.export, BuildBackup, CASCHost, CascLib, and tactmon. Test files added: - cascette-formats/tests/blte_fixtures.rs (25 tests) Covers BLTE parse, decompress, round-trip, MD5 content-key verification, LZ4 codec, and error rejection using binary fixtures. - cascette-formats/tests/bpsv_fixtures.rs (21 tests) Covers BPSV versions/cdns parse, field validation, seqn extraction, round-trip via CascFormat trait, and error cases using text fixtures. - cascette-installation/tests/mirror_tests.rs (14 tests) Covers MirrorConfig endpoint ordering for current vs historic builds, fallback flags, HTTPS scheme enforcement, and backoff constants. - cascette-import/tests/listfile_tests.rs (18 tests) Covers TactKeyStore TXT and CSV loading, 0x prefix parsing, comment skipping, deduplication, size limits, and ListfileProvider construction. - cascette-protocol/tests/cdn_bootstrap_tests.rs (11 tests) Covers TactClient querying BPSV endpoints via wiremock, error propagation (404, 503, invalid body), v1/products prefix rewriting, and the full two-step versions→cdns bootstrap workflow. Test fixtures added: - test_fixtures/blte/: 5 binary BLTE files (single/multi-chunk, none/zlib compression, known-content with pinned MD5) - test_fixtures/bpsv/: 3 BPSV text files matching real Blizzard CDN wire format (wow_classic_era, wow_retail with pinned build hashes) Also fixes a type mismatch in integration.rs where the public API passed Option<&TactKeyStore> to archive methods expecting Option<&dyn TactKeyProvider>. Fixed with zero-cost coercion at the two internal call sites. Workspace: 2069 tests passing.
- CDN override hosts are now prepended to Ribbit-advertised endpoints so local mirrors fall back to community and official CDN automatically. - Historical installs also prepend operator overrides before the community mirror and official CDN fallback list. - In-flight operations (initializing, downloading, verifying) are reset to queued on startup so interrupted installs resume after an agent restart. - Progress tracking sets phase = "downloading" when a file transfer begins. - README: add configuration reference, local CDN mirror guide, and fallback behavior documentation.
…ion tests
Five clippy violations across four test files were causing CI to fail
under RUSTFLAGS=-D warnings:
- cascette-import: replace push_str(&format!(...\n)) with writeln! to
fix format_push_string and write_with_newline
- cascette-formats, cascette-protocol: replace needless Vec collect just
for .len() with .count() directly on the iterator
- cascette-installation: remove redundant .clone() on a value immediately
moved into a struct; wrap constant comparison in const { assert!(..) }
to fix assertions_on_constants
Also update test_resolve_metadata_with_overrides in cascette-agent to
assert endpoints.len() >= 1 instead of == 1. The implementation prepends
overrides to Ribbit-advertised endpoints (changed in 089b95c) so the
total count is overrides + ribbit endpoints, not just overrides.
…ld handling Add five examples demonstrating cascette-rs as a replacement for common TACT/CASC reference tools, and fix silent bugs in existing examples where BPSV HEX:16 and DEC:4 fields were read with as_string(), returning None. New examples: - cascette-protocol: dump_product_info — replicates BuildBackup dumpinfo and TACTSharp --mode list; queries Ribbit then downloads and parses build config and CDN config - cascette-protocol: cdn_archive_sizes — replicates BuildBackup dumpsizes; downloads archive index files and reports entry counts and data sizes - cascette-client-storage: extract_by_content_key — replicates BuildBackup extractfilebycontenthash and TACTSharp --mode ckey/ekey; reads index entries and extracts file content from a local CASC installation - cascette-import: listfile_name_lookup — replicates BuildBackup calchashlistfile and wow.export listfile; demonstrates Jenkins96 hashing and ListfileProvider FDID<->name mapping Fixed examples: - wow_classic_native: use as_hex()+hex::encode() for BuildConfig/CDNConfig (HEX:16 typed), as_dec() for BuildId (DEC:4 typed) - wow_classic_wasm: same BPSV field type fixes Also fix clippy::len_zero in cascette-agent executor_tests (len() >= 1 → !is_empty()) and resolve all clippy warnings introduced by the new examples (map_unwrap_or, redundant_closure_for_method_calls, collapsible_if, expect_used, print_literal). All examples verified against live Ribbit/CDN endpoints and a local WoW Classic 1.13.2.31650 installation. All 7 quality gates pass.
…istfile_name_lookup
… example Rewrite the size manifest (DS magic) implementation with dedicated header, entry, builder, and error modules. The binary layout matches the wowdev.wiki format: 15-byte header, optional tag bitfields, 9-byte ekey + 4-byte esize entries, sorted descending by esize. Add build_file_tree example to cascette-protocol. Walks a full CASC build's file tree from build config and CDN config hashes. Supports both local mirror paths and online CDN URLs (auto-detected by http/https prefix). --paths mode emits one CDN path per tracked file for piping. Archive index scanning runs concurrently via futures::join_all. Fix broken intra-doc link in size/mod.rs and hex literal grouping in size header tests to satisfy all CI quality gates.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Signed-off-by: Ludea <ludovicw35@hotmail.com>
Contributor
Author
|
I rebase on feat/new-crates |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
Summary
use native cache for wasi target.
Type of Change
functionality to not work as expected)
Related Issues
Fixes #(issue number)
Testing
Tested On
WoW Versions Tested
Quality Checklist
By submitting this PR, I confirm that: