Repository navigation
Replies: 5 comments 8 replies
|
Thanks a lot for using zxc, @jeanga, and for taking the time to write this up. Really glad to confirm what your numbers show: the v6/v5 -> v7 migration went through cleanly, no regressions, every archive smaller or equal. That’s exactly how it’s meant to behave. Honestly, this kind of report is the best thing a maintainer can get. Seeing zxc used on a real corpus and holding up reliably across 328 GB, is genuinely motivating. It’s one thing to pass the test suite, another to see the format do its job on real-world data. Thanks again for sharing it! |
|
I am happy to contribute back some data, you contribute the code ;-) |
|
@hellobertrand, out of curiosity: why did you close |
|
@solbjorn I need to rewrite some parts, but don't worry, this PR will be back, as I'm expecting a 10% increased decompression speed at level 7 on all architectures. |
|
The updated perf numbers for both level 6 and 7 look incredible. Pure genius. |
|
@solbjorn PR #336 has been merged. Decompression speed increases by 10% at level 7 (the ratio varies by -0.27%). I have to build the new 0.13.2 release. I’m not a genius; I simply implemented the new PivCo Huffman method (https://github.com/MarcinZukowski/pivco-huffman) by @MarcinZukowski (https://marcinzukowski.github.io/pivco-huffman/paper-1.0/ph.html). |
|
Happy it's being used, more good stuff to come. And @dougallj is the person that deserves the credit for all the "joint" work and the amazing results it produces. |
|
@hellobertrand, sorry for a bit of offtopic, but if you'd like to have a bit more meaningful feedback on PRs than just SonarCloud / CodeQL, I highly recommend CodeRabbit AI reviewer. It's totally free for Open Source projects (the only limitation is that you need to trigger it manually when you create / update a PR) and has really good knowledge base (it's backed by several LLMs, not only one, including Opus and Gemini IIRC). I'm not related to the CR team anyhow and they don't pay me for advertisement (:D), just sharing my experience -- you can take a look at the merged PRs in my main repo and how it performs reviews if you're interested. You can also take a look at And here I just wanted to thank you one more time: ZXC is probably the discover of the year for me, maybe together with DwarFS. You did already beat lz4hc and zstd and you don't stop improving and optimizing it 🔥 |
|
Thx @solbjorn, I'm glad you're using zxc and are happy with it. |
|
@hellobertrand, I wasn't able to trigger DeepSource AI reviews, although I configured it in the dashboard, dunno why. Gemini says it needs a CI in order to work, but I don't really buy this statement. So I don't have any data to compare, but I really like CodeRabbit. The rest works more like just static analysis in my repo. |
|
@solbjorn In the next few days, I'll add Codderabbit. That way I can compare it with SonarCloud. |
|
@hellobertrand, I've noticed CodeRabbit started reviewing your PRs. What are your impressions? |
|
@solbjorn I like it finally: I didn't expect such a precise and useful code review. I think I'll keep it ;-) Thanks for the tip. 🙏 |
Uh oh!
There was an error while loading. Please reload this page.
zxc Format v7 Migration — Compression Report
Generated 2026-07-20 · corpus: this Flashback-Test checkout · tool:
tools/migrate-zxc013.ps1(+ manual v5 straggler)Important
7.62 GB reclaimed (7.0% smaller) migrating 177 memory-dump archives from zxc format v6/v5 to v7 (level-7 "Ultra").
Every archive came out smaller or equal — no regressions, no dump left behind.
Summary
Files: 177 (176 format v6->7, 1 format v5->7). Uncompressed data represented: 328.8 GB.
xychart-beta title "Compressed size on disk (GB)" x-axis ["v6 / v5 (before)", "v7 (after)"] y-axis "GB" 0 --> 110 bar [108.55, 100.93]Per-file size reduction
Distribution (file count by reduction band):
The tight cluster at 6-10% is the signature of a uniform level-7 / format-v7 (Huffman-coded) upgrade applied across already-similar v6 archives.
Biggest absolute savings
WindowsServer2019_Large.dmp.zxcanalyse-memoire.dmp.zxcWindows10_22H2.dmp.zxcWindowsServer2025.dmp.zxcWindows11_25H2.dmp.zxcWindows11_24H2.dmp.zxcmemdump.mem.zxcmemdump.dmp.zxcDmaBackDoorBoot.dmp.zxcDmaBackDoorBoot_windbg.dmp.zxcNote
The v5 straggler (
Windows10_22H2.dmp.zxc) over-performs (-14%) because it started as an un-optimized 64K-block, no-dictionary v5 archive; standardizing it to 16K blocks + a trained dictionary + level-7 helped far more than on the already-tuned v6 files. It was migrated by hand with a purpose-built zxc v0.11.0 (the script's-LegacyZxcPathcan't drive v0.11.0, which lacks-o).Files with no gain
WindowsServer2025.sdmp.zxcScope & method
zxc -l -j)..zxcpresent on this checkout only — the corpus references ~441 dumps; the remainder live on other checkouts and are not included here.tools/migrate-zxc013.ps1; the 1 format-v5 straggler migrated manually. 0 failures; all 177 verifiedformat_version = 7and integrity-tested (-t).Full per-file table (sorted by reduction, best first)
Win7x64.raw.zxcWin7x64.dmp.zxcWindows10_22H2.dmp.zxcW10-1803_live.dmp.zxcW8.dmp.zxcW10-1809_live.dmp.zxcW10-1709_live.dmp.zxcW10-1703_live.dmp.zxcW10-1903_live.dmp.zxcWindowsServer2008R2.dmp.zxcXPSP2.dmp.zxcWindowsVistaSP2.dmp.zxcXP.raw.zxc[PH]Win10_2004_CreatorInfo.dmp.zxcWindowsVistaSP2.raw.zxcWindows7_x64_rtm.dmp.zxcdkom.dmp.zxcXPSP3.dmp.zxcWindows7_x64_rtm.raw.zxcXPSP2.raw.zxcW10-2004_live.dmp.zxcXP.dmp.zxcWindows7_x86_rtm.raw.zxcWindows7_x86_rtm.dmp.zxcdkom.mem.zxcXPSP3.raw.zxcconficker.img.zxcWindowsServer2008R2_SP1.dmp.zxcWindows7SP1.raw.zxcconficker.dmp.zxcW10-20H2_live.dmp.zxcW11-21H2_live.dmp.zxcXPSP3.raw.zxcWindows7SP1.dmp.zxcapihooks.dmp.zxcW2016_msf.dmp.zxcapihooks.mem.zxcwin10_2004_meterpreter[IWVA].dmp.zxcWindows10_1903.dmp.zxc[RDI]Win10_19041_VirtualProtect2.dmp.zxcwin7_32_sp1_before.dmp.zxcwin7_32_sp1_before.vmem.zxc[DI2]Win10_2004_19041.dmp.zxcW10-21H1_live.dmp.zxcTDSS.dmp.zxcWindowsVista.dmp.zxcWindowsVista.raw.zxcSQL2012R2_beacon.raw.zxcSQL2012R2_beacon.dmp.zxcWindows11_24H2.dmp.zxcW11_22H2.dmp.zxc[RDI]Win10_19041_VirtualProtect.dmp.zxcWindows10_1607_x64_BMP.dmp.zxcXPSP3.dmp.zxc[PH]Win10_2004_NoHollowing.dmp.zxc[RDIVP]notepad.exe_RWRWX.dmp.zxcWindows11_23H2.dmp.zxcTDSS.img.zxcwin7_32_sp1_apres_eternalblue_doublepulsar_peddlecheap_danderspritz.dmp.zxc[RDIVP]notepad.exe_RWRWX_threadclose.dmp.zxcWindows7SP1.dmp.zxc[RDI]Win10_2004_19041.dmp.zxcWindowsVistaRTM_x64.dmp.zxcW10-22H2.dmp.zxcWindowsServer_20H2_windbg.dmp.zxc[RDIVP]Rapport_Notepad.dmp.zxcW2012_R2.dmp.zxcW2016_network.dmp.zxcW2016_network_decoded.dmp.zxcWindowsServer2025.dmp.zxcW10_21H2.dmp.zxcanalyse-memoire.dmp.zxcSQL2012R2_beacon_http.raw.zxcWINSERVER2012R2-20201201-150520.dmp.zxcWINSERVER2012R2-20201201-150520.raw.zxcSQL2012R2_beacon_http.dmp.zxcwin7_32_sp1_apres_eternalblue_doublepulsar_peddlecheap_danderspritz.vmem.zxc[PH]Rapport.dmp.zxcSQL2012R2_beacon_smb.raw.zxcWindows7_irm_dmp.dmp.zxcSQL2012R2_beacon_smb.dmp.zxc[DI+RDI]_Win10_2004_creator_info.dmp.zxc[RDIVP]Win11.dmp.zxcWindowsServer_20H2.dmp.zxcWindows10_1909.dmp.zxcWindows10_1909_windbg.dmp.zxcWindows11_25H2.dmp.zxcWindowsServer2003.raw.zxcWindowsServer2003.dmp.zxcsample006.bin.zxcWindowsVista_x86_RTM.dmp.zxcWindows10RTMx86.raw.zxc[DI]Win10_2004_19041.dmp.zxcW2019_2.dmp.zxc[PH]Test.dmp.zxcW2012.dmp.zxcAPT.dmp.zxcWindows10x64_1607.dmp.zxcWindows10RTMx86.dmp.zxcWindows10x64_1607.raw.zxcWindowsServer2025_vm2dmp.dmp.zxcSQL2012R2-20200904-115013.raw.zxcclientside.mem.zxcWindows7SP1.raw.zxcProcessHollowing.raw.zxcProcessHollowing.dmp.zxcclientside.dmp.zxcW81SP1.dmp.zxcSQL2012R2-20200904-115013.dmp.zxc[DI]Win10_2004_creator_info.dmp.zxcXPSP3WithPageFile.dmp.zxcXPSP3WithPageFile.raw.zxc[RDI]Win10_19041_VirtualProtect3.dmp.zxcAPT.img.zxcWindows11_23H2_v2.dmp.zxc[IWVA]WinServ2012_R2.dmp.zxc[InjectionWithoutVA]Win10_2004.dmp.zxcWindowsServer2019_Large.dmp.zxc[PH]Windows81_U1.dmp.zxcWindowsServer_2004_windbg.dmp.zxcWindowsServer2019Busy.dmp.zxcWIN11_22H2-20230705-151303.dmp.zxcW81.dmp.zxcWindowsServer_2004.dmp.zxcW2012.irm-raw.zxcWindowsServer2022.dmp.zxcsliver.dmpce-raw.zxcWindows10_1903_vm2dmp.dmp.zxcsample007.dmp.zxcmemscan.dmp.zxcmemscan.mem.zxcsample007.bin.zxc[PH]IexplorerRedshot.dmp.zxcDmaBackDoorBoot.dmp.zxcrootkit.mem.zxcrootkit.dmp.zxcDmaBackDoorBoot_windbg.dmp.zxcWindows8.dmp.zxcsample004.raw.zxcWindowsVistaSP1_x64.dmp.zxcWindowsServer2008.dmp.zxc[Sain]Win10_2004.dmp.zxcsample004.dmp.zxcWindowsServer2012.dmp.zxcWindowsServer2012R2_0001.dmp.zxcmemdump.mem.zxcmemdump.dmp.zxcTests_Arthur_memory.dmp.zxcWindows10_1903_vm2dmp.dmp.zxc[PH]Win10_2004.dmp.zxcWindows10_1909.raw.zxcWindows10_1909.dmp.zxcpierce.lime.raw.zxcWindowsServer2008SP2PU.dmp.zxcWIN10_CS_stageless.dmp.zxcW2019_TH_2.windbg.dmp.zxcW2019_TH_2.dmp.zxcAPPS_2019_ADCS_dump4bugfix_full.dmpce-raw.zxcWindows10x64RTM.raw.zxcWindows10x64RTM.dmp.zxcW2016.dmp.zxcWindowsServer_1903.dmp.zxcWindowsServer_1903_windbg.dmp.zxcWindowsServer2016.dmp.zxc[PH]Win10_19041_Mimikatz.dmp.zxcWindows_11_24H2_vm2dmp.dmp.zxcWindowsServer2019.dmp.zxcWindowsServer_1803.dmp.zxcWindowsServer_1803_windbg.dmp.zxcWindows10x641511.raw.zxcWindows10x641511.dmp.zxcWin11_22H2_002.dmp.zxcWindowsServer_1909.dmp.zxcWindowsServer_1709.dmp.zxcWindowsServer_1909_windbg.dmp.zxcWindowsServer_1709_windbg.dmp.zxcWindowsServer2025.sdmp.zxcAll reactions