perf: faster downloads with skip/retry, fix OOM on large assets

[Me-Maped]

Summary

• Fix effective CDN concurrency. Serial await per chunk part left
  CHUNK_CONCURRENCY unused (effective ~1). A sliding prefetch window now
  keeps the bounded pool busy while assembling.
• Fix out-of-memory on large assets (e.g. Content Examples). Full unique-GUID
  prefetch + whole-file Uint8Array assembly held every decoded chunk and the
  full file in RAM. Now:
  • refcounted chunk cache — decoded payload dropped when no remaining part needs it
  • sliding prefetch window (default concurrency * 3), not all GUIDs up front
  • stream write to .partial + incremental SHA1 → atomic rename
• Skip existing files when on-disk size + SHA1 match the manifest (default on;
  force with --no-skip). Skip/plan pass runs before any CDN work.
• Retry + multi-CDN. Transient failures get exponential backoff; fall through
  remaining manifestPointers distribution bases.
• CLI: --concurrency <n> (1–64), --no-skip; step/status on stderr;
  stdout stays JSON-pipeable and includes skipped.

Changes

File	What
src/download.ts	Plan/skip → selective work; sliding-window prefetch; refcounted ChunkCache; stream assemble + hasher; multi-base fetch + retries; DownloadOptions (concurrency, skipExisting, retries, prefetchWindow); return skipped.
src/cli.ts	--concurrency, --no-skip, help; stderr status for resolve/manifest/download/sync; pass options; JSON skipped.

Behavior notes

• Default skip-on: re-download into the same tree skips matching files. Use --no-skip to force rewrite.
• Memory: peak is roughly "prefetch window of decoded chunks + current write I/O", not "all chunks + full file".
• Partial files: failed mid-file leaves no final path (.partial cleaned up on error); successful files rename atomically.
• Multi-CDN: best-effort via other distribution base URLs from the manifest response.

Commits

1. perf: fix download concurrency + skip/retry/progress
2. fix: download out of memory

Test plan

• bun run typecheck → passes
• epic-fab --help → shows --into, --concurrency, --no-skip
• epic-fab download <id> --into /tmp/fab-test → stderr: resolve → check/skip → download with concurrency/window → per-file progress; stdout JSON valid + skipped
• Re-run same dir → all skips / Nothing to fetch; no CDN chunk traffic
• Large asset (e.g. Content Examples) completes without out of memory
• Kill mid-download → no corrupt final file; re-run resumes via skip of completed files
• --concurrency 4 / 12 reflected in status line; --concurrency 0 → user error
• --no-skip forces redownload
• epic-fab sync --project <ue-project> → asset-level + file-level stderr status

Usage

epic-fab download <asset-id> --into ./out
epic-fab download <asset-id> --into ./out --concurrency 12
epic-fab download <asset-id> --into ./out --concurrency 4   # lower RAM
epic-fab download <asset-id> --into ./out --no-skip
epic-fab sync --project /path/to/Project --concurrency 12

[StarksLabs]

First — I'm sorry this sat for a month with no reply. I'm in school full time and
the repo went quiet on my end. You wrote a lot of careful code and got silence
back, and that's on me, not on the work.

The performance work is merged. Both commits, with your authorship:

• perf: fix download concurrency + skip/retry/progress → bc86f64
• fix: download out of memory → dea2d1e

The diagnosis behind those is the best thing that's happened to this tool. The
serial await per part meant CHUNK_CONCURRENCY was decorative and effective
concurrency was 1 — I never caught that. And the OOM analysis is exactly right:
prefetching every unique GUID and assembling into one Uint8Array meant peak
memory scaled with asset size, which is why Content Examples fell over. The
refcounted chunk cache, sliding prefetch window, and streaming .partial +
incremental SHA1 + atomic rename are all in. --concurrency, --no-skip, the
skip-on-matching-SHA1 pass, retries with backoff, multi-CDN fallthrough, and
stderr status all shipped.

The web UI I'm not taking, and I want to give you the real reason rather than
let it sit.

src/serve.ts calls:

Bun.serve({ port: opts.port, fetch: handler })

With no hostname, Bun binds 0.0.0.0 — every interface, not loopback. That
process holds Epic OAuth tokens at ~/.config/epic-fab/auth.json, and the router
has four state-changing POST routes with no Origin, Host, or CSRF checks:
/api/auth, /api/logout, /api/download, and /api/download/<jobId>/cancel,
alongside GET /api/library returning the full owned-asset list.

That's reachable two independent ways:

1. Anyone on the same network. curl http://<lan-ip>:<port>/api/library reads
   the user's Epic identity. A POST logs them out or starts downloads.
2. Any website the user visits. localhost isn't a boundary in a browser —
   with no CSRF token and no Origin check, a page can cross-origin POST to
   http://localhost:<port>/api/logout, and with no Host check DNS rebinding
   reads responses too.

Also worth noting independently of the network exposure: into on
POST /api/download is caller-controlled and unvalidated, so on a reachable port
that's an arbitrary filesystem write, which is a worse outcome than the token
read.

None of that is a knock on the UI itself — it's a Bun.serve default that has
bitten a lot of people, and it's invisible unless you go looking for it.

I've added SECURITY.md
with the four things a local UI would have to do here: explicit 127.0.0.1 bind,
Origin allowlist on every state-changing method (absent Origin = reject),
Host allowlist to stop rebinding, and an unguessable per-session token sent as a
header rather than a cookie. There's now a bun run guard check in CI that fails
the build if a listener lands in src/, so this can't get in by accident later.

If you want to come back at the UI against those four requirements I'd genuinely
look at it. Leaving this PR open for you rather than closing it — your call.

Thanks for the perf work, and sorry again for the wait.

[Me-Maped]

I'm glad to have made even a small contribution to the project, and I truly appreciate you taking the time—despite your busy schedule—to review the changes so carefully and provide such important, constructive feedback.

The Web UI portion was indeed my oversight. I'm a game developer, and I initially built the UI mainly for my own convenience; I did not have enough knowledge of web security. Thank you for identifying these serious issues.
I will address the requirements you outlined and rework the UI accordingly. Once it is ready, I will push the updated implementation to this PR for review.

Thank you again for such a thoughtful response. I learned a great deal from it.

[Me-Maped]

I’ve adjusted direction based on the security constraints from your review. Since this repository is primarily a CLI tool, opening a Web UI is not especially appropriate. A minimal interface that clearly presents asset information and supports downloads is sufficient, so I chose a TUI rather than continuing to rework the web portion.

The new implementation has no HTTP listener, does not use Bun.serve(), and exposes no browser-accessible local API. Filtering and download selection happen entirely in the terminal; external Fab pages are opened only through explicit user action.

Since the performance commits in this PR have already been merged, and the remaining Web UI changes have been superseded by the new implementation, I will close this PR and submit the TUI as a separate, focused PR.