diff --git a/README.md b/README.md index 2c32e7b..6f27bc0 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # ResX -A ReScript framework for building server-driven web sites and applications. Use familiar tech like JSX and the component model from React, combined with simple server driven client side technologies like HTMX. Built on Bun and Vite. +A ReScript framework for building server-driven web sites and applications. Use familiar tech like JSX and the component model from React, combined with simple server driven client side technologies like HTMX. Built on Bun with a Bun-only asset pipeline. ResX is suitable for building everything from blogs to complex web applications. @@ -20,7 +20,7 @@ The `demo/` will contain a comprehensive example of using ResX. First, make sure you have [`Bun`](https://bun.sh) installed and setup. Then, install `rescript-x` and the dependencies needed: ```bash -npm i rescript-x vite @rescript/core rescript-bun +npm i rescript-x @rescript/core rescript-bun ``` Note that ResX requires these versions: @@ -52,21 +52,7 @@ Go ahead and install the dependencies for Tailwind as well if you want to use it npm i autoprefixer postcss tailwindcss ``` -Let's set everything up. Start by setting up `vite.config.js`: - -```javascript -import { defineConfig } from "vite"; -import { resXVitePlugin } from "rescript-x"; - -export default defineConfig({ - plugins: [resXVitePlugin()], - server: { - port: 9000, - }, -}); -``` - -Make sure you have both folders for static assets set up: `assets` and `public` in the root, next to `vite.config.js`. More on static assets later. +Make sure you have both folders for static assets set up: `assets` and `public` in the project root. More on static assets later. If you're using Tailwind, add `tailwind.config.js` and `postcss.config.js` as well: @@ -95,13 +81,12 @@ There! If you want, you can also set up a bunch of scripts in `package.json` tha { "scripts": { "start": "NODE_ENV=production bun run src/App.js", - "build": "NODE_ENV=production && bun run build:vite && bun run build:res", - "build:vite": "vite build", + "build": "NODE_ENV=production resx assets build --root . --clean && bun run build:res", "build:res": "rescript", "clean:res": "rescript clean", "dev:res": "rescript build -w", "dev:server": "bun --watch run src/App.js", - "dev:vite": "vite", + "dev:assets": "resx assets build --dev --watch --root . --clean", "dev": "concurrently 'bun:dev:*'" } } @@ -215,7 +200,7 @@ switch path { ## Static assets -ResX comes with full static asset (fonts, images, etc) handling via Vite, that you can use if you want. In order to actually serve the static assets, make sure you use `ResX.BunUtils.serveStaticFile` before trying to handle your request in another way: +ResX comes with full static asset (fonts, images, etc) handling via Bun tooling. In order to actually serve the static assets, make sure you use `ResX.BunUtils.serveStaticFile` before trying to handle your request in another way: ```rescript fetch: async (request, server) => { @@ -234,7 +219,7 @@ As for the assets themselves, there are two ways of handling them in ResX: ### `public` for assets that don't need transformation -Putting assets in the `public` directory. Any assets you put in the top level `public` directory next to `vite.config.js` will be copied as-is to your production environment. It's then available to you via the top level: +Putting assets in the `public` directory. Any assets you put in the top level `public` directory will be copied as-is to your production environment. It's then available to you via the top level: ``` // public/robots.txt exists @@ -243,7 +228,7 @@ GET /robots.txt ### `assets` for assets that do need transformation -If you have assets you'd like transformed by Vite before using, put them in the top level `assets` folder. This could be CSS, images, additional JavaScript, and so on. Anything you might want Vite to transform. +If you have assets you'd like transformed before using, put them in the top level `assets` folder. This could be CSS, images, additional JavaScript, and so on. Here's an example of how you wire up Tailwind: @@ -262,13 +247,13 @@ Then, include it in your ReScript: ``` -There! It's now available to you, and Vite will both transform and hot module reload the asset if it's possible. +There! It's now available to you through the managed asset pipeline. #### Referring to transformed `assets` Notice how we're not using a `"/assets/styles.css"` string to refer to `styles.css`, but rather `ResXAssets.assets.styles_css`? This is because ResX comes with a "type safe" asset layer - anything you put in `assets/` will be available via `ResXAssets.assets`. -**Always use this to refer to assets**. One for the type safety of course, but also because this is how Vite keeps track of all asset files, so you get the transformed asset in production, and so on. +**Always use this to refer to assets**. This keeps references type-safe and ensures you get the correct transformed file in production. > Bonus: Since the asset map is a regular ReScript record, you'll automatically get dead code analysis via the ReScript code analyzer. Dead code analysis for your assets! Makes it really easy to keep your assets folder clean. @@ -999,14 +984,13 @@ render: async ({path}) => { - `FormDataHelpers` - `FormData` -### Vite plugin - -ResX comes with its own Vite plugin that takes care of all configuration for you. It will: +### Asset CLI -- Ensure all ResX assets are handled and included properly -- Ensure that Hot Module Reloading works for all assets and that Vite dev mode is properly wired up to your local ResX dev server +ResX ships a Bun CLI for assets: -> Note: Right now, using ResX with more elaborate Vite config than what's preconfigured for you might be problematic. This will change in the future though so that ResX is just another part of your Vite config. Open issues please when you find use cases you'd like supported but that doesn't work now. +- `resx assets build --dev` builds to `.resx/dev` with stable URLs +- `resx assets build` builds to `dist` with hashed filenames +- `resx assets build --dev --watch` watches `assets/`, `public/`, and `src/ResXClient.*` ## Static site generation diff --git a/bin/resx b/bin/resx new file mode 100755 index 0000000..9a9542b --- /dev/null +++ b/bin/resx @@ -0,0 +1,4 @@ +#!/usr/bin/env bun +'use strict'; + +require('../src/Cli.js').runFromArgv(); diff --git a/demo/package.json b/demo/package.json index 44e660a..eb4e1c7 100644 --- a/demo/package.json +++ b/demo/package.json @@ -3,13 +3,12 @@ "version": "0.0.0", "scripts": { "start": "NODE_ENV=production bun run src/Demo.js", - "build": "NODE_ENV=production && bun run build:vite && bun run build:res", - "build:vite": "vite build", + "build": "NODE_ENV=production resx assets build --root . --clean && bun run build:res", "build:res": "rescript", "clean:res": "rescript clean", "dev:res": "rescript build -w", "dev:server": "bun --watch run src/Demo.js", - "dev:vite": "vite", + "dev:assets": "resx assets build --dev --watch --root . --clean", "dev": "concurrently 'bun:dev:*'" }, "keywords": [ @@ -23,7 +22,6 @@ "fast-glob": "^3.3.1", "rescript-x": "../", "rescript": "11.1.4", - "vite": "^4.4.11", "rescript-bun": "0.5.0" }, "devDependencies": { diff --git a/docs/bun-migration-plan.md b/docs/bun-migration-plan.md new file mode 100644 index 0000000..8f4407b --- /dev/null +++ b/docs/bun-migration-plan.md @@ -0,0 +1,577 @@ +# Bun-only Asset Pipeline Plan (v1 Usable Spec) + +## Goals +- Remove Vite entirely (no Vite dev server, no Vite plugin, no Vite dependency). +- Replace the asset pipeline with Bun-only tooling. +- Keep `assets/` + `public/` conventions and keep `ResXAssets.assets.` API. +- Transform assets automatically (CSS/JS via bundler; other files copied) and hash in prod. +- Support implicit JS entrypoints from `assets/`. +- Prebundle `ResXClient` and include it as `resXClient_js` in the asset map. +- Provide a CLI for builds and a runtime API to start dev processing inside the Bun app. + +## Non-goals (v1) +- HMR (CSS or HTML morphing). +- CDN/publicPath/basePath configuration. +- Embedding assets into the Bun single-binary output (tracked for later). +- Advanced user configuration (keep zero-config first). + +## Locked Decisions (Clarified) +- Dev pipeline starts explicitly in app code (no separate dev server process). +- Dev output goes to `.resx/dev/` with stable, unhashed URLs. +- Prod output goes to `dist/`, assets under `dist/assets/`. +- `public/` is copied as-is and is not part of `ResXAssets`. +- Implicit JS entrypoints: any `assets/**/*.{js,jsx,ts,tsx,mjs,cjs}`. +- CSS entrypoints: any `assets/**/*.css`. +- Single-file output per entry (no code splitting in v1). +- `ResX.Dev` becomes a no-op in v1. + +## Root + Path Resolution +- Default project root is current working directory. +- CLI supports `--root ` override. +- Runtime API supports `~root` override. +- `assets/` and `public/` are optional; missing directories are treated as empty. +- Generated files location is fixed to `src/__generated__/` in v1. +- Symlink policy for discovery/copying: do not follow symlink targets that resolve outside project root. + +## Artifact Contracts +### Asset key derivation +- Normalize discovered file paths to `/` separators before key generation. +- Keep current `toRescriptFieldName` behavior for compatibility. +- Process files in sorted order for deterministic collision resolution. +- Collision rule remains suffix `_` repeatedly until unique. + +### Manifest schema +- Emit JSON manifest in both dev and prod: + +```json +{ + "version": 1, + "mode": "dev", + "assets": { + "styles_css": "/assets/styles.css", + "resXClient_js": "/assets/resx-client.js" + } +} +``` + +- `version` is required and starts at `1`. +- `mode` is `dev` or `prod`. +- `assets` keys match generated ReScript fields. + +### Generated files +- Always generate `src/__generated__/ResXAssets.res`. +- Always generate `src/__generated__/res-x-assets.js`. +- Always include `resXClient_js` in both files. + +### Hashing +- Prod hashes are content-based. +- Algorithm: `sha256`, truncated to 8 hex chars. +- Filename format: `-.`. + +## Runtime Contracts +### `ResX.Assets.startDev()` +- Async, idempotent API. +- First call builds assets into `.resx/dev/`, writes `.resx/dev/resx-assets.json`, regenerates `src/__generated__/ResXAssets.res` and `src/__generated__/res-x-assets.js`, and starts file watching. +- Subsequent calls in same process are no-ops. +- Uses write-if-changed for generated files to reduce rebuild churn. +- Watches at least: `assets/`, `public/`, `src/ResXClient.*`. + +### `ResX.BunUtils.serveStaticFile` +- Dev behavior: serve `/assets/*` from `.resx/dev/assets/` and serve non-assets static files from `public/`. +- Prod behavior: serve `/assets/*` from `dist/assets/` and serve non-assets static files from `dist/`. +- `/assets/*` is reserved for managed pipeline assets; `public/assets/*` must not override it. +- Must reject traversal attempts (`..`) after URL decoding/normalization. + +## CLI Contract (`resx`) +### v1 command +- `resx assets build` + +### v1 flags +- `--dev` builds to `.resx/dev/` instead of `dist/`. +- `--watch` watches for changes (dev mode only). +- `--root ` overrides project root. +- `--clean` removes output directory before build (enabled by default). + +### Exit codes +- `0`: success +- `1`: runtime/build failure +- `2`: invalid CLI usage + +## Agent Handoff (Start Here) +### Current status snapshot +- Date: February 13, 2026. +- Planning/spec and verification design are in place. +- Verification fixture exists at `test/fixtures/bun-assets-smoke/`. +- Automated verification runner exists at `test/fixtures/bun-assets-smoke/scripts/verify.mjs`. +- Root convenience script exists: `bun run verify:bun-assets-plan`. +- Bun migration implementation is not complete yet; many checks are expected to fail until steps 2-10 are implemented. + +### First commands for a new agent +- `bun run verify:bun-assets-plan -- --only V3` +- `bun run verify:bun-assets-plan -- --only V2 --allow-missing-cli` +- `bun run verify:bun-assets-plan -- --only V1` + +### Recommended execution loop +1. Pick the next incomplete step in `Step-by-step Plan`. +2. Implement only that step and keep scope tight. +3. Run the step-relevant `V*` checks from this doc. +4. Fix regressions before starting the next step. +5. Re-run broader checks periodically (`V1-V20`) to catch cross-step breakage. +6. Run the full verification suite before final handoff. + +### Step to verification mapping (minimum) +- Step 2: run `V4`, `V5`, `V13`, `V18`. +- Step 3: run `V4`, `V8`, `V20`, `V26`. +- Step 4: run `V4`, `V6`, `V8`, `V31`. +- Step 5: run `V4`, `V9`, `V24`, `V27`. +- Step 6: run `V10`, `V11`, `V12`, `V16`, `V18`. +- Step 7: run `V7`, `V14`, `V25`. +- Step 8: run `V17`, `V29`, `V30`. +- Step 9: run `V2`, `V19`, `V34`. +- Step 10: run `V1`, `V32`, `V33`. + +### Final handoff checklist +- `bun run verify:bun-assets-plan` passes for all agent-verifiable checks (`V1-V34`). +- `U1` and `U2` are explicitly called out for user-assisted follow-up. +- `rg -n "vite|@vite/client|res-x-vite-plugin" -S . --glob '!CHANGELOG.md'` has no runtime/doc hits. +- Plan doc is updated if any contract changed during implementation. + +## Test Project for Verification +Create and keep a dedicated fixture project in this repo: +- Path: `test/fixtures/bun-assets-smoke/` +- Purpose: stable end-to-end target for agent-run verification of dev/prod build behavior. + +### Fixture contents (minimum) +- `assets/` +- `styles.css` (Tailwind directives) +- `main.ts` (JS entrypoint) +- `images/logo.png` (copied asset) +- `misc/data.txt` (copied asset) +- `public/` +- `robots.txt` +- `favicon.ico` (or another static file) +- `src/` +- minimal Bun server app using `ResX.BunUtils.serveStaticFile` +- app startup calls `await ResX.Assets.startDev()` in dev +- sample view uses `ResXAssets.assets.styles_css` and `ResXAssets.assets.resXClient_js` +- `postcss.config.js` + `tailwind.config.js` +- `rescript.json` and `package.json` scripts for `dev`, `build`, and `start` + +### Fixture conventions +- Use fixed port for test server (for example `4460`) to make scripted checks deterministic. +- Keep fixture minimal and deterministic; do not add unrelated app logic. +- Keep fixture committed so agent can run checks without setup prompts. + +## Target Developer Flow +### Dev +1. Run `rescript build -w`. +2. Run app with Bun (`bun --watch run src/App.js`). +3. In app startup, call `await ResX.Assets.startDev()`. +4. Use `ResXAssets.assets.*` in views and refresh browser manually. + +### Production +1. Run `resx assets build`. +2. Run Bun server with `NODE_ENV=production`. +3. `ResX.BunUtils.serveStaticFile` serves from `dist/`. + +## Architecture +### Asset discovery +- Scan `assets/**/*` and `public/**/*` using `Bun.Glob` or `fast-glob`. +- Partition rules: +- JS entrypoints are `.js/.jsx/.ts/.tsx/.mjs/.cjs`. +- CSS entrypoints are `.css`. +- Other assets (images/fonts/wasm/txt/etc) are copied. + +### Build pipeline (shared core for dev/prod) +- JS entrypoints: `bun build` with no code splitting. +- CSS entrypoints use `bun build` when sufficient; otherwise fallback to PostCSS via local `postcss.config.js`. +- Other assets: copy to output; hash names in prod. +- Add prebundled `resx-client.js` as additional asset entry (copied/mapped). +- Emit manifest (`.resx/dev/resx-assets.json` in dev, `dist/resx-assets.json` in prod). +- Regenerate `src/__generated__/ResXAssets.res` + `src/__generated__/res-x-assets.js`. + +### Cleanup behavior +- Dev builds clean `.resx/dev/` before writing outputs. +- Prod builds clean `dist/` before writing outputs. +- This guarantees deleted or renamed assets do not leave stale artifacts. + +## Step-by-step Plan (Each Step Independently Verifiable) +### Step 0: Build verification fixture project +- Add `test/fixtures/bun-assets-smoke/` with files listed above. +- Add scripts intended for agent-run local verification. + +### Step 1: Capability validation +- Verify Bun JS/CSS build behavior against fixture assets. +- Verify PostCSS fallback path works with fixture Tailwind setup. +- Verify no-code-splitting output for multi-entry JS. + +### Step 2: Core asset discovery + manifest/codegen (no bundling yet) +- Implement discovery for `assets/` + `public/`. +- Generate manifest object and both generated files using dev-style URLs. +- Ensure deterministic ordering and collision handling. + +### Step 3: JS bundling (dev output) +- Build JS entries into `.resx/dev/assets/`. +- Wire output URLs into manifest and generated files. + +### Step 4: CSS pipeline (dev output) +- Add CSS entry handling into `.resx/dev/assets/`. +- Use Bun path first, PostCSS fallback second. + +### Step 5: Copy non-entry assets (dev output) +- Copy non-entry files into `.resx/dev/assets/` with stable names. + +### Step 6: Production output + hashing +- Build prod outputs in `dist/assets/` with hashed names. +- Copy `public/` into `dist/`. +- Emit `dist/resx-assets.json`. + +### Step 7: Prebundle `ResXClient` +- Build and include `resXClient_js` in both dev/prod manifests and generated files. + +### Step 8: Runtime integration +- Implement `ResX.Assets.startDev()` (build + watch + regenerate). +- Make `ResX.Dev` a no-op. +- Update `ResX.BunUtils.serveStaticFile` for new roots. + +### Step 9: CLI integration +- Add `resx assets build` and flags listed above. +- Provide help output and stable exit codes. + +### Step 10: Migration cleanup +- Remove `res-x-vite-plugin.mjs`. +- Drop `vite` peer/deps and Vite scripts. +- Remove Vite references from README/demo code/docs. + +## Verification Plan +All checks below are designed to be executable by the agent with shell tools and local process control. + +### Automation-first policy +- Prefer executable tests/scripts over manual spot checks for every `V*` item. +- If a check can be automated, it should be implemented as: +- a Bun test in `test/`, or +- a deterministic verification script under `test/fixtures/bun-assets-smoke/scripts/`. +- Manual verification should only be used when there is no reliable automation path. +- CI should run the automated verification suite for regression prevention. + +### A. Environment and migration checks +#### V1: No Vite runtime references remain +- Command: +- `rg -n "vite|@vite/client|res-x-vite-plugin" -S . --glob '!CHANGELOG.md'` +- Pass: +- No runtime docs/code references remain (historical changelog entries allowed). +- Verifiable by agent: Yes + +#### V2: CLI command exists +- Command: +- `bun x resx --help` +- `bun x resx assets build --help` +- Pass: +- Help output includes `assets build` and expected flags. +- Verifiable by agent: Yes + +### B. Fixture bootstrap checks +#### V3: Fixture is present and complete +- Commands: +- `test -d test/fixtures/bun-assets-smoke` +- `test -f test/fixtures/bun-assets-smoke/assets/styles.css` +- `test -f test/fixtures/bun-assets-smoke/public/robots.txt` +- `test -f test/fixtures/bun-assets-smoke/src/AppView.res` +- Pass: +- All required fixture files exist. +- Verifiable by agent: Yes + +### C. Dev build checks +#### V4: Dev build emits expected roots +- Command: +- `bun x resx assets build --dev --root test/fixtures/bun-assets-smoke --clean` +- Pass: +- `.resx/dev/assets/` and `.resx/dev/resx-assets.json` exist under fixture root. +- Verifiable by agent: Yes + +#### V5: Dev generated files are emitted +- Commands: +- `test -f test/fixtures/bun-assets-smoke/src/__generated__/ResXAssets.res` +- `test -f test/fixtures/bun-assets-smoke/src/__generated__/res-x-assets.js` +- Pass: +- Both generated files exist. +- Verifiable by agent: Yes + +#### V6: Dev URLs are stable/unhashed +- Command: +- `rg -n "\"/assets/.*-[0-9a-f]{8}\\." test/fixtures/bun-assets-smoke/.resx/dev/resx-assets.json` +- Pass: +- No hashed asset URLs in dev manifest. +- Verifiable by agent: Yes + +#### V7: `resXClient_js` is always present +- Commands: +- `rg -n "\"resXClient_js\"" test/fixtures/bun-assets-smoke/.resx/dev/resx-assets.json` +- `rg -n "resXClient_js" test/fixtures/bun-assets-smoke/src/__generated__/ResXAssets.res` +- Pass: +- `resXClient_js` exists in manifest and ReScript type. +- Verifiable by agent: Yes + +#### V8: Watch mode updates changed files +- Script-style commands: +- Start watcher in background: +- `bun x resx assets build --dev --watch --root test/fixtures/bun-assets-smoke > /tmp/resx-watch.log 2>&1 &` +- Modify `assets/styles.css`. +- Wait briefly and assert `.resx/dev/assets/styles.css` changed. +- Kill watcher process. +- Pass: +- Updated source is reflected in dev output. +- Verifiable by agent: Yes + +#### V9: Deletion/rename does not leave stale files +- Script-style commands: +- Add temporary asset, run dev build, assert output exists. +- Remove/rename temporary asset, run dev build, assert old output is removed. +- Pass: +- No stale files from deleted/renamed assets. +- Verifiable by agent: Yes + +### D. Production build checks +#### V10: Prod build emits hashed assets +- Command: +- `bun x resx assets build --root test/fixtures/bun-assets-smoke --clean` +- Pass: +- `dist/assets/*` includes `-<8hex>` hashed filenames. +- Verifiable by agent: Yes + +#### V11: Public files copied unchanged +- Commands: +- `test -f test/fixtures/bun-assets-smoke/dist/robots.txt` +- Compare source and output hashes for a public file. +- Pass: +- Public files exist in `dist/` and bytes match source. +- Verifiable by agent: Yes + +#### V12: Prod manifest schema is correct +- Commands: +- Parse `dist/resx-assets.json` and assert: +- `version === 1` +- `mode === "prod"` +- `assets` is an object +- Pass: +- Manifest fields and types match contract. +- Verifiable by agent: Yes + +#### V13: Generated ReScript fields match manifest keys +- Commands: +- Extract manifest keys. +- Extract field names from `ResXAssets.res`. +- Compare sets. +- Pass: +- No missing or extra keys (except type-level ordering differences). +- Verifiable by agent: Yes + +### E. Runtime serving checks +#### V14: Dev server serves assets from `.resx/dev` +- Commands: +- Start fixture app with `NODE_ENV=development`. +- `curl` `http://localhost:4460/assets/`. +- Pass: +- Response is `200` and bytes match `.resx/dev/assets` output. +- Verifiable by agent: Yes + +#### V15: Dev server serves `public/` from top-level URLs +- Command: +- `curl -i http://localhost:4460/robots.txt` +- Pass: +- `200` with expected content from fixture `public/robots.txt`. +- Verifiable by agent: Yes + +#### V16: Prod server serves from `dist/` +- Commands: +- Run prod build. +- Start fixture app with `NODE_ENV=production`. +- Read one hashed URL from `dist/resx-assets.json`. +- `curl` that URL from server. +- Pass: +- `200` and content matches file in `dist/assets`. +- Verifiable by agent: Yes + +#### V17: Path traversal is blocked +- Commands: +- `curl -i "http://localhost:4460/assets/../src/App.js"` +- `curl -i "http://localhost:4460/%2e%2e/%2e%2e/etc/passwd"` +- Pass: +- Requests are denied (`404` or `403`) and do not return file contents. +- Verifiable by agent: Yes + +### F. Determinism and failure checks +#### V18: Deterministic manifest and generated outputs +- Commands: +- Run same build twice without source changes. +- Compare checksums of: +- `resx-assets.json` +- `ResXAssets.res` +- `res-x-assets.js` +- Pass: +- Checksums are identical between runs. +- Verifiable by agent: Yes + +#### V19: Invalid CLI usage returns code 2 +- Command: +- `bun x resx assets build --watch` (without `--dev`) or another invalid combination. +- Pass: +- Command fails with exit code `2` and actionable error message. +- Verifiable by agent: Yes + +#### V20: Build failures return code 1 +- Commands: +- Introduce a temporary syntax error in fixture asset entry. +- Run build command. +- Pass: +- Command exits with code `1` and reports the failing file. +- Verifiable by agent: Yes + +### G. Additional agent-verifiable checks +#### V21: Empty-project build succeeds +- Script-style commands: +- Copy fixture to a temp directory. +- Remove both `assets/` and `public/`. +- Run `bun x resx assets build --dev --root --clean`. +- Pass: +- Build succeeds and still emits manifest + generated files (with at least `resXClient_js`). +- Verifiable by agent: Yes + +#### V22: Missing-directory matrix succeeds +- Script-style commands: +- Case 1: `assets/` present, `public/` missing. +- Case 2: `public/` present, `assets/` missing. +- Case 3: both missing. +- Run build for each case. +- Pass: +- All cases succeed and outputs/manifest reflect only available inputs. +- Verifiable by agent: Yes + +#### V23: Field-name collision handling is deterministic +- Script-style commands: +- Add files that normalize to same field name (for example `a-b.css` and `a_b.css`). +- Build twice from clean state. +- Compare generated key mapping across runs. +- Pass: +- Collision suffixing is stable and identical across runs. +- Verifiable by agent: Yes + +#### V24: `/assets/*` conflict policy is enforced +- Script-style commands: +- Add `public/assets/conflict.txt` and `assets/conflict.txt`. +- Build and run server. +- Request `/assets/conflict.txt`. +- Pass: +- Response matches managed asset output, not `public/` override. +- Verifiable by agent: Yes + +#### V25: `startDev()` idempotency +- Script-style commands: +- Run fixture app variant that calls `ResX.Assets.startDev()` twice. +- Modify one watched asset once. +- Assert only one rebuild cycle occurs for that change (via deterministic counter/log assertion in test harness). +- Pass: +- Multiple `startDev()` calls do not create duplicate watchers/rebuilds. +- Verifiable by agent: Yes + +#### V26: Watch burst stability +- Script-style commands: +- Start dev watch. +- Perform rapid consecutive edits to one CSS and one JS entry. +- Wait for quiescence and inspect outputs + manifest. +- Pass: +- Final outputs and manifest reflect the final source state, with no corruption. +- Verifiable by agent: Yes + +#### V27: `--clean` safety boundaries +- Script-style commands: +- Create sentinel files outside output dirs and inside output dirs. +- Run builds with `--clean`. +- Pass: +- Only `.resx/dev/` or `dist/` contents are removed; external sentinels remain untouched. +- Verifiable by agent: Yes + +#### V28: `--root` path edge cases +- Script-style commands: +- Run builds with: +- relative root path, +- absolute root path with spaces, +- symlinked root path. +- Pass: +- Outputs are written under resolved root correctly in all cases. +- Verifiable by agent: Yes + +#### V29: Traversal hardening for encoded and separator variants +- Commands: +- `curl -i "http://localhost:4460/%252e%252e/%252e%252e/etc/passwd"` +- `curl -i "http://localhost:4460/assets/..%2f..%2fsrc%2fApp.js"` +- `curl -i "http://localhost:4460/assets/..\\..\\src\\App.js"` +- Pass: +- Requests are denied and no sensitive file contents are returned. +- Verifiable by agent: Yes + +#### V30: Symlink escape policy is enforced +- Script-style commands: +- Add symlink under `assets/` or `public/` that points outside root. +- Build and run server. +- Pass: +- External target is not copied/served; warning or error is emitted consistently. +- Verifiable by agent: Yes + +#### V31: Content-Type headers are correct in dev and prod +- Commands: +- `curl -I` for one CSS, one JS, one image, one text file in dev and prod. +- Pass: +- Responses include expected MIME types for each file kind. +- Verifiable by agent: Yes + +#### V32: Package contents are correct +- Commands: +- Run `npm pack --json`. +- Inspect tarball file list. +- Pass: +- Includes new CLI/runtime files; excludes removed Vite/plugin artifacts. +- Verifiable by agent: Yes + +#### V33: Fresh-install smoke test from packed tarball +- Script-style commands: +- Create temp project. +- Install packed tarball. +- Run `resx assets build` against minimal fixture-like project. +- Pass: +- Command runs successfully in clean install context. +- Verifiable by agent: Yes + +#### V34: Error message ergonomics +- Script-style commands: +- Trigger representative failure modes (invalid CLI args, missing root, syntax error). +- Capture stderr. +- Pass: +- Messages include actionable reason and relevant file/path. +- Verifiable by agent: Yes + +### H. User-assisted checks (only where agent verification is limited) +#### U1: Browser-level manual refresh behavior in dev +- Why user help may be needed: +- Agent can validate file/build/HTTP behavior, but not subjective browser UX quality. +- Expected: +- Editing an asset and refreshing the browser loads updated content with no HMR dependency. + +#### U2: Cross-platform behavior outside current environment (if needed) +- Why user help may be needed: +- Agent verifies in current environment; Windows-specific path semantics may still need separate run. +- Expected: +- Same verification suite passes on target deployment OSes. + +## Risks and mitigations +- Bun CSS entrypoint support unclear: keep PostCSS fallback path in v1. +- Hash naming support unclear: perform explicit post-build content hashing. +- Asset map generation timing vs ReScript compile: provide explicit bootstrap command (`resx assets build --dev`). +- Watch-mode churn risk: use write-if-changed and idempotent watcher startup. + +## Acceptance criteria +- No Vite dependency or Vite-specific docs/code in runtime path. +- `resx assets build` produces `dist/` with hashed assets and a correct `ResXAssets` map. +- `ResX.Assets.startDev()` works inside app runtime and keeps assets current in dev. +- Verification fixture project exists and the verification suite above passes. +- Demo runs with Bun-only tooling. diff --git a/package.json b/package.json index 34c7ca4..e616fb6 100644 --- a/package.json +++ b/package.json @@ -1,11 +1,15 @@ { "name": "rescript-x", "version": "1.2.1", + "bin": { + "resx": "bin/resx" + }, "scripts": { "res:build": "rescript", "res:clean": "rescript clean", "res:dev": "rescript build -w", - "test": "bun test ./test/*.test.js" + "test": "bun test ./test/*.test.js", + "verify:bun-assets-plan": "RESX_CMD='bun run ./bin/resx' bun run test/fixtures/bun-assets-smoke/scripts/verify.mjs" }, "keywords": [ "rescript" @@ -15,21 +19,17 @@ "CHANGELOG.md", "rescript.json", "src/**/*", - "res-x-vite-plugin.mjs" + "bin/**/*" ], "author": "Gabriel Nordeborn", "license": "MIT", "peerDependencies": { "rescript": ">=12.0.0-0 <13.0.0", - "vite": ">=4.4.11", "rescript-bun": ">=2.1.0" }, "devDependencies": { - "fast-glob": "^3.3.1", "rescript": "12.0.0-rc.3", "rescript-bun": "2.1.0" }, - "dependencies": { - "fast-glob": "^3.3.1" - } + "dependencies": {} } diff --git a/res-x-vite-plugin.mjs b/res-x-vite-plugin.mjs deleted file mode 100644 index 9a5dc1b..0000000 --- a/res-x-vite-plugin.mjs +++ /dev/null @@ -1,309 +0,0 @@ -import path from "path"; -import fs from "fs"; -import fg from "fast-glob"; -import chokidar from "chokidar"; - -export default function resXVitePlugin(options = {}) { - const { - generated = "src/__generated__", - serverUri = "http://localhost:4444", - resXClientLocation = "node_modules/rescript-x/src/ResXClient.js", - } = options; - - const assetDir = "assets"; - let outDir = "dist"; - let isBuild = false; - let watcher; - let indexContent = ""; - const virtualIndexId = "@res-x-index"; - - function regenerate(context) { - indexContent = getDummyFile(assetDir, resXClientLocation); - writeResTypeFile(assetDir, generated); - - if (!isBuild) { - const dummyDevFile = getDummyDevFile(assetDir, resXClientLocation); - writeIfChanged(getAssetJsFileLoc(generated), dummyDevFile); - const content = getAssetDirContent(assetDir); - - // TODO: Check if really needed - content.forEach((c) => { - context.load({ - id: path.resolve(c), - }); - }); - } - } - - return { - name: "res-x-vite-plugin", - - configResolved(config) { - options.outDir = config.build.outDir || outDir; - outDir = options.outDir; - - isBuild = config.command === "build"; - - config.css = config.css || {}; - config.css.codeSplit = false; - config.css.extract = true; - - config.build.assetsInlineLimit = 0; - config.build.rollupOptions.input = virtualIndexId; - - config.build.rollupOptions.preserveEntrySignatures = "strict"; - - config.build.rollupOptions.output = - config.build.rollupOptions.output || {}; - - config.build.rollupOptions.output.preserveModules = true; - - if (config.command === "serve") { - config.server = config.server || {}; - config.server.proxy = config.server.proxy || {}; - const proxy = config.server.proxy; - if (proxy["/"] != null) { - console.warn("[WARN] Path `/` is already proxied. Skipping."); - } else { - proxy["/"] = { - target: serverUri, - changeOrigin: true, - bypass: (req) => { - if ( - req.url?.startsWith("/assets/") || - req.url?.includes("@vite/client") || - req.url?.includes("node_modules/") - ) { - return req.url; - } - return null; - }, - }; - } - } - }, - - resolveId(id) { - if (id === virtualIndexId) { - return virtualIndexId; - } - }, - - load(id) { - if (id === virtualIndexId) { - return indexContent; - } - }, - - buildStart() { - regenerate(this); - - if (!isBuild) { - // Watch the asset directory for changes in development mode - watcher = chokidar.watch(assetDir, { - persistent: true, - ignoreInitial: true, - }); - - watcher.on("add", (_) => { - regenerate(this); - }); - - watcher.on("unlink", (_) => { - regenerate(this); - }); - } - }, - - buildEnd() { - if (watcher) { - watcher.close(); - } - }, - - closeBundle() { - if (watcher) { - watcher.close(); - } - }, - - generateBundle(_options, bundle) { - const map = {}; - - const content = getAssetDirContent(assetDir); - const contentResolved = content.map((c) => path.resolve(assetDir, c)); - contentResolved.push(fs.realpathSync(resXClientLocation)); - - Object.entries(bundle).forEach(([key, v]) => { - if (v.facadeModuleId != null) { - const moduleId = v.facadeModuleId.split("?")[0]; - const path = key; - const importedCss = v.viteMetadata?.importedCss.values().next().value; - - if (importedCss != null) { - map[moduleId] = importedCss.toString(); - } else { - const importedAsset = v.viteMetadata?.importedAssets - .values() - .next().value; - - if (importedAsset != null) { - map[moduleId] = importedAsset.toString(); - } else { - map[moduleId] = path.toString(); - } - } - } - }); - - const transformed = contentResolved.reduce((acc, curr) => { - const generated = map[curr]; - if (generated != null) { - const adjustedKeyName = curr.endsWith("ResXClient.js") - ? "resXClient.js" - : path.relative(path.resolve(assetDir), curr); - - const [_, transformed] = toRescriptFieldName(adjustedKeyName, acc); - - acc[transformed] = "/" + generated; - } - return acc; - }, {}); - - // TODO: Fix cleanup? - // Do some basic cleanup of assets Vite emits that we're not really interested in. - /*Object.keys(bundle).forEach((key) => { - const entry = bundle[key]; - if ( - key.includes("/_virtual/@res-x-index-") || - (key.endsWith(".js") && original[entry.name] != null) - ) { - delete bundle[key]; - } - });*/ - - // Write JS file - const jsFileContent = `export const assets = ${JSON.stringify( - transformed, - null, - 2 - )}`; - - writeIfChanged(getAssetJsFileLoc(generated), jsFileContent); - }, - }; -} - -function getAssetDirContent(assetDir) { - return fg.globSync(["**/*"], { - dot: false, - cwd: path.resolve(assetDir), - }); -} - -function getAssetJsFileLoc(generated) { - return path.resolve(generated, "res-x-assets.js"); -} - -function getAssetResFileLoc(generated) { - return path.resolve(generated, "ResXAssets.res"); -} - -function getManifestStructure(assetDir, map = null) { - const content = getAssetDirContent(assetDir); - return content.reduce((acc, curr) => { - const generated = map != null ? map[curr] : curr; - if (generated != null) { - const [_, transformed] = toRescriptFieldName(curr, acc); - acc[transformed] = generated; - } - return acc; - }, {}); -} - -function writeIfChanged(p, content) { - let currentContent = ""; - - try { - currentContent = fs.readFileSync(p, "utf8"); - } catch {} - - if (currentContent !== content) { - const dir = path.dirname(p); - if (!fs.existsSync(dir)) { - fs.mkdirSync(dir, { recursive: true }); - } - fs.writeFileSync(p, content); - } -} - -function writeResTypeFile(assetDir, generated) { - const mapped = getManifestStructure(assetDir); - let content = `// Generated by ResX, do not edit manually\n\n`; - - content += `type assets = {\n /** ResX Client Bundle */\n resXClient_js: string,\n\n${Object.entries( - mapped - ) - .map( - ([k, v], index) => - `${index > 0 ? "\n" : ""} /** \`${v}\` */\n ${k}: string,` - ) - .join("\n")}\n}\n\n`; - - content += `@module("./res-x-assets.js") external assets: assets = "assets"`; - - const assetResFileLoc = getAssetResFileLoc(generated); - writeIfChanged(assetResFileLoc, content); -} - -function getDummyFile(assetDir, resXClientLocation) { - const content = getAssetDirContent(assetDir); - let text = [`// Generated by ResX, do not edit manually\n`]; - content.forEach((c, i) => { - if (c.endsWith(".js")) return; - text.push(`import v${i} from "${path.resolve(assetDir, c)}"`); - }); - text.push(`import "${path.resolve(resXClientLocation)}"`); - text.push( - `\nexport const assets = {\n${content - .map((c, i) => ` "${c}": v${i}`) - .join(",\n")}\n}` - ); - - const txt = text.join("\n"); - return txt; -} - -function getDummyDevFile(assetDir, resXClientLocation) { - const content = getAssetDirContent(assetDir); - let text = [`// Generated by ResX, do not edit manually\n`]; - const mapped = Object.fromEntries(content.map((k) => [k, k])); - - text.push( - `export const assets = {\n "resXClient_js": "/${resXClientLocation}",\n${content - .map((c) => { - const [_, transformed] = toRescriptFieldName(c, mapped); - return ` "${transformed}": "/assets/${c}"`; - }) - .join(",\n")}\n}` - ); - - const txt = text.join("\n"); - return txt; -} - -function toRescriptFieldName(fieldName, existingObj) { - let transformedFieldName = fieldName - .replace(/\//g, "__") - .replace(/[^a-zA-Z0-9_]/g, "_"); - - if (!/[a-z]/g.test(transformedFieldName[0])) { - transformedFieldName = `a${transformedFieldName}`; - } - - while (typeof existingObj[transformedFieldName] === "string") { - transformedFieldName += "_"; - } - - return [fieldName, transformedFieldName]; -} diff --git a/src/AssetPipeline.js b/src/AssetPipeline.js new file mode 100644 index 0000000..4972f74 --- /dev/null +++ b/src/AssetPipeline.js @@ -0,0 +1,1387 @@ +// Generated by ReScript, PLEASE EDIT WITH CARE +'use strict'; + +let $$Bun = require("bun"); +let Nodefs = require("node:fs"); +let Process = require("process"); +let Nodepath = require("node:path"); +let Nodecrypto = require("node:crypto"); +let Nodemodule = require("node:module"); +let Primitive_option = require("@rescript/runtime/lib/js/Primitive_option.js"); +let Primitive_string = require("@rescript/runtime/lib/js/Primitive_string.js"); +let Promises = require("node:fs/promises"); +let Primitive_exceptions = require("@rescript/runtime/lib/js/Primitive_exceptions.js"); + +let jsEntryExtensions = [ + ".js", + ".jsx", + ".ts", + ".tsx", + ".mjs", + ".cjs" +]; + +let cssEntryExtension = ".css"; + +let devOutputDir = Nodepath.join(".resx", "dev"); + +let prodOutputDir = "dist"; + +let generatedDir = Nodepath.join("src", "__generated__"); + +let generatedResFile = Nodepath.join(generatedDir, "ResXAssets.res"); + +let generatedJsFile = Nodepath.join(generatedDir, "res-x-assets.js"); + +let manifestFile = "resx-assets.json"; + +let resXClientKey = "resXClient_js"; + +let resXClientOutput = "resx-client.js"; + +function makeResXUsageError(message) { + let error = new Error(message); + error.name = "ResXUsageError"; + error.code = "RESX_USAGE"; + return error; +} + +function makeResXBuildError(message) { + let error = new Error(message); + error.name = "ResXBuildError"; + error.code = "RESX_BUILD"; + return error; +} + +function throwJsError(error) { + throw { + RE_EXN_ID: "JsExn", + _1: error, + Error: new Error() + }; +} + +function isNullish(value) { + return value == null; +} + +function boolFromAny(value) { + return value; +} + +function tryOrNone(f) { + let value; + try { + value = f(); + } catch (exn) { + return; + } + return Primitive_option.some(value); +} + +async function tryOrNoneAsync(f) { + let value; + try { + value = await f(); + } catch (exn) { + return; + } + return Primitive_option.some(value); +} + +function isJsFunction(value) { + return typeof value === "function"; +} + +function isJsBoolean(value) { + return typeof value === "boolean"; +} + +function isJsString(value) { + return typeof value === "string"; +} + +function isJsObject(value) { + return typeof value === "object"; +} + +function toPosixPath(value) { + if (Nodepath.sep === "/") { + return value; + } else { + return value.split(Nodepath.sep).join("/"); + } +} + +function isPathInside(candidatePath, basePath) { + let relative = Nodepath.relative(basePath, candidatePath); + if (relative === "") { + return true; + } else if (relative.startsWith("..")) { + return false; + } else { + return !Nodepath.isAbsolute(relative); + } +} + +async function pathExists(targetPath) { + try { + await Nodefs.promises.access(targetPath); + return true; + } catch (exn) { + return false; + } +} + +function pathExistsSync(targetPath) { + return Nodefs.existsSync(targetPath); +} + +function sha256HexBuffer(input) { + let hash = Nodecrypto.createHash("sha256"); + hash.update(input); + return hash.digest().toString("hex"); +} + +function sha256HexString(input) { + return sha256HexBuffer(Buffer.from(input)); +} + +function hash8Buffer(input) { + return sha256HexBuffer(input).slice(0, 8); +} + +function ensureTrailingSlash(value) { + if (value.endsWith("/")) { + return value; + } else { + return value + `/`; + } +} + +async function writeIfChangedBuffer(filePath, content) { + let previous; + let exit = 0; + let bytes; + try { + bytes = await Promises.readFile(filePath); + exit = 1; + } catch (exn) { + previous = undefined; + } + if (exit === 1) { + previous = bytes; + } + if (previous !== undefined && Buffer.compare(previous, content) === 0) { + return false; + } + await Nodefs.promises.mkdir(Nodepath.dirname(filePath), { + recursive: true + }); + await Promises.writeFile(filePath, content); + return true; +} + +async function writeIfChangedString(filePath, content) { + let bytes = Buffer.from(content); + await writeIfChangedBuffer(filePath, bytes); +} + +function isAsciiLower(code) { + if (code >= 97) { + return code <= 122; + } else { + return false; + } +} + +function isAsciiUpper(code) { + if (code >= 65) { + return code <= 90; + } else { + return false; + } +} + +function isAsciiDigit(code) { + if (code >= 48) { + return code <= 57; + } else { + return false; + } +} + +function sanitizeFieldChars(input) { + let chars = Array.from(input); + let sanitized = chars.map(char => { + if (char === "_") { + return "_"; + } + let code = char.charCodeAt(0); + if (isAsciiLower(code) || isAsciiUpper(code) || isAsciiDigit(code)) { + return char; + } else { + return "_"; + } + }); + return sanitized.join(""); +} + +function toRescriptFieldName(fieldName, existing) { + let slashNormalized = fieldName.split("/").join("__"); + let transformedRef = sanitizeFieldChars(slashNormalized); + if (transformedRef === "") { + transformedRef = "a"; + } else { + let first = transformedRef.slice(0, 1); + let code = first.charCodeAt(0); + if (!isAsciiLower(code)) { + transformedRef = `a` + transformedRef; + } + } + while (existing.has(transformedRef)) { + transformedRef = transformedRef + `_`; + }; + return transformedRef; +} + +let jsEntryExtensionSet = new Set(jsEntryExtensions); + +function formatBunLogs(logs, projectRoot) { + if (logs.length === 0) { + return ""; + } + let messages = logs.map(log => { + let match; + match = log.name === "BuildMessage" ? [ + log.message, + Primitive_option.fromNull(log.position) + ] : [ + log.message, + Primitive_option.fromNull(log.position) + ]; + let position = match[1]; + let rawMessage = match[0]; + let message = rawMessage !== "" ? rawMessage : "Unknown build error"; + if (position === undefined) { + return message; + } + let filePath = position.file; + let relative = isPathInside(filePath, projectRoot) ? toPosixPath(Nodepath.relative(projectRoot, filePath)) : filePath; + let line = position.line.toString(); + let column = position.column.toString(); + return relative + `:` + line + `:` + column + ` ` + message; + }); + return messages.join("\n"); +} + +function normalizeRootInput(root) { + if (root !== undefined && root !== "") { + return root; + } else { + return Process.cwd(); + } +} + +function scanGlobPaths(pattern, cwd, dot) { + let glob = new $$Bun.Glob(pattern); + let iter = glob.scanSync({ + cwd: cwd, + dot: dot, + followSymlinks: false, + onlyFiles: true + }); + return Array.from(iter).map(toPosixPath); +} + +function scanGlobPatternsUniqueSorted(patterns, cwd, dot) { + let seen = new Set(); + let results = []; + for (let i = 0, i_finish = patterns.length; i < i_finish; ++i) { + let pattern = patterns[i]; + let scanned = scanGlobPaths(pattern, cwd, dot); + for (let j = 0, j_finish = scanned.length; j < j_finish; ++j) { + let relPath = scanned[j]; + if (!seen.has(relPath)) { + seen.add(relPath); + results.push(relPath); + } + } + } + results.sort(Primitive_string.compare); + return results; +} + +async function resolveProjectRoot(root) { + let rootInput = normalizeRootInput(root); + let absolute = Nodepath.resolve(rootInput); + if (!await pathExists(absolute)) { + let error = makeResXBuildError(`Project root not found: ` + absolute); + throw { + RE_EXN_ID: "JsExn", + _1: error, + Error: new Error() + }; + } + let stats = await Promises.stat(absolute); + if (!stats.isDirectory()) { + let error$1 = makeResXBuildError(`Project root is not a directory: ` + absolute); + throw { + RE_EXN_ID: "JsExn", + _1: error$1, + Error: new Error() + }; + } + return await Promises.realpath(absolute); +} + +function resolveClientSource() { + return Nodepath.resolve(__dirname, "ResXClient.js"); +} + +async function discoverManagedFiles(projectRoot, directoryName) { + let baseDir = Nodepath.join(projectRoot, directoryName); + if (!await pathExists(baseDir)) { + return []; + } + let projectRootReal = await Promises.realpath(projectRoot); + let sortedRelPaths = scanGlobPatternsUniqueSorted(["**/*"], baseDir, false); + let files = []; + for (let i = 0, i_finish = sortedRelPaths.length; i < i_finish; ++i) { + let relPath = sortedRelPaths[i]; + let absPath = Nodepath.join(baseDir, relPath); + let exit = 0; + let realPath; + try { + realPath = await Promises.realpath(absPath); + exit = 1; + } catch (exn) { + + } + if (exit === 1) { + if (isPathInside(realPath, projectRootReal)) { + files.push({ + relPath: relPath, + absPath: absPath, + realPath: realPath + }); + } + } + } + return files; +} + +function hasJsEntryExtension(extension) { + return jsEntryExtensionSet.has(extension.toLowerCase()); +} + +function classifyAssetFiles(assetFiles) { + let jsEntries = []; + let cssEntries = []; + let copiedAssets = []; + for (let i = 0, i_finish = assetFiles.length; i < i_finish; ++i) { + let file = assetFiles[i]; + let extension = Nodepath.extname(file.relPath).toLowerCase(); + if (jsEntryExtensionSet.has(extension.toLowerCase())) { + jsEntries.push(file); + } else if (extension === cssEntryExtension) { + cssEntries.push(file); + } else { + copiedAssets.push(file); + } + } + return { + jsEntries: jsEntries, + cssEntries: cssEntries, + copiedAssets: copiedAssets + }; +} + +function jsOutputRelativePath(sourceRelPath) { + let parsed = Nodepath.posix.parse(sourceRelPath); + let name = parsed.name + `.js`; + if (parsed.dir !== "") { + return parsed.dir + `/` + name; + } else { + return name; + } +} + +function hashedRelativePath(relPath, bytes) { + let parsed = Nodepath.posix.parse(relPath); + let name = parsed.name + `-` + hash8Buffer(bytes) + parsed.ext; + if (parsed.dir !== "") { + return parsed.dir + `/` + name; + } else { + return name; + } +} + +function extractLogs(result) { + return result.logs; +} + +async function runBunBuild(entryPath, minify) { + let result = await tryOrNoneAsync(() => Bun.build({ + entrypoints: [entryPath], + target: "browser", + format: "esm", + splitting: false, + sourcemap: "none", + minify: minify + })); + if (result === undefined) { + return { + ok: false, + logs: [], + bytes: undefined + }; + } + if (!result.success) { + return { + ok: false, + logs: result.logs, + bytes: undefined + }; + } + let outputs = result.outputs; + if (outputs.length === 0) { + return { + ok: false, + logs: [], + bytes: undefined + }; + } + let value = outputs.find(output => output.kind === "entry-point"); + let primary = value !== undefined ? value : outputs[0]; + let rawBytes = await primary.arrayBuffer(); + let bytes = Buffer.from(rawBytes); + return { + ok: true, + logs: result.logs, + bytes: bytes + }; +} + +function createRootRequire(projectRoot) { + let packageJson = Nodepath.join(projectRoot, "package.json"); + if (Nodefs.existsSync(packageJson)) { + return Nodemodule.createRequire(packageJson); + } else { + return Nodemodule.createRequire(Nodepath.join(projectRoot, "index.js")); + } +} + +async function maybeTransformCssWithPostcss(entryPath, projectRoot) { + let configCandidates = [ + Nodepath.join(projectRoot, "postcss.config.js"), + Nodepath.join(projectRoot, "postcss.config.cjs") + ]; + let configPathRef; + for (let i = 0, i_finish = configCandidates.length; i < i_finish; ++i) { + if (configPathRef === undefined) { + let candidate = configCandidates[i]; + if (await pathExists(candidate)) { + configPathRef = candidate; + } + } + } + let configPath = configPathRef; + if (configPath === undefined) { + return; + } + let rootRequire = tryOrNone(() => createRootRequire(projectRoot)); + if (rootRequire === undefined) { + return; + } + let value = tryOrNone(() => rootRequire("postcss")); + let postcssFactory = value !== undefined ? Primitive_option.valFromOption(value) : undefined; + if (postcssFactory === undefined) { + return; + } + let configValue = tryOrNone(() => rootRequire(configPath)); + if (configValue === undefined) { + return; + } + let initialConfig = Primitive_option.valFromOption(configValue); + let config = typeof initialConfig === "function" ? tryOrNone(() => initialConfig({ + env: "development" + })) : Primitive_option.some(initialConfig); + if (config === undefined) { + return; + } + let plugins = []; + let configuredPlugins = Primitive_option.valFromOption(config).plugins; + if (!(configuredPlugins == null)) { + if (Array.isArray(configuredPlugins)) { + for (let i$1 = 0, i_finish$1 = configuredPlugins.length; i$1 < i_finish$1; ++i$1) { + let pluginEntry = configuredPlugins[i$1]; + if (!isNullish(pluginEntry) && pluginEntry !== false) { + if (typeof pluginEntry === "function") { + plugins.push(pluginEntry); + } else if (typeof pluginEntry === "string") { + let exit = 0; + let loaded; + try { + loaded = rootRequire(pluginEntry); + exit = 1; + } catch (exn) { + + } + if (exit === 1) { + if (typeof loaded === "function") { + plugins.push(loaded()); + } else { + plugins.push(loaded); + } + } + } else if (Array.isArray(pluginEntry)) { + let nameAny = pluginEntry[0]; + if (nameAny !== undefined) { + let nameAny$1 = Primitive_option.valFromOption(nameAny); + if (typeof nameAny$1 === "string") { + let exit$1 = 0; + let loaded$1; + try { + loaded$1 = rootRequire(nameAny$1); + exit$1 = 1; + } catch (exn$1) { + + } + if (exit$1 === 1) { + if (typeof loaded$1 === "function") { + let options = pluginEntry[1]; + if (options !== undefined) { + plugins.push(loaded$1(Primitive_option.valFromOption(options))); + } else { + plugins.push(loaded$1()); + } + } else { + plugins.push(loaded$1); + } + } + } else { + plugins.push(pluginEntry); + } + } else { + plugins.push(pluginEntry); + } + } else { + plugins.push(pluginEntry); + } + } + } + } else if (typeof configuredPlugins === "object") { + let configuredPluginEntries = Object.entries(configuredPlugins); + for (let i$2 = 0, i_finish$2 = configuredPluginEntries.length; i$2 < i_finish$2; ++i$2) { + let match = configuredPluginEntries[i$2]; + let pluginOptions = match[1]; + if (pluginOptions !== false) { + let exit$2 = 0; + let loaded$2; + try { + loaded$2 = rootRequire(match[0]); + exit$2 = 1; + } catch (exn$2) { + + } + if (exit$2 === 1) { + if (typeof loaded$2 === "function") { + if (isNullish(pluginOptions) || pluginOptions === true) { + plugins.push(loaded$2()); + } else { + plugins.push(loaded$2(pluginOptions)); + } + } else { + plugins.push(loaded$2); + } + } + } + } + } + } + if (plugins.length === 0) { + return; + } + let source = await Promises.readFile(entryPath, { + encoding: "utf8" + }); + let transformed; + try { + let processor = postcssFactory(plugins); + transformed = await processor.process(source, { + from: entryPath, + to: entryPath, + map: false + }); + } catch (exn$3) { + return; + } + return Buffer.from(transformed.css); +} + +async function buildJavaScript(entryPath, projectRoot, mode) { + let built = await runBunBuild(entryPath, mode === "prod"); + if (built.ok) { + let bytes = built.bytes; + if (bytes !== undefined) { + return bytes; + } + let error = makeResXBuildError(`JavaScript build failed: ` + entryPath); + throw { + RE_EXN_ID: "JsExn", + _1: error, + Error: new Error() + }; + } + let details = formatBunLogs(built.logs, projectRoot); + let error$1 = makeResXBuildError(details !== "" ? details : `JavaScript build failed: ` + entryPath); + throw { + RE_EXN_ID: "JsExn", + _1: error$1, + Error: new Error() + }; +} + +async function buildCss(entryPath, projectRoot, mode) { + let built = await runBunBuild(entryPath, mode === "prod"); + let transformed = await maybeTransformCssWithPostcss(entryPath, projectRoot); + if (transformed !== undefined) { + return transformed; + } + if (built.ok) { + let bytes = built.bytes; + if (bytes !== undefined) { + return bytes; + } + let error = makeResXBuildError(`CSS build failed: ` + entryPath); + throw { + RE_EXN_ID: "JsExn", + _1: error, + Error: new Error() + }; + } + let details = formatBunLogs(built.logs, projectRoot); + let error$1 = makeResXBuildError(details !== "" ? details : `CSS build failed: ` + entryPath); + throw { + RE_EXN_ID: "JsExn", + _1: error$1, + Error: new Error() + }; +} + +async function emitManagedAsset(mode, outputAssetsDir, logicalRelPath, bytes) { + let outputRelPath = mode === "prod" ? hashedRelativePath(logicalRelPath, bytes) : logicalRelPath; + let normalizedRelPath = toPosixPath(outputRelPath); + let destination = Nodepath.join(outputAssetsDir, normalizedRelPath); + await Nodefs.promises.mkdir(Nodepath.dirname(destination), { + recursive: true + }); + await Promises.writeFile(destination, bytes); + return { + outputRelPath: normalizedRelPath, + outputPath: destination, + urlPath: `/` + toPosixPath(Nodepath.posix.join("assets", normalizedRelPath)), + contentHash: sha256HexBuffer(bytes) + }; +} + +async function copyPublicFiles(publicFiles, projectRoot, outputRoot) { + let copied = []; + for (let i = 0, i_finish = publicFiles.length; i < i_finish; ++i) { + let file = publicFiles[i]; + let destination = Nodepath.join(outputRoot, file.relPath); + if (isPathInside(destination, outputRoot)) { + await Nodefs.promises.mkdir(Nodepath.dirname(destination), { + recursive: true + }); + await Nodefs.promises.copyFile(file.absPath, destination); + let bytes = await Promises.readFile(destination); + copied.push({ + relPath: file.relPath, + outputPath: destination, + contentHash: sha256HexBuffer(bytes) + }); + } + } + return copied; +} + +function buildGeneratedJs(keyedRecords) { + let lines = keyedRecords.map(record => ` ` + record.key + `: ` + JSON.stringify(record.urlPath) + `,`); + return `export const assets = {\n` + lines.join("\n") + `\n};\n`; +} + +function buildGeneratedRes(keyedRecords, sourceByKey) { + let lines = []; + lines.push("// Generated by ResX, do not edit manually"); + lines.push(""); + lines.push("type assets = {"); + lines.push(" /** ResX Client Bundle */"); + lines.push(" resXClient_js: string,"); + for (let i = 0, i_finish = keyedRecords.length; i < i_finish; ++i) { + let record = keyedRecords[i]; + if (record.key !== resXClientKey) { + let value = sourceByKey[record.key]; + let source = value !== undefined ? value : record.key; + let escapedSource = source.split("`").join("\\`"); + lines.push(""); + lines.push(` /** \`` + escapedSource + `\` */`); + lines.push(` ` + record.key + `: string,`); + } + } + lines.push("}"); + lines.push(""); + lines.push("@module(\"./res-x-assets.js\") external assets: assets = \"assets\""); + lines.push(""); + return lines.join("\n"); +} + +function buildManifest(mode, keyedRecords) { + let seedParts = keyedRecords.map(record => record.key + `|` + record.urlPath + `|` + record.contentHash); + let input = seedParts.join("\n"); + let fingerprint = sha256HexBuffer(Buffer.from(input)); + let assetsLines = []; + for (let i = 0, i_finish = keyedRecords.length; i < i_finish; ++i) { + let record = keyedRecords[i]; + let comma = i < (keyedRecords.length - 1 | 0) ? "," : ""; + assetsLines.push(` ` + JSON.stringify(record.key) + `: ` + JSON.stringify(record.urlPath) + comma); + } + let assetsBody = assetsLines.length === 0 ? "{}" : `{ +` + assetsLines.join("\n") + ` + }`; + return `{ + "version": 1, + "mode": ` + JSON.stringify(mode) + `, + "assets": ` + assetsBody + `, + "fingerprint": ` + JSON.stringify(fingerprint) + ` +} +`; +} + +async function cleanOutputDirectory(outputRoot, projectRoot) { + if (!isPathInside(outputRoot, projectRoot)) { + let error = makeResXBuildError(`Refusing to clean outside project root: ` + outputRoot); + throw { + RE_EXN_ID: "JsExn", + _1: error, + Error: new Error() + }; + } + return await Promises.rm(outputRoot, { + recursive: true, + force: true + }); +} + +async function buildAssets(root, devOpt, cleanOpt) { + let dev = devOpt !== undefined ? devOpt : false; + let clean = cleanOpt !== undefined ? cleanOpt : true; + let mode = dev ? "dev" : "prod"; + let projectRoot = await resolveProjectRoot(root); + let outputRoot = mode === "dev" ? Nodepath.join(projectRoot, devOutputDir) : Nodepath.join(projectRoot, prodOutputDir); + let outputAssetsDir = Nodepath.join(outputRoot, "assets"); + if (clean) { + await cleanOutputDirectory(outputRoot, projectRoot); + } + await Nodefs.promises.mkdir(outputAssetsDir, { + recursive: true + }); + let assetFiles = await discoverManagedFiles(projectRoot, "assets"); + let publicFiles = await discoverManagedFiles(projectRoot, "public"); + let classified = classifyAssetFiles(assetFiles); + let managedAssetRecords = []; + for (let i = 0, i_finish = classified.jsEntries.length; i < i_finish; ++i) { + let entry = classified.jsEntries[i]; + let bytes = await buildJavaScript(entry.absPath, projectRoot, mode); + let emitted = await emitManagedAsset(mode, outputAssetsDir, jsOutputRelativePath(entry.relPath), bytes); + managedAssetRecords.push({ + sourceRelPath: entry.relPath, + outputRelPath: emitted.outputRelPath, + outputPath: emitted.outputPath, + urlPath: emitted.urlPath, + contentHash: emitted.contentHash + }); + } + for (let i$1 = 0, i_finish$1 = classified.cssEntries.length; i$1 < i_finish$1; ++i$1) { + let entry$1 = classified.cssEntries[i$1]; + let bytes$1 = await buildCss(entry$1.absPath, projectRoot, mode); + let emitted$1 = await emitManagedAsset(mode, outputAssetsDir, entry$1.relPath, bytes$1); + managedAssetRecords.push({ + sourceRelPath: entry$1.relPath, + outputRelPath: emitted$1.outputRelPath, + outputPath: emitted$1.outputPath, + urlPath: emitted$1.urlPath, + contentHash: emitted$1.contentHash + }); + } + for (let i$2 = 0, i_finish$2 = classified.copiedAssets.length; i$2 < i_finish$2; ++i$2) { + let file = classified.copiedAssets[i$2]; + let bytes$2 = await Promises.readFile(file.absPath); + let emitted$2 = await emitManagedAsset(mode, outputAssetsDir, file.relPath, bytes$2); + managedAssetRecords.push({ + sourceRelPath: file.relPath, + outputRelPath: emitted$2.outputRelPath, + outputPath: emitted$2.outputPath, + urlPath: emitted$2.urlPath, + contentHash: emitted$2.contentHash + }); + } + managedAssetRecords.sort((a, b) => Primitive_string.compare(a.sourceRelPath, b.sourceRelPath)); + let clientSource = Nodepath.resolve(__dirname, "ResXClient.js"); + let clientBytes = await buildJavaScript(clientSource, projectRoot, mode); + let emittedClient = await emitManagedAsset(mode, outputAssetsDir, resXClientOutput, clientBytes); + if (mode === "prod") { + await copyPublicFiles(publicFiles, projectRoot, outputRoot); + } + let usedKeys = new Set(); + usedKeys.add(resXClientKey); + let sourceByKey = {}; + sourceByKey[resXClientKey] = "resXClient.js"; + let keyedRecords = [{ + key: resXClientKey, + urlPath: emittedClient.urlPath, + contentHash: emittedClient.contentHash + }]; + for (let i$3 = 0, i_finish$3 = managedAssetRecords.length; i$3 < i_finish$3; ++i$3) { + let record = managedAssetRecords[i$3]; + let key = toRescriptFieldName(record.sourceRelPath, usedKeys); + usedKeys.add(key); + sourceByKey[key] = record.sourceRelPath; + keyedRecords.push({ + key: key, + urlPath: record.urlPath, + contentHash: record.contentHash + }); + } + keyedRecords.sort((a, b) => Primitive_string.compare(a.key, b.key)); + let generatedResPath = Nodepath.join(projectRoot, generatedResFile); + let generatedJsPath = Nodepath.join(projectRoot, generatedJsFile); + await writeIfChangedString(generatedResPath, buildGeneratedRes(keyedRecords, sourceByKey)); + await writeIfChangedString(generatedJsPath, buildGeneratedJs(keyedRecords)); + let manifestPath = Nodepath.join(outputRoot, manifestFile); + let manifestText = buildManifest(mode, keyedRecords); + await writeIfChangedString(manifestPath, manifestText); +} + +function shouldTriggerSrcRebuild(filename) { + if (filename === undefined) { + return false; + } + if (filename === "") { + return false; + } + let normalized = toPosixPath(filename); + if (normalized === "ResXClient.js") { + return true; + } else { + return normalized === "ResXClient.res"; + } +} + +function makeDebouncedTask(task, waitMs) { + let timer = { + contents: undefined + }; + let running = { + contents: false + }; + let pending = { + contents: false + }; + let schedule = () => { + let existing = timer.contents; + if (existing !== undefined) { + clearTimeout(Primitive_option.valFromOption(existing)); + } + timer.contents = Primitive_option.some(setTimeout(() => { + timer.contents = undefined; + runOnce(); + }, waitMs)); + }; + let runOnce = async () => { + if (running.contents) { + pending.contents = true; + return; + } + running.contents = true; + try { + await task(); + } catch (exn) { + + } + running.contents = false; + if (pending.contents) { + pending.contents = false; + return schedule(); + } + }; + return schedule; +} + +function watchDirectory(targetPath, onChange) { + if (!Nodefs.existsSync(targetPath)) { + return; + } + let watcher; + try { + watcher = Nodefs.watch(targetPath, { + recursive: true, + encoding: "utf8" + }, (_eventType, filename) => onChange(filename)); + } catch (exn) { + return; + } + return Primitive_option.some(watcher); +} + +async function startDevWatch(root, cleanOpt, onBuildError) { + let clean = cleanOpt !== undefined ? cleanOpt : true; + let projectRoot = await resolveProjectRoot(root); + let notifyError = onBuildError !== undefined ? onBuildError : param => {}; + let rebuild = makeDebouncedTask(async () => { + try { + await buildAssets(projectRoot, true, clean); + return; + } catch (raw_error) { + let error = Primitive_exceptions.internalToException(raw_error); + if (error.RE_EXN_ID === "JsExn") { + return notifyError(error._1); + } else { + return notifyError(new Error("Unknown build error")); + } + } + }, 120); + let watchers = []; + let value = watchDirectory(Nodepath.join(projectRoot, "assets"), param => rebuild()); + if (value !== undefined) { + watchers.push(Primitive_option.valFromOption(value)); + } + let value$1 = watchDirectory(Nodepath.join(projectRoot, "public"), param => rebuild()); + if (value$1 !== undefined) { + watchers.push(Primitive_option.valFromOption(value$1)); + } + let value$2 = watchDirectory(Nodepath.join(projectRoot, "src"), filename => { + if (shouldTriggerSrcRebuild(filename)) { + return rebuild(); + } + }); + if (value$2 !== undefined) { + watchers.push(Primitive_option.valFromOption(value$2)); + } + let watchPatterns = [ + "assets/**/*", + "public/**/*", + "src/ResXClient.*" + ]; + let pollBusy = { + contents: false + }; + let computeWatchSignature = async () => { + let normalized = scanGlobPatternsUniqueSorted(watchPatterns, projectRoot, true); + let parts = []; + for (let i = 0, i_finish = normalized.length; i < i_finish; ++i) { + let relPath = normalized[i]; + let absPath = Nodepath.join(projectRoot, relPath); + let exit = 0; + let stats; + try { + stats = await Promises.stat(absPath); + exit = 1; + } catch (exn) { + + } + if (exit === 1) { + parts.push(relPath + `:` + stats.size.toString() + `:` + stats.mtimeMs.toString()); + } + } + let input = parts.join("\n"); + return sha256HexBuffer(Buffer.from(input)); + }; + let previousSignature = { + contents: await computeWatchSignature() + }; + let poller = setInterval(() => { + if (pollBusy.contents) { + return; + } else { + pollBusy.contents = true; + (async () => { + let exit = 0; + let nextSignature; + try { + nextSignature = await computeWatchSignature(); + exit = 1; + } catch (exn) { + + } + if (exit === 1) { + if (nextSignature !== previousSignature.contents) { + previousSignature.contents = nextSignature; + rebuild(); + } + } + pollBusy.contents = false; + })(); + return; + } + }, 500); + return { + close: () => { + clearInterval(poller); + for (let i = 0, i_finish = watchers.length; i < i_finish; ++i) { + try { + watchers[i].close(); + } catch (exn) { + + } + } + } + }; +} + +function closeWatcher(watcher) { + watcher.close(); +} + +function decodePathname(pathname) { + let current = pathname; + let failed = false; + for (let _index = 0; _index <= 3; ++_index) { + if (!failed) { + let exit = 0; + let decoded; + try { + decoded = decodeURIComponent(current); + exit = 1; + } catch (exn) { + failed = true; + } + if (exit === 1 && decoded !== current) { + current = decoded; + } + } + } + if (failed) { + return; + } else { + return current; + } +} + +function stripLeadingSlashes(value) { + let current = value; + while (current.startsWith("/")) { + current = current.slice(1); + }; + return current; +} + +function sanitizeUrlPath(pathname) { + let decoded = decodePathname(pathname); + if (decoded === undefined) { + return; + } + let normalizedSlashes = decoded.split("\\").join("/"); + let startsWithSlash = normalizedSlashes.startsWith("/"); + let withSlash = startsWithSlash ? normalizedSlashes : `/` + normalizedSlashes; + let segments = withSlash.split("/"); + let blocked = false; + for (let i = 0, i_finish = segments.length; i < i_finish; ++i) { + if (segments[i] === "..") { + blocked = true; + } + } + if (blocked) { + return; + } + let normalized = Nodepath.posix.normalize(withSlash); + if (normalized.startsWith("/")) { + return normalized; + } +} + +function getServeRoots(projectRoot, isDev) { + if (isDev) { + return { + managedAssetsRoot: Nodepath.join(projectRoot, "assets"), + staticRoot: Nodepath.join(projectRoot, "public") + }; + } else { + return { + managedAssetsRoot: Nodepath.join(Nodepath.join(projectRoot, prodOutputDir), "assets"), + staticRoot: Nodepath.join(projectRoot, prodOutputDir) + }; + } +} + +async function resolveServedFilePath(baseRoot, relativePath) { + let relativePosix = stripLeadingSlashes(toPosixPath(relativePath)); + let target = Nodepath.resolve(baseRoot, relativePosix); + if (!isPathInside(target, baseRoot)) { + return; + } + let exit = 0; + let stats; + try { + stats = await Promises.stat(target); + exit = 1; + } catch (exn) { + return; + } + if (exit === 1) { + if (!stats.isFile()) { + return; + } + let exit$1 = 0; + let realTarget; + try { + realTarget = await Promises.realpath(target); + exit$1 = 2; + } catch (exn$1) { + return; + } + if (exit$1 === 2) { + let exit$2 = 0; + let realBase; + try { + realBase = await Promises.realpath(baseRoot); + exit$2 = 3; + } catch (exn$2) { + return; + } + if (exit$2 === 3) { + if (isPathInside(realTarget, realBase)) { + return target; + } else { + return; + } + } + } + } +} + +function jsSourceCandidatesFromRequest(relativePath) { + let parsed = Nodepath.posix.parse(relativePath); + let base = parsed.dir !== "" ? parsed.dir + `/` + parsed.name : parsed.name; + return jsEntryExtensions.map(ext => base + ext); +} + +function responseWithContentType(body, contentType) { + let headers = new Headers(); + headers.set("Content-Type", contentType); + return new Response(body, { + headers: Primitive_option.some(headers) + }); +} + +function responseWithStatus(status) { + return new Response("", { + status: status + }); +} + +function fileResponseFromPath(filePath) { + return new Response(Bun.file(filePath)); +} + +function bufferToUtf8String(buffer) { + return buffer.toString("utf8"); +} + +function requestUrlPathname(request) { + return new URL(request.url).pathname; +} + +async function resolveDevManagedAssetResponse(projectRoot, managedAssetsRoot, relativePath) { + let extension = Nodepath.posix.extname(relativePath).toLowerCase(); + if (extension === ".js") { + let sourcePathRef = await resolveServedFilePath(managedAssetsRoot, relativePath); + if (sourcePathRef === undefined) { + let sourceCandidates = jsSourceCandidatesFromRequest(relativePath); + for (let i = 0, i_finish = sourceCandidates.length; i < i_finish; ++i) { + if (sourcePathRef === undefined) { + sourcePathRef = await resolveServedFilePath(managedAssetsRoot, sourceCandidates[i]); + } + } + } + let sourcePath = sourcePathRef; + if (sourcePath === undefined) { + return; + } + let jsBytes = await buildJavaScript(sourcePath, projectRoot, "dev"); + return responseWithContentType(jsBytes.toString("utf8"), "text/javascript; charset=utf-8"); + } + if (extension === ".css") { + let sourcePath$1 = await resolveServedFilePath(managedAssetsRoot, relativePath); + if (sourcePath$1 === undefined) { + return; + } + let cssBytes = await buildCss(sourcePath$1, projectRoot, "dev"); + return responseWithContentType(cssBytes.toString("utf8"), "text/css; charset=utf-8"); + } + let exactPath = await resolveServedFilePath(managedAssetsRoot, relativePath); + if (exactPath !== undefined) { + return new Response(Bun.file(exactPath)); + } +} + +let suspiciousRootNames = [ + "bin", + "etc", + "home", + "opt", + "private", + "proc", + "root", + "sbin", + "sys", + "tmp", + "usr", + "var", + "windows" +]; + +function hasSuspiciousRoot(value) { + return suspiciousRootNames.includes(value); +} + +async function serveStaticFileFromBuild(request, projectRoot, isDev) { + let pathname = new URL(request.url).pathname; + let sanitized = sanitizeUrlPath(pathname); + if (sanitized === undefined) { + return responseWithStatus(403); + } + let segments = sanitized.split("/"); + let value = segments.find(segment => segment !== ""); + let firstSegment = value !== undefined ? value : ""; + if (suspiciousRootNames.includes(firstSegment)) { + return responseWithStatus(403); + } + if (firstSegment === "src") { + let sourceRelative = sanitized.slice(1); + let sourceCandidate = Nodepath.resolve(projectRoot, sourceRelative); + if (isPathInside(sourceCandidate, projectRoot) && Nodefs.existsSync(sourceCandidate)) { + return responseWithStatus(403); + } else { + return; + } + } + if (sanitized === "/") { + return; + } + let roots = getServeRoots(projectRoot, isDev); + let managedAssetsRequest = sanitized === "/assets" || sanitized.startsWith("/assets/"); + if (managedAssetsRequest) { + let relative = sanitized === "/assets" ? "" : sanitized.slice(8); + if (isDev) { + let response; + try { + response = await resolveDevManagedAssetResponse(projectRoot, roots.managedAssetsRoot, relative); + } catch (raw_exn) { + let exn = Primitive_exceptions.internalToException(raw_exn); + console.error(`[resx] failed to serve dev asset /assets/` + relative + `\n` + String(exn)); + return responseWithStatus(500); + } + if (response !== undefined) { + return response; + } else { + return responseWithStatus(404); + } + } + let filePath = await resolveServedFilePath(roots.managedAssetsRoot, relative); + if (filePath !== undefined) { + return new Response(Bun.file(filePath)); + } else { + return responseWithStatus(404); + } + } + let relative$1 = sanitized.slice(1); + let filePath$1 = await resolveServedFilePath(roots.staticRoot, relative$1); + if (filePath$1 !== undefined) { + return new Response(Bun.file(filePath$1)); + } +} + +exports.jsEntryExtensions = jsEntryExtensions; +exports.cssEntryExtension = cssEntryExtension; +exports.devOutputDir = devOutputDir; +exports.prodOutputDir = prodOutputDir; +exports.generatedDir = generatedDir; +exports.generatedResFile = generatedResFile; +exports.generatedJsFile = generatedJsFile; +exports.manifestFile = manifestFile; +exports.resXClientKey = resXClientKey; +exports.resXClientOutput = resXClientOutput; +exports.makeResXUsageError = makeResXUsageError; +exports.makeResXBuildError = makeResXBuildError; +exports.throwJsError = throwJsError; +exports.isNullish = isNullish; +exports.boolFromAny = boolFromAny; +exports.tryOrNone = tryOrNone; +exports.tryOrNoneAsync = tryOrNoneAsync; +exports.isJsFunction = isJsFunction; +exports.isJsBoolean = isJsBoolean; +exports.isJsString = isJsString; +exports.isJsObject = isJsObject; +exports.toPosixPath = toPosixPath; +exports.isPathInside = isPathInside; +exports.pathExists = pathExists; +exports.pathExistsSync = pathExistsSync; +exports.sha256HexBuffer = sha256HexBuffer; +exports.sha256HexString = sha256HexString; +exports.hash8Buffer = hash8Buffer; +exports.ensureTrailingSlash = ensureTrailingSlash; +exports.writeIfChangedBuffer = writeIfChangedBuffer; +exports.writeIfChangedString = writeIfChangedString; +exports.isAsciiLower = isAsciiLower; +exports.isAsciiUpper = isAsciiUpper; +exports.isAsciiDigit = isAsciiDigit; +exports.sanitizeFieldChars = sanitizeFieldChars; +exports.toRescriptFieldName = toRescriptFieldName; +exports.jsEntryExtensionSet = jsEntryExtensionSet; +exports.formatBunLogs = formatBunLogs; +exports.normalizeRootInput = normalizeRootInput; +exports.scanGlobPaths = scanGlobPaths; +exports.scanGlobPatternsUniqueSorted = scanGlobPatternsUniqueSorted; +exports.resolveProjectRoot = resolveProjectRoot; +exports.resolveClientSource = resolveClientSource; +exports.discoverManagedFiles = discoverManagedFiles; +exports.hasJsEntryExtension = hasJsEntryExtension; +exports.classifyAssetFiles = classifyAssetFiles; +exports.jsOutputRelativePath = jsOutputRelativePath; +exports.hashedRelativePath = hashedRelativePath; +exports.extractLogs = extractLogs; +exports.runBunBuild = runBunBuild; +exports.createRootRequire = createRootRequire; +exports.maybeTransformCssWithPostcss = maybeTransformCssWithPostcss; +exports.buildJavaScript = buildJavaScript; +exports.buildCss = buildCss; +exports.emitManagedAsset = emitManagedAsset; +exports.copyPublicFiles = copyPublicFiles; +exports.buildGeneratedJs = buildGeneratedJs; +exports.buildGeneratedRes = buildGeneratedRes; +exports.buildManifest = buildManifest; +exports.cleanOutputDirectory = cleanOutputDirectory; +exports.buildAssets = buildAssets; +exports.shouldTriggerSrcRebuild = shouldTriggerSrcRebuild; +exports.makeDebouncedTask = makeDebouncedTask; +exports.watchDirectory = watchDirectory; +exports.startDevWatch = startDevWatch; +exports.closeWatcher = closeWatcher; +exports.decodePathname = decodePathname; +exports.stripLeadingSlashes = stripLeadingSlashes; +exports.sanitizeUrlPath = sanitizeUrlPath; +exports.getServeRoots = getServeRoots; +exports.resolveServedFilePath = resolveServedFilePath; +exports.jsSourceCandidatesFromRequest = jsSourceCandidatesFromRequest; +exports.responseWithContentType = responseWithContentType; +exports.responseWithStatus = responseWithStatus; +exports.fileResponseFromPath = fileResponseFromPath; +exports.bufferToUtf8String = bufferToUtf8String; +exports.requestUrlPathname = requestUrlPathname; +exports.resolveDevManagedAssetResponse = resolveDevManagedAssetResponse; +exports.suspiciousRootNames = suspiciousRootNames; +exports.hasSuspiciousRoot = hasSuspiciousRoot; +exports.serveStaticFileFromBuild = serveStaticFileFromBuild; +/* devOutputDir Not a pure module */ diff --git a/src/AssetPipeline.res b/src/AssetPipeline.res new file mode 100644 index 0000000..784a0a8 --- /dev/null +++ b/src/AssetPipeline.res @@ -0,0 +1,1432 @@ +type watcher = {close: unit => unit} + +type rmOptions = {recursive: bool, force: bool} +type readFileOptions = {encoding: string} +type fsWatchOptions = {recursive: bool, encoding: string} + +@module("node:fs/promises") external fsStat: string => promise = "stat" +@module("node:fs/promises") external fsRealpath: string => promise = "realpath" +@module("node:fs/promises") external fsReadFile: string => promise = "readFile" +@module("node:fs/promises") +external fsReadFileUtf8: (string, readFileOptions) => promise = "readFile" +@module("node:fs/promises") +external fsWriteFileBuffer: (string, Buffer.t) => promise = "writeFile" +@module("node:fs/promises") external fsRm: (string, rmOptions) => promise = "rm" + +type fsWatcher +@send external fsWatcherClose: fsWatcher => unit = "close" +@module("node:fs") +external fsWatch: (string, fsWatchOptions, (string, option) => unit) => fsWatcher = "watch" + +@module("node:module") external createRequire: string => string => 'a = "createRequire" + +@val external jsonStringify: 'a => string = "JSON.stringify" + +@val external __dirname: string = "__dirname" + +@set external setErrorName: (JsError.t, string) => unit = "name" +@set external setErrorCode: (JsError.t, string) => unit = "code" + +external errorToString: 'a => string = "String" + +type postcssProcessOptions = { + from: string, + @as("to") to_: string, + map: bool, +} + +type postcssProcessor +type postcssResult = {css: string} +type postcssEnv = {env: string} +@send +external postcssProcess: ( + postcssProcessor, + string, + postcssProcessOptions, +) => promise = "process" + +@get external getConfigPlugins: 'a => Nullable.t<'b> = "plugins" + +let jsEntryExtensions = [".js", ".jsx", ".ts", ".tsx", ".mjs", ".cjs"] +let cssEntryExtension = ".css" + +let devOutputDir = Path.join2(".resx", "dev") +let prodOutputDir = "dist" + +let generatedDir = Path.join2("src", "__generated__") +let generatedResFile = Path.join2(generatedDir, "ResXAssets.res") +let generatedJsFile = Path.join2(generatedDir, "res-x-assets.js") + +let manifestFile = "resx-assets.json" +let resXClientKey = "resXClient_js" +let resXClientOutput = "resx-client.js" + +type managedFile = { + relPath: string, + absPath: string, + realPath: string, +} + +type managedAssetRecord = { + sourceRelPath: string, + outputRelPath: string, + outputPath: string, + urlPath: string, + contentHash: string, +} + +type emittedAsset = { + outputRelPath: string, + outputPath: string, + urlPath: string, + contentHash: string, +} + +type copiedPublicRecord = { + relPath: string, + outputPath: string, + contentHash: string, +} + +type keyedRecord = { + key: string, + urlPath: string, + contentHash: string, +} + +type classifiedAssets = { + jsEntries: array, + cssEntries: array, + copiedAssets: array, +} + +type serveRoots = { + managedAssetsRoot: string, + staticRoot: string, +} + +type bunBuildRunResult = { + ok: bool, + logs: array, + bytes: option, +} + +let makeResXUsageError = message => { + let error = JsError.make(message) + error->setErrorName("ResXUsageError") + error->setErrorCode("RESX_USAGE") + error->JsError.toJsExn +} + +let makeResXBuildError = message => { + let error = JsError.make(message) + error->setErrorName("ResXBuildError") + error->setErrorCode("RESX_BUILD") + error->JsError.toJsExn +} + +let throwJsError = (error: JsExn.t) => throw(JsExn(error)) + +let isNullish = value => { + let nullableValue: Nullable.t<'a> = Obj.magic(value) + switch nullableValue->Nullable.toOption { + | None => true + | Some(_) => false + } +} + +let boolFromAny = value => { + let boolValue: bool = Obj.magic(value) + boolValue +} + +let tryOrNone = f => + switch f() { + | value => Some(value) + | exception _ => None + } + +let tryOrNoneAsync = async f => + switch await f() { + | value => Some(value) + | exception _ => None + } + +let isJsFunction = value => Type.typeof(value) == #function +let isJsBoolean = value => Type.typeof(value) == #boolean +let isJsString = value => Type.typeof(value) == #string +let isJsObject = value => Type.typeof(value) == #object + +let toPosixPath = value => { + if Path.sep == "/" { + value + } else { + value->String.split(Path.sep)->Array.join("/") + } +} + +let isPathInside = (~candidatePath, ~basePath) => { + let relative = Path.relative(~from=basePath, ~to_=candidatePath) + relative == "" || (!(relative->String.startsWith("..")) && !Path.isAbsolute(relative)) +} + +let pathExists = async targetPath => { + switch await Fs.access(targetPath) { + | () => true + | exception _ => false + } +} + +let pathExistsSync = targetPath => Fs.existsSync(targetPath) + +let sha256HexBuffer = input => { + let hash = RescriptBun.Crypto.createHash("sha256") + hash->RescriptBun.Crypto.Hash.update(input) + hash->RescriptBun.Crypto.Hash.digest->Buffer.toStringWithEncoding(StringEncoding.hex) +} + +let sha256HexString = input => sha256HexBuffer(Buffer.fromString(input)) + +let hash8Buffer = input => sha256HexBuffer(input)->String.slice(~start=0, ~end=8) + +let ensureTrailingSlash = value => + if value->String.endsWith("/") { + value + } else { + `${value}/` + } + +let writeIfChangedBuffer = async (~filePath, ~content) => { + let previous = switch await fsReadFile(filePath) { + | bytes => Some(bytes) + | exception _ => None + } + + switch previous { + | Some(existing) if Buffer.compare(existing, content) == 0 => false + | _ => + await Fs.mkdir(Path.dirname(filePath), {recursive: true}) + await fsWriteFileBuffer(filePath, content) + true + } +} + +let writeIfChangedString = async (~filePath, ~content) => { + let bytes = Buffer.fromString(content) + ignore(await writeIfChangedBuffer(~filePath, ~content=bytes)) +} + +let isAsciiLower = code => code >= 97 && code <= 122 +let isAsciiUpper = code => code >= 65 && code <= 90 +let isAsciiDigit = code => code >= 48 && code <= 57 + +let sanitizeFieldChars = input => { + let chars = input->Array.fromString + let sanitized = chars->Array.map(char => { + if char == "_" { + "_" + } else { + let code = char->String.charCodeAtUnsafe(0) + if isAsciiLower(code) || isAsciiUpper(code) || isAsciiDigit(code) { + char + } else { + "_" + } + } + }) + sanitized->Array.join("") +} + +let toRescriptFieldName = (fieldName, existing: Set.t) => { + let slashNormalized = fieldName->String.split("/")->Array.join("__") + let transformedRef = ref(sanitizeFieldChars(slashNormalized)) + + if transformedRef.contents == "" { + transformedRef := "a" + } else { + let first = transformedRef.contents->String.slice(~start=0, ~end=1) + let code = first->String.charCodeAtUnsafe(0) + if !isAsciiLower(code) { + transformedRef := `a${transformedRef.contents}` + } + } + + while Set.has(existing, transformedRef.contents) { + transformedRef := `${transformedRef.contents}_` + } + + transformedRef.contents +} + +let jsEntryExtensionSet = Set.fromArray(jsEntryExtensions) + +let formatBunLogs = (logs, projectRoot) => { + if logs->Array.length == 0 { + "" + } else { + let messages = logs->Array.map(log => { + let (rawMessage, position) = switch log { + | Bun.Build.BuildMessage(payload) => (payload.message, payload.position->Null.toOption) + | Bun.Build.ResolveMessage(payload) => (payload.message, payload.position->Null.toOption) + } + + let message = if rawMessage != "" { + rawMessage + } else { + "Unknown build error" + } + + switch position { + | Some(location) => + let filePath = location.file + let relative = if isPathInside(~candidatePath=filePath, ~basePath=projectRoot) { + Path.relative(~from=projectRoot, ~to_=filePath)->toPosixPath + } else { + filePath + } + + let line = Int.toString(location.line) + let column = Int.toString(location.column) + + `${relative}:${line}:${column} ${message}` + | None => message + } + }) + + messages->Array.join("\n") + } +} + +let normalizeRootInput = root => + switch root { + | Some(value) if value != "" => value + | _ => Process.process->Process.cwd + } + +let scanGlobPaths = (~pattern, ~cwd, ~dot) => { + let glob = Bun.Glob.make(pattern) + let iter = glob->Bun.Glob.scanSync(~options={cwd, dot, onlyFiles: true, followSymlinks: false}) + Array.fromIterator(iter)->Array.map(toPosixPath) +} + +let scanGlobPatternsUniqueSorted = (~patterns, ~cwd, ~dot) => { + let seen = Set.make() + let results: array = [] + + for i in 0 to patterns->Array.length - 1 { + let pattern = patterns->Array.getUnsafe(i) + let scanned = scanGlobPaths(~pattern, ~cwd, ~dot) + + for j in 0 to scanned->Array.length - 1 { + let relPath = scanned->Array.getUnsafe(j) + if !Set.has(seen, relPath) { + Set.add(seen, relPath) + results->Array.push(relPath) + } + } + } + + results->Array.sort(String.compare) + results +} + +let resolveProjectRoot = async (~root=?) => { + let rootInput = normalizeRootInput(root) + let absolute = Path.resolve([rootInput]) + + if !(await pathExists(absolute)) { + throwJsError(makeResXBuildError(`Project root not found: ${absolute}`)) + } + + let stats = await fsStat(absolute) + if !(stats->Fs.Stats.isDirectory) { + throwJsError(makeResXBuildError(`Project root is not a directory: ${absolute}`)) + } + + await fsRealpath(absolute) +} + +let resolveClientSource = () => Path.resolve([__dirname, "ResXClient.js"]) + +let discoverManagedFiles = async (~projectRoot, ~directoryName): array => { + let baseDir = Path.join2(projectRoot, directoryName) + if !(await pathExists(baseDir)) { + [] + } else { + let projectRootReal = await fsRealpath(projectRoot) + let sortedRelPaths = scanGlobPatternsUniqueSorted(~patterns=["**/*"], ~cwd=baseDir, ~dot=false) + + let files: array = [] + + for i in 0 to sortedRelPaths->Array.length - 1 { + let relPath = sortedRelPaths->Array.getUnsafe(i) + let absPath = Path.join2(baseDir, relPath) + + switch await fsRealpath(absPath) { + | realPath => + if isPathInside(~candidatePath=realPath, ~basePath=projectRootReal) { + files->Array.push({ + relPath, + absPath, + realPath, + }) + () + } + | exception _ => () + } + } + + files + } +} + +let hasJsEntryExtension = extension => { + Set.has(jsEntryExtensionSet, extension->String.toLowerCase) +} + +let classifyAssetFiles = (assetFiles: array): classifiedAssets => { + let jsEntries: array = [] + let cssEntries: array = [] + let copiedAssets: array = [] + + for i in 0 to assetFiles->Array.length - 1 { + let file = assetFiles->Array.getUnsafe(i) + let extension = file.relPath->Path.extname->String.toLowerCase + + if hasJsEntryExtension(extension) { + jsEntries->Array.push(file) + () + } else if extension == cssEntryExtension { + cssEntries->Array.push(file) + () + } else { + copiedAssets->Array.push(file) + () + } + } + + {jsEntries, cssEntries, copiedAssets} +} + +let jsOutputRelativePath = sourceRelPath => { + let parsed = Path.Posix.parse(sourceRelPath) + let name = `${parsed.name}.js` + if parsed.dir != "" { + `${parsed.dir}/${name}` + } else { + name + } +} + +let hashedRelativePath = (~relPath, ~bytes) => { + let parsed = Path.Posix.parse(relPath) + let name = `${parsed.name}-${hash8Buffer(bytes)}${parsed.ext}` + if parsed.dir != "" { + `${parsed.dir}/${name}` + } else { + name + } +} + +let extractLogs = (result: Bun.Build.buildOutput) => result.logs + +let runBunBuild = async (~entryPath, ~minify) => { + let result = await tryOrNoneAsync(() => Bun.build({ + entrypoints: [entryPath], + target: Bun.Browser, + format: Bun.Build.Esm, + splitting: false, + minify: Bun.Build.Bool(minify), + sourcemap: Bun.Build.None, + })) + + switch result { + | None => { + ok: false, + logs: [], + bytes: None, + } + | Some(result) => + if !result.success { + { + ok: false, + logs: extractLogs(result), + bytes: None, + } + } else { + let outputs = result.outputs + + if outputs->Array.length == 0 { + { + ok: false, + logs: [], + bytes: None, + } + } else { + let primary = switch outputs->Array.find(output => output.kind == Bun.Build.BuildArtifact.EntryPoint) { + | Some(value) => value + | None => outputs->Array.getUnsafe(0) + } + + let rawBytes = await primary->Bun.Build.BuildArtifact.asBlob->Blob.arrayBuffer + let bytes = Buffer.fromArrayBuffer(rawBytes) + { + ok: true, + logs: extractLogs(result), + bytes: Some(bytes), + } + } + } + } +} + +let createRootRequire = projectRoot => { + let packageJson = Path.join2(projectRoot, "package.json") + if pathExistsSync(packageJson) { + createRequire(packageJson) + } else { + createRequire(Path.join2(projectRoot, "index.js")) + } +} + +let maybeTransformCssWithPostcss = async (~entryPath, ~projectRoot) => { + let configCandidates = [ + Path.join2(projectRoot, "postcss.config.js"), + Path.join2(projectRoot, "postcss.config.cjs"), + ] + + let configPathRef: ref> = ref(None) + + for i in 0 to configCandidates->Array.length - 1 { + if configPathRef.contents == None { + let candidate = configCandidates->Array.getUnsafe(i) + if await pathExists(candidate) { + configPathRef := Some(candidate) + } + } + } + + switch configPathRef.contents { + | None => None + | Some(configPath) => + let rootRequire = tryOrNone(() => createRootRequire(projectRoot)) + + switch rootRequire { + | None => None + | Some(rootRequire) => + let postcssFactory: option => postcssProcessor> = + switch tryOrNone(() => rootRequire("postcss")) { + | Some(value) => Some(Obj.magic(value)) + | None => None + } + + switch postcssFactory { + | None => None + | Some(postcssFactory) => + let configValue: option<'a> = tryOrNone(() => rootRequire(configPath)) + + switch configValue { + | None => None + | Some(initialConfig) => + let config = if isJsFunction(initialConfig) { + tryOrNone(() => { + let runConfig: postcssEnv => 'a = Obj.magic(initialConfig) + runConfig({env: "development"}) + }) + } else { + Some(initialConfig) + } + + switch config { + | None => None + | Some(config) => + let plugins: array<'a> = [] + let configuredPlugins = config->getConfigPlugins->Nullable.toOption + + switch configuredPlugins { + | None => () + | Some(configuredPluginsAny) => + if Array.isArray(configuredPluginsAny) { + let configuredPluginArray: array<'a> = Obj.magic(configuredPluginsAny) + + for i in 0 to configuredPluginArray->Array.length - 1 { + let pluginEntry = configuredPluginArray->Array.getUnsafe(i) + + if isNullish(pluginEntry) { + () + } else if isJsBoolean(pluginEntry) && !boolFromAny(pluginEntry) { + () + } else if isJsFunction(pluginEntry) { + plugins->Array.push(pluginEntry) + () + } else if isJsString(pluginEntry) { + let name: string = Obj.magic(pluginEntry) + switch rootRequire(name) { + | loaded => + if isJsFunction(loaded) { + let createPlugin: unit => 'a = Obj.magic(loaded) + plugins->Array.push(createPlugin()) + () + } else { + plugins->Array.push(loaded) + () + } + | exception _ => () + } + } else if Array.isArray(pluginEntry) { + let pair: array<'a> = Obj.magic(pluginEntry) + switch pair->Array.get(0) { + | Some(nameAny) if isJsString(nameAny) => + let name: string = Obj.magic(nameAny) + switch rootRequire(name) { + | loaded => + if isJsFunction(loaded) { + let createPlugin: 'a => 'b = Obj.magic(loaded) + switch pair->Array.get(1) { + | Some(options) => + plugins->Array.push(createPlugin(options)) + () + | None => + let createPluginNoArg: unit => 'b = Obj.magic(loaded) + plugins->Array.push(createPluginNoArg()) + () + } + } else { + plugins->Array.push(loaded) + () + } + | exception _ => () + } + | _ => plugins->Array.push(pluginEntry) + } + } else { + plugins->Array.push(pluginEntry) + () + } + } + } else if isJsObject(configuredPluginsAny) { + let configuredPluginEntries: array<(string, 'a)> = (Obj.magic(configuredPluginsAny): dict<'a>)->Dict.toArray + + for i in 0 to configuredPluginEntries->Array.length - 1 { + let (name, pluginOptions) = configuredPluginEntries->Array.getUnsafe(i) + + if isJsBoolean(pluginOptions) && !boolFromAny(pluginOptions) { + () + } else { + switch rootRequire(name) { + | loaded => + if isJsFunction(loaded) { + if ( + isNullish(pluginOptions) || + (isJsBoolean(pluginOptions) && boolFromAny(pluginOptions)) + ) { + let createPlugin: unit => 'a = Obj.magic(loaded) + plugins->Array.push(createPlugin()) + () + } else { + let createPlugin: 'a => 'b = Obj.magic(loaded) + plugins->Array.push(createPlugin(pluginOptions)) + () + } + } else { + plugins->Array.push(loaded) + () + } + | exception _ => () + } + } + } + } else { + () + } + } + + if plugins->Array.length == 0 { + None + } else { + let source = await fsReadFileUtf8(entryPath, {encoding: "utf8"}) + + switch { + let processor = postcssFactory(plugins) + await postcssProcess( + processor, + source, + { + from: entryPath, + to_: entryPath, + map: false, + }, + ) + } { + | transformed => Some(Buffer.fromString(transformed.css)) + | exception _ => None + } + } + } + } + } + } + } +} + +let buildJavaScript = async (~entryPath, ~projectRoot, ~mode) => { + let built = await runBunBuild(~entryPath, ~minify=mode == "prod") + if built.ok { + switch built.bytes { + | Some(bytes) => bytes + | None => throwJsError(makeResXBuildError(`JavaScript build failed: ${entryPath}`)) + } + } else { + let details = formatBunLogs(built.logs, projectRoot) + throwJsError( + makeResXBuildError(details != "" ? details : `JavaScript build failed: ${entryPath}`), + ) + } +} + +let buildCss = async (~entryPath, ~projectRoot, ~mode) => { + let built = await runBunBuild(~entryPath, ~minify=mode == "prod") + + switch await maybeTransformCssWithPostcss(~entryPath, ~projectRoot) { + | Some(transformed) => transformed + | None => + if built.ok { + switch built.bytes { + | Some(bytes) => bytes + | None => throwJsError(makeResXBuildError(`CSS build failed: ${entryPath}`)) + } + } else { + let details = formatBunLogs(built.logs, projectRoot) + throwJsError(makeResXBuildError(details != "" ? details : `CSS build failed: ${entryPath}`)) + } + } +} + +let emitManagedAsset = async (~mode, ~outputAssetsDir, ~logicalRelPath, ~bytes): emittedAsset => { + let outputRelPath = if mode == "prod" { + hashedRelativePath(~relPath=logicalRelPath, ~bytes) + } else { + logicalRelPath + } + + let normalizedRelPath = toPosixPath(outputRelPath) + let destination = Path.join2(outputAssetsDir, normalizedRelPath) + + await Fs.mkdir(Path.dirname(destination), {recursive: true}) + await fsWriteFileBuffer(destination, bytes) + + { + outputRelPath: normalizedRelPath, + outputPath: destination, + urlPath: `/${Path.Posix.join(["assets", normalizedRelPath])->toPosixPath}`, + contentHash: sha256HexBuffer(bytes), + } +} + +let copyPublicFiles = async (~publicFiles: array, ~projectRoot, ~outputRoot) => { + let _ = projectRoot + let copied: array = [] + + for i in 0 to publicFiles->Array.length - 1 { + let file = publicFiles->Array.getUnsafe(i) + let destination = Path.join2(outputRoot, file.relPath) + + if isPathInside(~candidatePath=destination, ~basePath=outputRoot) { + await Fs.mkdir(Path.dirname(destination), {recursive: true}) + await Fs.copyFile(file.absPath, ~dest=destination) + let bytes = await fsReadFile(destination) + + copied->Array.push({ + relPath: file.relPath, + outputPath: destination, + contentHash: sha256HexBuffer(bytes), + }) + () + } + } + + copied +} + +let buildGeneratedJs = keyedRecords => { + let lines = + keyedRecords->Array.map(record => ` ${record.key}: ${jsonStringify(record.urlPath)},`) + `export const assets = {\n${lines->Array.join("\n")}\n};\n` +} + +let buildGeneratedRes = (~keyedRecords, ~sourceByKey: dict) => { + let lines: array = [] + lines->Array.push("// Generated by ResX, do not edit manually") + lines->Array.push("") + lines->Array.push("type assets = {") + lines->Array.push(" /** ResX Client Bundle */") + lines->Array.push(" resXClient_js: string,") + + for i in 0 to keyedRecords->Array.length - 1 { + let record = keyedRecords->Array.getUnsafe(i) + if record.key != resXClientKey { + let source = switch Dict.get(sourceByKey, record.key) { + | Some(value) => value + | None => record.key + } + + let escapedSource = source->String.split("`")->Array.join("\\`") + lines->Array.push("") + lines->Array.push(` /** \`${escapedSource}\` */`) + lines->Array.push(` ${record.key}: string,`) + () + } + } + + lines->Array.push("}") + lines->Array.push("") + lines->Array.push("@module(\"./res-x-assets.js\") external assets: assets = \"assets\"") + lines->Array.push("") + + lines->Array.join("\n") +} + +let buildManifest = (~mode, ~keyedRecords) => { + let seedParts = + keyedRecords->Array.map(record => `${record.key}|${record.urlPath}|${record.contentHash}`) + let fingerprint = sha256HexString(seedParts->Array.join("\n")) + + let assetsLines: array = [] + for i in 0 to keyedRecords->Array.length - 1 { + let record = keyedRecords->Array.getUnsafe(i) + let comma = if i < keyedRecords->Array.length - 1 { + "," + } else { + "" + } + assetsLines->Array.push( + ` ${jsonStringify(record.key)}: ${jsonStringify(record.urlPath)}${comma}`, + ) + } + + let assetsBody = if assetsLines->Array.length == 0 { + "{}" + } else { + `{ +${assetsLines->Array.join("\n")} + }` + } + + `{ + "version": 1, + "mode": ${jsonStringify(mode)}, + "assets": ${assetsBody}, + "fingerprint": ${jsonStringify(fingerprint)} +} +` +} + +let cleanOutputDirectory = async (~outputRoot, ~projectRoot) => { + if !isPathInside(~candidatePath=outputRoot, ~basePath=projectRoot) { + throwJsError(makeResXBuildError(`Refusing to clean outside project root: ${outputRoot}`)) + } + + await fsRm(outputRoot, {recursive: true, force: true}) +} + +let buildAssets = async (~root=?, ~dev=false, ~clean=true) => { + let mode = if dev { + "dev" + } else { + "prod" + } + let projectRoot = await resolveProjectRoot(~root?) + + let outputRoot = if mode == "dev" { + Path.join2(projectRoot, devOutputDir) + } else { + Path.join2(projectRoot, prodOutputDir) + } + + let outputAssetsDir = Path.join2(outputRoot, "assets") + + if clean { + await cleanOutputDirectory(~outputRoot, ~projectRoot) + } + + await Fs.mkdir(outputAssetsDir, {recursive: true}) + + let assetFiles = await discoverManagedFiles(~projectRoot, ~directoryName="assets") + let publicFiles = await discoverManagedFiles(~projectRoot, ~directoryName="public") + let classified = classifyAssetFiles(assetFiles) + + let managedAssetRecords: array = [] + + for i in 0 to classified.jsEntries->Array.length - 1 { + let entry = classified.jsEntries->Array.getUnsafe(i) + let bytes = await buildJavaScript(~entryPath=entry.absPath, ~projectRoot, ~mode) + let emitted = await emitManagedAsset( + ~mode, + ~outputAssetsDir, + ~logicalRelPath=jsOutputRelativePath(entry.relPath), + ~bytes, + ) + + managedAssetRecords->Array.push({ + sourceRelPath: entry.relPath, + outputRelPath: emitted.outputRelPath, + outputPath: emitted.outputPath, + urlPath: emitted.urlPath, + contentHash: emitted.contentHash, + }) + () + } + + for i in 0 to classified.cssEntries->Array.length - 1 { + let entry = classified.cssEntries->Array.getUnsafe(i) + let bytes = await buildCss(~entryPath=entry.absPath, ~projectRoot, ~mode) + let emitted = await emitManagedAsset( + ~mode, + ~outputAssetsDir, + ~logicalRelPath=entry.relPath, + ~bytes, + ) + + managedAssetRecords->Array.push({ + sourceRelPath: entry.relPath, + outputRelPath: emitted.outputRelPath, + outputPath: emitted.outputPath, + urlPath: emitted.urlPath, + contentHash: emitted.contentHash, + }) + () + } + + for i in 0 to classified.copiedAssets->Array.length - 1 { + let file = classified.copiedAssets->Array.getUnsafe(i) + let bytes = await fsReadFile(file.absPath) + let emitted = await emitManagedAsset( + ~mode, + ~outputAssetsDir, + ~logicalRelPath=file.relPath, + ~bytes, + ) + + managedAssetRecords->Array.push({ + sourceRelPath: file.relPath, + outputRelPath: emitted.outputRelPath, + outputPath: emitted.outputPath, + urlPath: emitted.urlPath, + contentHash: emitted.contentHash, + }) + () + } + + managedAssetRecords->Array.sort((a, b) => String.compare(a.sourceRelPath, b.sourceRelPath)) + + let clientSource = resolveClientSource() + let clientBytes = await buildJavaScript(~entryPath=clientSource, ~projectRoot, ~mode) + let emittedClient = await emitManagedAsset( + ~mode, + ~outputAssetsDir, + ~logicalRelPath=resXClientOutput, + ~bytes=clientBytes, + ) + + if mode == "prod" { + let _copied = await copyPublicFiles(~publicFiles, ~projectRoot, ~outputRoot) + } + + let usedKeys = Set.make() + Set.add(usedKeys, resXClientKey) + + let sourceByKey: dict = Dict.make() + Dict.set(sourceByKey, resXClientKey, "resXClient.js") + + let keyedRecords: array = [ + { + key: resXClientKey, + urlPath: emittedClient.urlPath, + contentHash: emittedClient.contentHash, + }, + ] + + for i in 0 to managedAssetRecords->Array.length - 1 { + let record = managedAssetRecords->Array.getUnsafe(i) + let key = toRescriptFieldName(record.sourceRelPath, usedKeys) + Set.add(usedKeys, key) + Dict.set(sourceByKey, key, record.sourceRelPath) + + keyedRecords->Array.push({ + key, + urlPath: record.urlPath, + contentHash: record.contentHash, + }) + () + } + + keyedRecords->Array.sort((a, b) => String.compare(a.key, b.key)) + + let generatedResPath = Path.join2(projectRoot, generatedResFile) + let generatedJsPath = Path.join2(projectRoot, generatedJsFile) + + await writeIfChangedString( + ~filePath=generatedResPath, + ~content=buildGeneratedRes(~keyedRecords, ~sourceByKey), + ) + await writeIfChangedString(~filePath=generatedJsPath, ~content=buildGeneratedJs(keyedRecords)) + + let manifestPath = Path.join2(outputRoot, manifestFile) + let manifestText = buildManifest(~mode, ~keyedRecords) + await writeIfChangedString(~filePath=manifestPath, ~content=manifestText) + + () +} + +let shouldTriggerSrcRebuild = filename => { + switch filename { + | None => false + | Some(value) if value == "" => false + | Some(value) => { + let normalized = toPosixPath(value) + normalized == "ResXClient.js" || normalized == "ResXClient.res" + } + } +} + +let makeDebouncedTask = (~task: unit => promise, ~waitMs) => { + let timer: ref> = ref(None) + let running = ref(false) + let pending = ref(false) + + let rec schedule = () => { + switch timer.contents { + | Some(existing) => Timers.clearTimeout(existing) + | None => () + } + + timer := Some(Timers.setTimeout(() => { + timer := None + let _ = runOnce() + }, waitMs)) + } + + and runOnce = async () => { + if running.contents { + pending := true + } else { + running := true + + switch await task() { + | () => () + | exception _ => () + } + + running := false + if pending.contents { + pending := false + schedule() + } + } + } + + schedule +} + +let watchDirectory = (~targetPath, ~onChange) => { + if !pathExistsSync(targetPath) { + None + } else { + switch fsWatch(targetPath, {recursive: true, encoding: "utf8"}, (_eventType, filename) => { + onChange(filename) + }) { + | watcher => Some(watcher) + | exception _ => None + } + } +} + +let startDevWatch = async (~root=?, ~clean=true, ~onBuildError=?) => { + let projectRoot = await resolveProjectRoot(~root?) + + let notifyError = switch onBuildError { + | Some(callback) => callback + | None => _ => () + } + + let rebuild = makeDebouncedTask(~task=async () => { + switch await buildAssets(~root=projectRoot, ~dev=true, ~clean) { + | () => () + | exception JsExn(error) => notifyError(error) + | exception _ => notifyError(JsError.make("Unknown build error")->JsError.toJsExn) + } + }, ~waitMs=120) + + let watchers: array = [] + + switch watchDirectory(~targetPath=Path.join2(projectRoot, "assets"), ~onChange=_ => rebuild()) { + | Some(value) => + watchers->Array.push(value) + () + | None => () + } + + switch watchDirectory(~targetPath=Path.join2(projectRoot, "public"), ~onChange=_ => rebuild()) { + | Some(value) => + watchers->Array.push(value) + () + | None => () + } + + switch watchDirectory(~targetPath=Path.join2(projectRoot, "src"), ~onChange=filename => { + if shouldTriggerSrcRebuild(filename) { + rebuild() + } + }) { + | Some(value) => + watchers->Array.push(value) + () + | None => () + } + + let watchPatterns = ["assets/**/*", "public/**/*", "src/ResXClient.*"] + let pollBusy = ref(false) + + let computeWatchSignature = async () => { + let normalized = scanGlobPatternsUniqueSorted( + ~patterns=watchPatterns, + ~cwd=projectRoot, + ~dot=true, + ) + + let parts: array = [] + + for i in 0 to normalized->Array.length - 1 { + let relPath = normalized->Array.getUnsafe(i) + let absPath = Path.join2(projectRoot, relPath) + + switch await fsStat(absPath) { + | stats => + parts->Array.push(`${relPath}:${Int.toString(stats.size)}:${Float.toString(stats.mtimeMs)}`) + () + | exception _ => () + } + } + + sha256HexString(parts->Array.join("\n")) + } + + let previousSignature = ref(await computeWatchSignature()) + + let poller = Timers.setInterval(() => { + if pollBusy.contents { + () + } else { + pollBusy := true + + let _ = ( + async () => { + switch await computeWatchSignature() { + | nextSignature => + if nextSignature != previousSignature.contents { + previousSignature := nextSignature + rebuild() + } + | exception _ => () + } + + pollBusy := false + } + )() + } + }, 500) + + { + close: () => { + Timers.clearInterval(poller) + + for i in 0 to watchers->Array.length - 1 { + switch watchers->Array.getUnsafe(i)->fsWatcherClose { + | () => () + | exception _ => () + } + } + }, + } +} + +let closeWatcher = watcher => watcher.close() + +let decodePathname = pathname => { + let current = ref(pathname) + let failed = ref(false) + + for _index in 0 to 3 { + if !failed.contents { + switch decodeURIComponent(current.contents) { + | decoded => + if decoded == current.contents { + () + } else { + current := decoded + } + | exception _ => failed := true + } + } + } + + if failed.contents { + None + } else { + Some(current.contents) + } +} + +let stripLeadingSlashes = value => { + let current = ref(value) + while current.contents->String.startsWith("/") { + current := current.contents->String.slice(~start=1) + } + current.contents +} + +let sanitizeUrlPath = pathname => { + switch decodePathname(pathname) { + | None => None + | Some(decoded) => + let normalizedSlashes = decoded->String.split("\\")->Array.join("/") + let startsWithSlash = normalizedSlashes->String.startsWith("/") + let withSlash = if startsWithSlash { + normalizedSlashes + } else { + `/${normalizedSlashes}` + } + let segments = withSlash->String.split("/") + + let blocked = ref(false) + for i in 0 to segments->Array.length - 1 { + if segments->Array.getUnsafe(i) == ".." { + blocked := true + } + } + + if blocked.contents { + None + } else { + let normalized = Path.Posix.normalize(withSlash) + if normalized->String.startsWith("/") { + Some(normalized) + } else { + None + } + } + } +} + +let getServeRoots = (~projectRoot, ~isDev): serveRoots => + if isDev { + { + managedAssetsRoot: Path.join2(projectRoot, "assets"), + staticRoot: Path.join2(projectRoot, "public"), + } + } else { + { + managedAssetsRoot: Path.join2(Path.join2(projectRoot, prodOutputDir), "assets"), + staticRoot: Path.join2(projectRoot, prodOutputDir), + } + } + +let resolveServedFilePath = async (~baseRoot, ~relativePath) => { + let relativePosix = stripLeadingSlashes(toPosixPath(relativePath)) + let target = Path.resolve([baseRoot, relativePosix]) + + if !isPathInside(~candidatePath=target, ~basePath=baseRoot) { + None + } else { + switch await fsStat(target) { + | stats => + if !(stats->Fs.Stats.isFile) { + None + } else { + switch await fsRealpath(target) { + | realTarget => + switch await fsRealpath(baseRoot) { + | realBase => + if isPathInside(~candidatePath=realTarget, ~basePath=realBase) { + Some(target) + } else { + None + } + | exception _ => None + } + | exception _ => None + } + } + | exception _ => None + } + } +} + +let jsSourceCandidatesFromRequest = relativePath => { + let parsed = Path.Posix.parse(relativePath) + let base = if parsed.dir != "" { + `${parsed.dir}/${parsed.name}` + } else { + parsed.name + } + jsEntryExtensions->Array.map(ext => `${base}${ext}`) +} + +let responseWithContentType = (~body, ~contentType) => { + let headers = Headers.make() + headers->Headers.set("Content-Type", contentType) + Response.makeWithHeaders(body, ~options={headers: headers}) +} + +let responseWithStatus = (~status) => Response.make("", ~options={status: status}) + +let fileResponseFromPath = filePath => Response.makeFromFile(Bun.file(filePath)) + +let bufferToUtf8String = buffer => buffer->Buffer.toStringWithEncoding(StringEncoding.utf8) + +let requestUrlPathname = request => request->Request.url->URL.make->URL.pathname + +let resolveDevManagedAssetResponse = async (~projectRoot, ~managedAssetsRoot, ~relativePath) => { + let extension = relativePath->Path.Posix.extname->String.toLowerCase + + if extension == ".js" { + let sourcePathRef = ref(await resolveServedFilePath(~baseRoot=managedAssetsRoot, ~relativePath)) + + if sourcePathRef.contents == None { + let sourceCandidates = jsSourceCandidatesFromRequest(relativePath) + + for i in 0 to sourceCandidates->Array.length - 1 { + if sourcePathRef.contents == None { + sourcePathRef := + ( + await resolveServedFilePath( + ~baseRoot=managedAssetsRoot, + ~relativePath=sourceCandidates->Array.getUnsafe(i), + ) + ) + } + } + } + + switch sourcePathRef.contents { + | None => None + | Some(sourcePath) => + let jsBytes = await buildJavaScript(~entryPath=sourcePath, ~projectRoot, ~mode="dev") + Some( + responseWithContentType( + ~body=bufferToUtf8String(jsBytes), + ~contentType="text/javascript; charset=utf-8", + ), + ) + } + } else if extension == ".css" { + switch await resolveServedFilePath(~baseRoot=managedAssetsRoot, ~relativePath) { + | None => None + | Some(sourcePath) => + let cssBytes = await buildCss(~entryPath=sourcePath, ~projectRoot, ~mode="dev") + Some( + responseWithContentType( + ~body=bufferToUtf8String(cssBytes), + ~contentType="text/css; charset=utf-8", + ), + ) + } + } else { + switch await resolveServedFilePath(~baseRoot=managedAssetsRoot, ~relativePath) { + | None => None + | Some(exactPath) => Some(fileResponseFromPath(exactPath)) + } + } +} + +let suspiciousRootNames = [ + "bin", + "etc", + "home", + "opt", + "private", + "proc", + "root", + "sbin", + "sys", + "tmp", + "usr", + "var", + "windows", +] + +let hasSuspiciousRoot = value => { + suspiciousRootNames->Array.includes(value) +} + +let serveStaticFileFromBuild = async (~request, ~projectRoot, ~isDev) => { + let pathname = requestUrlPathname(request) + + switch sanitizeUrlPath(pathname) { + | None => Some(responseWithStatus(~status=403)) + | Some(sanitized) => + let segments = sanitized->String.split("/") + let firstSegment = switch segments->Array.find(segment => segment != "") { + | Some(value) => value + | None => "" + } + + if hasSuspiciousRoot(firstSegment) { + Some(responseWithStatus(~status=403)) + } else if firstSegment == "src" { + let sourceRelative = sanitized->String.slice(~start=1) + let sourceCandidate = Path.resolve([projectRoot, sourceRelative]) + if ( + isPathInside(~candidatePath=sourceCandidate, ~basePath=projectRoot) && + pathExistsSync(sourceCandidate) + ) { + Some(responseWithStatus(~status=403)) + } else { + None + } + } else if sanitized == "/" { + None + } else { + let roots = getServeRoots(~projectRoot, ~isDev) + let managedAssetsRequest = sanitized == "/assets" || sanitized->String.startsWith("/assets/") + + if managedAssetsRequest { + let relative = if sanitized == "/assets" { + "" + } else { + sanitized->String.slice(~start=8) + } + + if isDev { + switch await resolveDevManagedAssetResponse( + ~projectRoot, + ~managedAssetsRoot=roots.managedAssetsRoot, + ~relativePath=relative, + ) { + | Some(response) => Some(response) + | None => Some(responseWithStatus(~status=404)) + | exception exn => + Console.error( + `[resx] failed to serve dev asset /assets/${relative}\n${exn->errorToString}`, + ) + Some(responseWithStatus(~status=500)) + } + } else { + switch await resolveServedFilePath( + ~baseRoot=roots.managedAssetsRoot, + ~relativePath=relative, + ) { + | Some(filePath) => Some(fileResponseFromPath(filePath)) + | None => Some(responseWithStatus(~status=404)) + } + } + } else { + let relative = sanitized->String.slice(~start=1) + switch await resolveServedFilePath(~baseRoot=roots.staticRoot, ~relativePath=relative) { + | Some(filePath) => Some(fileResponseFromPath(filePath)) + | None => None + } + } + } + } +} diff --git a/src/Assets.js b/src/Assets.js new file mode 100644 index 0000000..1c59d7b --- /dev/null +++ b/src/Assets.js @@ -0,0 +1,68 @@ +// Generated by ReScript, PLEASE EDIT WITH CARE +'use strict'; + +let AssetPipeline$ResX = require("./AssetPipeline.js"); + +let isDevelopment = process.env.NODE_ENV !== "production"; + +let startDevPromise = { + contents: undefined +}; + +let startDevWatcher = { + contents: undefined +}; + +let startedRoot = { + contents: undefined +}; + +function closeWatcher() { + let watcher = startDevWatcher.contents; + if (watcher !== undefined) { + AssetPipeline$ResX.closeWatcher(watcher); + startDevWatcher.contents = undefined; + return; + } +} + +async function startDev(root, param) { + if (!isDevelopment) { + return; + } + let resolvedRoot = await AssetPipeline$ResX.resolveProjectRoot(root); + let existingPromise = startDevPromise.contents; + if (existingPromise !== undefined) { + let existingRoot = startedRoot.contents; + if (existingRoot !== undefined && existingRoot !== resolvedRoot) { + console.warn(`[resx] startDev() already initialized for ` + existingRoot + `; ignoring additional root ` + resolvedRoot); + } + return await existingPromise; + } + let startupPromise = (async () => { + let watcher = await AssetPipeline$ResX.startDevWatch(resolvedRoot, true, error => { + console.error(`[resx] dev asset rebuild failed\n` + String(error)); + }); + startDevWatcher.contents = watcher; + try { + await AssetPipeline$ResX.buildAssets(resolvedRoot, true, true); + } catch (exn) { + closeWatcher(); + throw exn; + } + startedRoot.contents = resolvedRoot; + })(); + startDevPromise.contents = startupPromise; + try { + await startupPromise; + return; + } catch (exn) { + startDevPromise.contents = undefined; + startedRoot.contents = undefined; + closeWatcher(); + throw exn; + } +} + +exports.startDev = startDev; +/* isDevelopment Not a pure module */ diff --git a/src/Assets.res b/src/Assets.res new file mode 100644 index 0000000..bc4eea7 --- /dev/null +++ b/src/Assets.res @@ -0,0 +1,68 @@ +external process: 'process = "process" +external consoleWarn: string => unit = "console.warn" +external consoleError: string => unit = "console.error" + +let isDevelopment = process["env"]["NODE_ENV"] !== "production" + +let startDevPromise: ref>> = ref(None) +let startDevWatcher: ref> = ref(None) +let startedRoot: ref> = ref(None) + +let closeWatcher = () => + switch startDevWatcher.contents { + | Some(watcher) => + watcher->AssetPipeline.closeWatcher + startDevWatcher := None + | None => () + } + +let startDev = async (~root=?, ()) => { + if !isDevelopment { + () + } else { + let resolvedRoot = await AssetPipeline.resolveProjectRoot(~root?) + + switch startDevPromise.contents { + | Some(existingPromise) => + switch startedRoot.contents { + | Some(existingRoot) if existingRoot !== resolvedRoot => + consoleWarn( + `[resx] startDev() already initialized for ${existingRoot}; ignoring additional root ${resolvedRoot}`, + ) + | _ => () + } + + await existingPromise + | None => + let startupPromise = (async () => { + let watcher = await AssetPipeline.startDevWatch( + ~root=resolvedRoot, + ~clean=true, + ~onBuildError=error => + consoleError(`[resx] dev asset rebuild failed\n${error->AssetPipeline.errorToString}`), + ) + startDevWatcher := Some(watcher) + + switch await AssetPipeline.buildAssets(~root=resolvedRoot, ~dev=true, ~clean=true) { + | () => () + | exception exn => + closeWatcher() + throw(exn) + } + + startedRoot := Some(resolvedRoot) + })() + + startDevPromise := Some(startupPromise) + + switch await startupPromise { + | () => () + | exception exn => + startDevPromise := None + startedRoot := None + closeWatcher() + throw(exn) + } + } + } +} diff --git a/src/Assets.resi b/src/Assets.resi new file mode 100644 index 0000000..1cfb20b --- /dev/null +++ b/src/Assets.resi @@ -0,0 +1 @@ +let startDev: (~root: string=?, unit) => promise diff --git a/src/BunUtils.js b/src/BunUtils.js index 55dd3bc..97d90a9 100644 --- a/src/BunUtils.js +++ b/src/BunUtils.js @@ -1,74 +1,31 @@ // Generated by ReScript, PLEASE EDIT WITH CARE 'use strict'; -let FastGlob = require("fast-glob"); +let AssetPipeline$ResX = require("./AssetPipeline.js"); let isDev = process.env.NODE_ENV !== "production"; -async function loadStaticFiles(root) { - return await FastGlob.glob(isDev ? [ - "public/**/*", - "assets/**/*" - ] : ["dist/**/*"], { - dot: true, - cwd: root !== undefined ? root : process.cwd() - }); -} - -let staticFiles = { +let projectRootPromise = { contents: undefined }; -async function serveStaticFile(request) { - let s = staticFiles.contents; - let staticFiles$1; - if (s !== undefined) { - staticFiles$1 = s; - } else { - let files = await loadStaticFiles(undefined); - let files$1 = new Map(files.map(f => [ - isDev ? ( - f.startsWith("public/") ? f.slice(7) : f - ) : ( - f.startsWith("dist/") ? f.slice(5) : f - ), - f - ])); - staticFiles.contents = files$1; - staticFiles$1 = files$1; - } - let url = new URL(request.url); - let pathname = url.pathname; - let path = pathname.split("/").filter(p => p !== ""); - let joined = path.join("/"); - let fileLoc = staticFiles$1.get(joined); - if (fileLoc === undefined) { - return; +function getProjectRoot() { + let existingPromise = projectRootPromise.contents; + if (existingPromise !== undefined) { + return existingPromise; } - let bunFile = Bun.file("./" + fileLoc); - let match = bunFile.size; - return match !== 0 ? new Response(bunFile) : new Response("", { - status: 404 - }); + let nextPromise = AssetPipeline$ResX.resolveProjectRoot(undefined); + projectRootPromise.contents = nextPromise; + return nextPromise; +} + +async function serveStaticFile(request) { + let projectRoot = await getProjectRoot(); + return await AssetPipeline$ResX.serveStaticFileFromBuild(request, projectRoot, isDev); } function runDevServer(port) { - Bun.serve({ - development: true, - port: port + 1 | 0, - fetch: async (request, server) => { - if (server.upgrade(request)) { - return undefined; - } else { - return new Response("", { - status: 404 - }); - } - }, - websocket: { - open: _v => {} - } - }); + } function copy(search) { diff --git a/src/BunUtils.res b/src/BunUtils.res index b89ba94..b92c514 100644 --- a/src/BunUtils.res +++ b/src/BunUtils.res @@ -2,95 +2,25 @@ external process: 'process = "process" let isDev = process["env"]["NODE_ENV"] !== "production" -type globConfig = { - dot?: bool, - cwd?: string, -} - -@module("fast-glob") -external glob: (array, globConfig) => promise> = "glob" - -let loadStaticFiles = async (~root=?) => { - await glob( - switch isDev { - | true => ["public/**/*", "assets/**/*"] - | false => ["dist/**/*"] - }, - { - dot: true, - cwd: switch root { - | None => process["cwd"]() - | Some(cwd) => cwd - }, - }, - ) -} - -let staticFiles = ref(None) - -let serveStaticFile = async request => { - open Bun +let projectRootPromise: ref>> = ref(None) - let staticFiles = switch staticFiles.contents { +let getProjectRoot = () => + switch projectRootPromise.contents { + | Some(existingPromise) => existingPromise | None => - let files = await loadStaticFiles() - let files = - files - ->Array.map(f => { - ( - switch isDev { - | true if f->String.startsWith("public/") => f->String.slice(~start=7) - | false if f->String.startsWith("dist/") => f->String.slice(~start=5) - | _ => f - }, - f, - ) - }) - ->Map.fromArray - staticFiles := Some(files) - files - | Some(s) => s + let nextPromise = AssetPipeline.resolveProjectRoot() + projectRootPromise := Some(nextPromise) + nextPromise } - let url = request->Request.url->URL.make - let pathname = url->URL.pathname - - let path = pathname->String.split("/")->Array.filter(p => p !== "") - let joined = path->Array.join("/") - - switch staticFiles->Map.get(joined) { - | None => None - | Some(fileLoc) => - let bunFile = Bun.file("./" ++ fileLoc) - - Some( - switch bunFile->BunFile.size { - | 0. => Response.make("", ~options={status: 404}) - | _ => Response.makeFromFile(bunFile) - }, - ) - } +let serveStaticFile = async request => { + let projectRoot = await getProjectRoot() + await AssetPipeline.serveStaticFileFromBuild(~request, ~projectRoot, ~isDev) } -let runDevServer = (~port) => { - let _devServer = Bun.serveWithWebSocket({ - port: port + 1, - development: true, - websocket: { - open_: _v => { - () - }, - }, - fetch: async (request, server) => { - open Bun - - if server->Server.upgrade(request) { - Response.defer - } else { - Response.make("", ~options={status: 404}) - } - }, - }) +let runDevServer = (~port: int) => { + let _unusedPort = port + () } module URLSearchParams = { diff --git a/src/Cli.js b/src/Cli.js new file mode 100644 index 0000000..45ac3cf --- /dev/null +++ b/src/Cli.js @@ -0,0 +1,302 @@ +// Generated by ReScript, PLEASE EDIT WITH CARE +'use strict'; + +let AssetPipeline$ResX = require("./AssetPipeline.js"); +let Primitive_exceptions = require("@rescript/runtime/lib/js/Primitive_exceptions.js"); + +let UsageError = /* @__PURE__ */Primitive_exceptions.create("Cli-ResX.UsageError"); + +function printRootHelp() { + console.log(`Usage: resx + +Commands: + assets build Build managed assets + +Run \`resx assets build --help\` for command-specific options.`); +} + +function printAssetsBuildHelp() { + console.log(`Usage: resx assets build [options] + +Options: + --dev Build dev assets into .resx/dev + --watch Watch for changes (requires --dev) + --root Project root directory (default: current working directory) + --clean Clean output directory before build (default) + --no-clean Do not clean output directory before build + -h, --help Show this help`); +} + +function parseAssetsBuildArgs(argv) { + let options = { + dev: false, + watch: false, + clean: true, + root: undefined, + help: false + }; + let i = 0; + while (i < argv.length) { + let arg = argv[i]; + let exit = 0; + switch (arg) { + case "--clean" : + let init = options; + options = { + dev: init.dev, + watch: init.watch, + clean: true, + root: init.root, + help: init.help + }; + break; + case "--dev" : + let init$1 = options; + options = { + dev: true, + watch: init$1.watch, + clean: init$1.clean, + root: init$1.root, + help: init$1.help + }; + break; + case "--no-clean" : + let init$2 = options; + options = { + dev: init$2.dev, + watch: init$2.watch, + clean: false, + root: init$2.root, + help: init$2.help + }; + break; + case "--root" : + let nextIndex = i + 1 | 0; + let value = argv[nextIndex]; + if (value !== undefined) { + if (value.startsWith("--")) { + throw { + RE_EXN_ID: UsageError, + _1: "Missing value for --root", + Error: new Error() + }; + } + let init$3 = options; + options = { + dev: init$3.dev, + watch: init$3.watch, + clean: init$3.clean, + root: value, + help: init$3.help + }; + i = nextIndex; + } else { + throw { + RE_EXN_ID: UsageError, + _1: "Missing value for --root", + Error: new Error() + }; + } + break; + case "--watch" : + let init$4 = options; + options = { + dev: init$4.dev, + watch: true, + clean: init$4.clean, + root: init$4.root, + help: init$4.help + }; + break; + case "--help" : + case "-h" : + exit = 1; + break; + default: + throw { + RE_EXN_ID: UsageError, + _1: `Unknown option: ` + arg, + Error: new Error() + }; + } + if (exit === 1) { + let init$5 = options; + options = { + dev: init$5.dev, + watch: init$5.watch, + clean: init$5.clean, + root: init$5.root, + help: true + }; + } + i = i + 1 | 0; + }; + if (options.watch && !options.dev) { + throw { + RE_EXN_ID: UsageError, + _1: "Invalid option combination: --watch requires --dev", + Error: new Error() + }; + } + return options; +} + +async function runAssetsBuild(parsed) { + if (parsed.help) { + console.log(`Usage: resx assets build [options] + +Options: + --dev Build dev assets into .resx/dev + --watch Watch for changes (requires --dev) + --root Project root directory (default: current working directory) + --clean Clean output directory before build (default) + --no-clean Do not clean output directory before build + -h, --help Show this help`); + return { + TAG: "ExitCode", + _0: 0 + }; + } + let root = parsed.root; + if (parsed.watch) { + let watcher = await AssetPipeline$ResX.startDevWatch(root, parsed.clean, error => { + console.error(String(error)); + }); + try { + await AssetPipeline$ResX.buildAssets(root, parsed.dev, parsed.clean); + } catch (exn) { + AssetPipeline$ResX.closeWatcher(watcher); + throw exn; + } + process.on("SIGINT", () => { + AssetPipeline$ResX.closeWatcher(watcher); + process.exit(0); + }); + process.on("SIGTERM", () => { + AssetPipeline$ResX.closeWatcher(watcher); + process.exit(0); + }); + return "Watching"; + } + await AssetPipeline$ResX.buildAssets(root, parsed.dev, parsed.clean); + return { + TAG: "ExitCode", + _0: 0 + }; +} + +async function run(argv) { + let command = argv[0]; + if (command !== undefined) { + switch (command) { + case "--help" : + case "-h" : + console.log(`Usage: resx + +Commands: + assets build Build managed assets + +Run \`resx assets build --help\` for command-specific options.`); + return { + TAG: "ExitCode", + _0: 0 + }; + case "assets" : + let other = argv[1]; + if (other !== undefined) { + switch (other) { + case "--help" : + case "-h" : + console.log(`Usage: resx assets build [options] + +Options: + --dev Build dev assets into .resx/dev + --watch Watch for changes (requires --dev) + --root Project root directory (default: current working directory) + --clean Clean output directory before build (default) + --no-clean Do not clean output directory before build + -h, --help Show this help`); + return { + TAG: "ExitCode", + _0: 0 + }; + case "build" : + let parsed = parseAssetsBuildArgs(argv.slice(2)); + return await runAssetsBuild(parsed); + default: + throw { + RE_EXN_ID: UsageError, + _1: `Unknown assets subcommand: ` + other, + Error: new Error() + }; + } + } else { + console.log(`Usage: resx assets build [options] + +Options: + --dev Build dev assets into .resx/dev + --watch Watch for changes (requires --dev) + --root Project root directory (default: current working directory) + --clean Clean output directory before build (default) + --no-clean Do not clean output directory before build + -h, --help Show this help`); + return { + TAG: "ExitCode", + _0: 0 + }; + } + default: + throw { + RE_EXN_ID: UsageError, + _1: `Unknown command: ` + command, + Error: new Error() + }; + } + } else { + console.log(`Usage: resx + +Commands: + assets build Build managed assets + +Run \`resx assets build --help\` for command-specific options.`); + return { + TAG: "ExitCode", + _0: 0 + }; + } +} + +async function runFromArgv() { + let argv = process.argv.slice(2); + let code; + try { + code = await run(argv); + } catch (raw_message) { + let message = Primitive_exceptions.internalToException(raw_message); + if (message.RE_EXN_ID === UsageError) { + console.error(message._1); + console.error("Run `resx assets build --help` for usage details."); + return process.exit(2); + } else if (message.RE_EXN_ID === "JsExn") { + console.error(String(message._1)); + return process.exit(1); + } else { + console.error("Unknown error"); + return process.exit(1); + } + } + if (typeof code !== "object") { + return; + } else { + return process.exit(code._0); + } +} + +exports.UsageError = UsageError; +exports.printRootHelp = printRootHelp; +exports.printAssetsBuildHelp = printAssetsBuildHelp; +exports.parseAssetsBuildArgs = parseAssetsBuildArgs; +exports.runAssetsBuild = runAssetsBuild; +exports.run = run; +exports.runFromArgv = runFromArgv; +/* AssetPipeline-ResX Not a pure module */ diff --git a/src/Cli.res b/src/Cli.res new file mode 100644 index 0000000..2be2ff9 --- /dev/null +++ b/src/Cli.res @@ -0,0 +1,152 @@ +exception UsageError(string) + +type buildCliOptions = { + dev: bool, + watch: bool, + clean: bool, + root: option, + help: bool, +} + +type runResult = + | ExitCode(int) + | Watching + +external process: 'process = "process" +external processExit: int => 'a = "process.exit" +external processOn: (string, unit => unit) => unit = "process.on" +external consoleLog: string => unit = "console.log" +external consoleError: string => unit = "console.error" + +let printRootHelp = () => + consoleLog(`Usage: resx + +Commands: + assets build Build managed assets + +Run \`resx assets build --help\` for command-specific options.`) + +let printAssetsBuildHelp = () => + consoleLog(`Usage: resx assets build [options] + +Options: + --dev Build dev assets into .resx/dev + --watch Watch for changes (requires --dev) + --root Project root directory (default: current working directory) + --clean Clean output directory before build (default) + --no-clean Do not clean output directory before build + -h, --help Show this help`) + +let parseAssetsBuildArgs = argv => { + let options = ref({dev: false, watch: false, clean: true, root: None, help: false}) + let i = ref(0) + + while i.contents < argv->Array.length { + let arg = argv->Array.getUnsafe(i.contents) + + switch arg { + | "--dev" => options := {...options.contents, dev: true} + | "--watch" => options := {...options.contents, watch: true} + | "--clean" => options := {...options.contents, clean: true} + | "--no-clean" => options := {...options.contents, clean: false} + | "--root" => + let nextIndex = i.contents + 1 + switch argv->Array.get(nextIndex) { + | Some(value) if !(value->String.startsWith("--")) => + options := {...options.contents, root: Some(value)} + i := nextIndex + | _ => throw(UsageError("Missing value for --root")) + } + | "--help" | "-h" => options := {...options.contents, help: true} + | unknown => throw(UsageError(`Unknown option: ${unknown}`)) + } + + i := i.contents + 1 + } + + if options.contents.watch && !options.contents.dev { + throw(UsageError("Invalid option combination: --watch requires --dev")) + } + + options.contents +} + +let runAssetsBuild = async parsed => { + if parsed.help { + printAssetsBuildHelp() + ExitCode(0) + } else { + let root = parsed.root + + if !parsed.watch { + await AssetPipeline.buildAssets(~root?, ~dev=parsed.dev, ~clean=parsed.clean) + ExitCode(0) + } else { + let watcher = await AssetPipeline.startDevWatch( + ~root?, + ~clean=parsed.clean, + ~onBuildError=error => consoleError(error->AssetPipeline.errorToString), + ) + + switch await AssetPipeline.buildAssets(~root?, ~dev=parsed.dev, ~clean=parsed.clean) { + | () => () + | exception exn => + watcher->AssetPipeline.closeWatcher + throw(exn) + } + + processOn("SIGINT", () => { + watcher->AssetPipeline.closeWatcher + processExit(0) + }) + + processOn("SIGTERM", () => { + watcher->AssetPipeline.closeWatcher + processExit(0) + }) + + Watching + } + } +} + +let run = async argv => { + switch argv->Array.get(0) { + | None => + printRootHelp() + ExitCode(0) + | Some("--help") | Some("-h") => + printRootHelp() + ExitCode(0) + | Some("assets") => + switch argv->Array.get(1) { + | None | Some("--help") | Some("-h") => + printAssetsBuildHelp() + ExitCode(0) + | Some("build") => + let parsed = parseAssetsBuildArgs(argv->Array.slice(~start=2)) + await runAssetsBuild(parsed) + | Some(other) => throw(UsageError(`Unknown assets subcommand: ${other}`)) + } + | Some(command) => throw(UsageError(`Unknown command: ${command}`)) + } +} + +let runFromArgv = async () => { + let argv = process["argv"]->Array.slice(~start=2) + + switch await run(argv) { + | ExitCode(code) => processExit(code) + | Watching => () + | exception UsageError(message) => + consoleError(message) + consoleError("Run `resx assets build --help` for usage details.") + processExit(2) + | exception JsExn(error) => + consoleError(error->AssetPipeline.errorToString) + processExit(1) + | exception _ => + consoleError("Unknown error") + processExit(1) + } +} diff --git a/src/Dev.js b/src/Dev.js index 43cce2b..7752850 100644 --- a/src/Dev.js +++ b/src/Dev.js @@ -1,129 +1,12 @@ // Generated by ReScript, PLEASE EDIT WITH CARE 'use strict'; -let Hjsx$ResX = require("./Hjsx.js"); -let BunUtils$ResX = require("./BunUtils.js"); - -function getScript(port) { - return `(() => { -let hasMadeInitialConnection = false; -let timeout; -let socket = null; -let reconnectInterval; -let debugging = true; - -let debug = (...msg) => { - if (debugging) { - console.log(...msg) - } -} - -function reload() { - clearTimeout(timeout) - timeout = setTimeout(() => { - window.location.reload() - }, 200) -} - -function connect() { - clearInterval(reconnectInterval) - reconnectInterval = setInterval(() => { - if (socket == null || socket.readyState === 2 || socket.readyState === 3) { - bootSocket() - } else if (socket != null && (socket.readyState === 0 || socket.readyState === 1)) { - clearInterval(reconnectInterval) - } - }, 200) -} - -function updateContent() { - fetch(document.location.href).then(async res => { - let text = await res.text() - try { - let domParser = new DOMParser() - let fromDom = document.documentElement - let toDom = domParser.parseFromString(text, "text/html").querySelector("html") - morphdom(fromDom, toDom, { - onBeforeElUpdated: function(fromEl, toEl) { - if (fromEl.isEqualNode(toEl)) { - return false; - } - - if (fromEl.tagName === 'INPUT') { - if (fromEl.type === 'checkbox' || fromEl.type === 'radio') { - toEl.checked = fromEl.checked; - } else { - toEl.value = fromEl.value; - } - } else if (fromEl.tagName === 'TEXTAREA') { - toEl.value = fromEl.value; - } - - if (fromEl.tagName === 'SELECT') { - toEl.selectedIndex = fromEl.selectedIndex; - } - - return true; - } - }) - debug("[dev] Content reloaded.") - } catch(e) { - console.warn("[dev] Error morphing DOM. Doing full reload.") - console.error(e) - document.documentElement.innerHTML = text - - } - }) -} - -function bootSocket() { - socket = new WebSocket("ws://localhost:` + (port + 1 | 0).toString() + `") - socket.addEventListener("close", event => { - debug("[dev] Server restarting") - if (event.isTrusted) { - socket = null - connect() - } - }) - socket.addEventListener("open", event => { - debug("[dev] Server connection opened.") - if (hasMadeInitialConnection) { - updateContent() - } - hasMadeInitialConnection = true - }) -} - -bootSocket() -})() -`; -} function Dev(props) { - let __port = props.port; - let port = __port !== undefined ? __port : 4444; - if (BunUtils$ResX.isDev) { - return [ - Hjsx$ResX.Elements.jsx("script", { - dangerouslySetInnerHTML: { - __html: getScript(port) - } - }), - Hjsx$ResX.Elements.jsx("script", { - src: "https://unpkg.com/morphdom/dist/morphdom-umd.js" - }), - Hjsx$ResX.Elements.jsx("script", { - src: "http://localhost:9000/@vite/client", - type: "module" - }) - ]; - } else { - return null; - } + return null; } let make = Dev; -exports.getScript = getScript; exports.make = make; -/* Hjsx-ResX Not a pure module */ +/* No side effect */ diff --git a/src/Dev.res b/src/Dev.res index e736e86..7c8642f 100644 --- a/src/Dev.res +++ b/src/Dev.res @@ -1,108 +1,7 @@ @@jsxConfig({module_: "Hjsx"}) -let getScript = (~port) => - `(() => { -let hasMadeInitialConnection = false; -let timeout; -let socket = null; -let reconnectInterval; -let debugging = true; - -let debug = (...msg) => { - if (debugging) { - console.log(...msg) - } -} - -function reload() { - clearTimeout(timeout) - timeout = setTimeout(() => { - window.location.reload() - }, 200) -} - -function connect() { - clearInterval(reconnectInterval) - reconnectInterval = setInterval(() => { - if (socket == null || socket.readyState === 2 || socket.readyState === 3) { - bootSocket() - } else if (socket != null && (socket.readyState === 0 || socket.readyState === 1)) { - clearInterval(reconnectInterval) - } - }, 200) -} - -function updateContent() { - fetch(document.location.href).then(async res => { - let text = await res.text() - try { - let domParser = new DOMParser() - let fromDom = document.documentElement - let toDom = domParser.parseFromString(text, "text/html").querySelector("html") - morphdom(fromDom, toDom, { - onBeforeElUpdated: function(fromEl, toEl) { - if (fromEl.isEqualNode(toEl)) { - return false; - } - - if (fromEl.tagName === 'INPUT') { - if (fromEl.type === 'checkbox' || fromEl.type === 'radio') { - toEl.checked = fromEl.checked; - } else { - toEl.value = fromEl.value; - } - } else if (fromEl.tagName === 'TEXTAREA') { - toEl.value = fromEl.value; - } - - if (fromEl.tagName === 'SELECT') { - toEl.selectedIndex = fromEl.selectedIndex; - } - - return true; - } - }) - debug("[dev] Content reloaded.") - } catch(e) { - console.warn("[dev] Error morphing DOM. Doing full reload.") - console.error(e) - document.documentElement.innerHTML = text - - } - }) -} - -function bootSocket() { - socket = new WebSocket("ws://localhost:${(port + 1)->Int.toString}") - socket.addEventListener("close", event => { - debug("[dev] Server restarting") - if (event.isTrusted) { - socket = null - connect() - } - }) - socket.addEventListener("open", event => { - debug("[dev] Server connection opened.") - if (hasMadeInitialConnection) { - updateContent() - } - hasMadeInitialConnection = true - }) -} - -bootSocket() -})() -` - @jsx.component let make = (~port=4444) => { - if BunUtils.isDev { - [ - `; +} + +async function boot() { + const startDevAvailable = await maybeStartDevAssets(); + + const server = Bun.serve({ + port, + development: BunUtils.isDev, + fetch: async request => { + const url = new URL(request.url); + + if (url.pathname === "/healthz") { + return Response.json({ + ok: true, + env: BunUtils.isDev ? "development" : "production", + startDevAvailable, + }); + } + + const staticResponse = await BunUtils.serveStaticFile(request); + if (staticResponse != null) { + return staticResponse; + } + + return new Response(htmlPage(getGeneratedAssets()), { + headers: {"Content-Type": "text/html; charset=utf-8"}, + }); + }, + }); + + const actualPort = server.port; + console.log(`[bun-assets-smoke] listening on http://localhost:${actualPort}`); +} + +void boot(); diff --git a/test/fixtures/bun-assets-smoke/src/App.start-dev-twice.js b/test/fixtures/bun-assets-smoke/src/App.start-dev-twice.js new file mode 100644 index 0000000..044bcc3 --- /dev/null +++ b/test/fixtures/bun-assets-smoke/src/App.start-dev-twice.js @@ -0,0 +1,85 @@ +const path = require("path"); +const fs = require("fs"); + +function requireResXModule(moduleName) { + const requested = path.posix.join("src", moduleName); + const envRoot = process.env.RESX_REPO_ROOT; + + if (typeof envRoot === "string" && envRoot.length > 0) { + const fromEnv = path.join(envRoot, requested); + if (fs.existsSync(fromEnv)) { + return require(fromEnv); + } + } + + const fromRepoLayout = path.resolve(__dirname, "../../../../", requested); + if (fs.existsSync(fromRepoLayout)) { + return require(fromRepoLayout); + } + + return require(`rescript-x/${requested}`); +} + +const BunUtils = requireResXModule("BunUtils.js"); + +let AssetsApi = null; +try { + AssetsApi = requireResXModule("Assets.js"); +} catch { + AssetsApi = null; +} + +const port = Number.parseInt(process.env.PORT || "4460", 10); + +function getGeneratedAssets() { + const generatedPath = path.resolve(__dirname, "__generated__/res-x-assets.js"); + delete require.cache[generatedPath]; + try { + return require(generatedPath).assets; + } catch { + return {styles_css: "/assets/styles.css", resXClient_js: "/assets/resx-client.js"}; + } +} + +async function maybeStartDevAssetsTwice() { + if (!BunUtils.isDev) return false; + if (AssetsApi == null || typeof AssetsApi.startDev !== "function") return false; + await Promise.all([AssetsApi.startDev(), AssetsApi.startDev()]); + return true; +} + +async function boot() { + const startDevAvailable = await maybeStartDevAssetsTwice(); + + const server = Bun.serve({ + port, + development: BunUtils.isDev, + fetch: async request => { + const url = new URL(request.url); + + if (url.pathname === "/healthz") { + return Response.json({ + ok: true, + env: BunUtils.isDev ? "development" : "production", + startDevAvailable, + }); + } + + const staticResponse = await BunUtils.serveStaticFile(request); + if (staticResponse != null) return staticResponse; + + const assets = getGeneratedAssets(); + const css = assets.styles_css || "/assets/styles.css"; + return new Response( + `start-dev-twice`, + { + headers: {"Content-Type": "text/html; charset=utf-8"}, + }, + ); + }, + }); + + console.log(`[bun-assets-smoke] start-dev-twice on http://localhost:${server.port}`); +} + +void boot(); diff --git a/test/fixtures/bun-assets-smoke/src/AppView.res b/test/fixtures/bun-assets-smoke/src/AppView.res new file mode 100644 index 0000000..305bf21 --- /dev/null +++ b/test/fixtures/bun-assets-smoke/src/AppView.res @@ -0,0 +1,13 @@ +@@jsxConfig({module_: "Hjsx"}) + +@jsx.component +let make = () => + + + +