Skip to content

packaging.md

Latest commit

 

History

History
190 lines (134 loc) · 38.1 KB

packaging.md

File metadata and controls

190 lines (134 loc) · 38.1 KB
Raw

Packaging

One build per platform (Linux ships two packages) carries everything the desktop app needs to run offline: the API and the worker frozen into PyInstaller binaries, the renderer, the embedding, voice and parser model packs, the llama.cpp, sd.cpp and audio.cpp runtimes, and opencode with ripgrep for the agent. PyInstaller follows only import statements, so anything the app reaches by a path or a string must be named in a spec, and each omission shows up only in a frozen build on a clean machine; the specs name those files, and packaging tests freeze real binaries to catch the ones that slip. Generation weights are never bundled; the app downloads them when the user asks (egress).

Code: surfsense_local/backend/bundling/, surfsense_local/backend/scripts/build_binaries.py, surfsense_local/backend/scripts/notices/, surfsense_local/electron/electron-builder.yml, surfsense_local/electron/scripts/, .github/workflows/release-local.yml, surfsense_local/backend/tests/packaging/ Decisions: ADR 0021, ADR 0012, ADR 0056

How to cut a release is in surfsense_local/RELEASE.md; how a release reaches installed apps is in updates.

An installed app

The asar holds only the Electron main and preload bundles. Everything else rides beside it in resources/, where the main process spawns the binaries and loads the SPA by path:

extraResources target Staged in Holds
backend/api, backend/worker backend/dist/ the frozen onedir binaries
frontend/dist frontend/dist the Vite SPA, loaded from disk
models backend/models the embedding model and the Docling parser pack
llamacpp electron/llamacpp llama-server and the libraries it links
sdcpp electron/sdcpp sd-server, for local image generation
audiocpp electron/audiocpp audiocpp_server, its libraries, the curated model specs and eSpeak-ng, for podcast voices and speech recognition
opencode electron/opencode opencode and rg, for the agent, with their licences; absent only from a build staged with SURFSENSE_LOCAL_OPENCODE_ENABLED=0
licenses electron/licenses THIRD_PARTY_NOTICES.json and .txt, the licence notices for everything above (third-party notices)

The app icon lives in electron/build/icons/: packaged/ holds the .icns, .ico and .png that electron-builder.yml names per OS, and dev/ a variant with a "DEV" badge, which electron/src/main/dev-app-identity.ts sets on the Dock, taskbar, window and About panel only while unpackaged, alongside the name "SurfSense Dev", because development runs inside Electron's own bundle and would otherwise show Electron's icon. The macOS menu bar name and the About panel icon stay Electron’s in development; only packaging changes them. Artwork on Windows and Linux fills its canvas; on macOS it sits at 824 of 1024 px with a transparent margin, Apple's icon grid, so icon.icns and dev/icon-macos.png carry that margin and the .ico and .png files do not.

Packaged, Electron runs resources/backend/api/api and one worker process per queue, ingest and studio, and gives both SURFSENSE_LOCAL_MODELS_DIR pointing at resources/models and HF_HUB_OFFLINE=1 (electron/src/main/sidecars/python.ts). In development the same sidecars run on the backend's .venv interpreter (overview.md says why not uv run). The Studio worker starts its own binary again, as worker --run-document-script <folder>, for each document script it runs, and for each analysis script the agent's surfsense_analyze_data queues, so a script imports only what the frozen worker carries: python-docx, ReportLab, python-pptx, xlsxwriter, openpyxl, matplotlib, Pillow, numpy and pandas (studio). The PDF tools run in the API on pypdf, which the API imports statically (agent).

Freezing the backend

scripts/build_binaries.py runs PyInstaller on bundling/api.spec and bundling/worker.spec into dist/api/ and dist/worker/. PyInstaller cannot cross-compile, so each platform builds on itself. The script refuses a Python whose sqlite3 lacks enable_load_extension: python.org's macOS builds omit it, and the frozen app would then die on its first migration, which is why CI freezes with uv's managed Python (UV_PYTHON_PREFERENCE: only-managed).

Both specs take the database inputs from bundling/common.py, so a dependency bump cannot fix one binary and miss the other:

  • collect_all("sqlite_vec") ships vec0, which sqlite_vec.load() finds by the package's own path. Every connection loads it, so without it the migration on launch is the first thing to fail.
  • alembic/ ships as data. shared/migrations.py resolves it from its own __file__, which PyInstaller sets correctly. env.py is data rather than analysed code, so alembic.context, alembic.runtime.migration and alembic.runtime.environment are hidden imports.

What else each spec names, and why the analyser cannot find it on its own:

Spec Adds Because
api.spec collect_submodules("uvicorn") uvicorn loads its loop, protocol and lifespan implementations by string
api.spec onnxruntime and tokenizers libraries the query encoder's native libraries load from C
api.spec the local model manifest catalog/local/manifest/models.json, the remote model manifest catalog/remote/manifest/models.json, the chat prompts, the agent's prompt, the agent's skills/*/SKILL.md read by path or through importlib.resources; opencode's configuration names the skills folder by path (agent)
api.spec excludes Docling, torch, torchvision, transformers, pandas, scipy, OpenCV and lameenc only the worker parses files and encodes podcasts, and the analyser cannot tell these are optional; the API's voice check keeps the server's WAV
worker.spec the local model manifest, the remote model manifest Studio finds its chosen image and audio models through the local catalog, and classifies a remote model through the same discovery the API uses; both files are read by path
worker.spec Docling and its packages, RapidOCR, transformers, torchvision lazy and native imports Docling reaches only on the first PDF
worker.spec python-docx, python-pptx, xlsxwriter, reportlab, openpyxl the Office formats and document scripts run model-written code that imports them, so no static import exists; openpyxl, named in pyproject.toml though Docling brings it too, also reads every document script's workbook for its summary
worker.spec matplotlib.pyplot, matplotlib.backends.backend_pdf, matplotlib.backends.backend_svg, with the matplotlib hook set to the Agg backend document scripts draw charts, and nothing imports matplotlib statically. Naming pyplot runs PyInstaller's hook, which adds mpl-data and the backend chosen; a chart saved as PDF or SVG, or PdfPages, imports its canvas by name, which the hook does not follow. Agg alone keeps Tk's GUI backend out, since scripts run with MPLBACKEND=Agg
worker.spec pandas analysis scripts read spreadsheets with it, and only Docling reaches it statically
worker.spec modules.documents.tasks, modules.artifacts.tasks, modules.agent.data_analysis.task Huey resolves a task by its name
worker.spec copy_metadata("lameenc"), and bundling/licenses/lameenc/ as lameenc-licenses/ a podcast is encoded as MP3 by LAME, statically linked into lameenc's extension, which the analyser finds by its import (studio) and which ships as its own file a user can replace. The wheel's dist-info holds lameenc's LGPL-3.0 text and ships only when named. The folder adds the GPL-3.0 that LGPL-3 §4(b) also asks for, LAME 3.100's COPYING and copyright notice from its release tarball, and where lameenc v1.8.4's and LAME 3.100's sources are

Office support adds nothing to either binary. LibreOffice is downloaded only when the user turns it on, or is the user's own, and runs as its own process from <data>/runtime/office/ (office pack): the API starts it for a Word or PowerPoint version's previews and a source's pages, the Studio worker for a workbook's recalculation and a PDF conversion. The worker writes recalculated values with lxml, which python-docx already brings; openpyxl reads LibreOffice's values in the worker only, so the API binary needs none.

Model packs

Four packs are staged into backend/models before packaging and ship as resources/models, so the first PDF parses, the first query embeds and the first podcast voices with no network, and silence never reaches a speech model. Every other audio model is downloaded like any other model (local-models/catalog.md).

Pack Staged by Holds
Embedding build:model, scripts/fetch_embedding_model.py bge-small-en-v1.5: the ONNX model and its tokenizer, from a pinned revision, sha256-checked
Voice build:voice, scripts/fetch_bundled_voice.py audio/: the manifest's first audio model in its default build, Kokoro 82M Q8_0 (190 MB), with its install record, fetched as a catalog install is: from its pinned commit, sha256-checked
Voice activity build:vad, scripts/fetch_silero_vad.py silero-vad/: Silero VAD's 16 kHz ONNX build, silero_vad_16k_op15.onnx (1,289,603 B, MIT), and its licence, from v6.2.3's commit of snakers4/silero-vad, sha256-checked; the API and the worker read it in place (admission)
Parser build:parser, scripts/fetch_docling_models.py Docling's layout, table and RapidOCR weights, pruned of the variants ingest never loads

The release workflow runs the four scripts directly. Without the parser pack, Docling downloads its weights on first use and RapidOCR writes into site-packages, which is read-only inside a frozen bundle, so the first PDF would fail rather than merely be slow. worker/ingestion/parsing.py points HF_HOME at the models directory before Docling loads and, once the parser pack is complete, sets HF_HUB_OFFLINE=1.

Native runtimes

  • scripts/fetch-llamacpp.mjs stages a pinned llama.cpp build, checked against its pinned SHA-256, into electron/llamacpp/, keeping llama-server and the libraries it links. Its rules, the pin, the Vulkan-only GPU backend and the pruning, are in local-models/runtime.md. It also stages each licence file the archive carries under its own name, and llama.cpp's own LICENSE on every platform. The Windows archives carry only LLVM OpenMP's (LICENSE-LLVM-OpenMP, for their libomp.dll), so that LICENSE comes from the copy in scripts/llamacpp/, taken at the pinned build. An archive that carries its own must match that copy, or the fetch stops.

  • scripts/sdcpp/stage.mjs, which build:sdcpp runs, stages stable-diffusion.cpp's sd-server into electron/sdcpp/ with the libraries it loads from beside itself and its licences, leaving sd-cli behind. It runs sd-server --help from the staged folder before it swaps the folder into place.

    • Windows downloads upstream's Vulkan archive, checked against its pinned SHA-256, and copies the MSVC and OpenMP runtimes beside it, which the archive needs and does not carry.
    • Linux and macOS compile the pinned commit, because upstream's Linux archives need glibc 2.38 and its macOS archive needs macOS 26.0. The recipe, scripts/sdcpp/recipe.mjs, builds Vulkan on Linux, with one ggml library per micro-architecture, and Metal on macOS, for 13.3.
    • Compiling needs CMake, a C++ compiler and the Vulkan SDK (glslc and headers) on Linux, or Xcode's command line tools on macOS; Windows needs Visual Studio 2022 or newer with the C++ tools, for the runtime it ships. Without them the script stages an empty folder and says why, and the app runs without local images. Release CI and pnpm dist pass --strict, which fails instead, naming everything missing and the command that installs it with the machine's package manager.

  • scripts/audiocpp/stage.mjs, which build:audiocpp runs, stages audio.cpp's audiocpp_server into electron/audiocpp/ with the model specs of the five curated families, FAMILIES in scripts/audiocpp/pins.mjs: Kokoro, Supertonic and Kitten for voices, and Nemotron's nemotron_asr and Canary's canary_asr for speech recognition, which the same binary serves from a second process (ADR 0056). It adds eSpeak-ng 1.52.0 from the pinned espeakng-loader wheel, which Kokoro and Kitten phonemise through (Kokoro finds it through the server's environment, Kitten through server.json, local-models/catalog.md), and eSpeak-ng's GPL licence text, which the wheel does not carry. Before it swaps the folder into place, it runs --list-devices from it, then starts the server once per curated family on an empty model package (scripts/audiocpp/built-families.mjs): a family the build has fails inside its own loader, and one it lacks is refused by name first, so a family missing from the build stops the stage. It prints the staged size and how long staging took. A server already staged is kept only when the same check finds every curated family in it (scripts/audiocpp/staged-server.mjs); one staged before a family joined the pins is compiled and swapped again, since it would answer that family's model with "unsupported model family hint", and so is one that does not start at all. Without --strict the stale server stays, with a warning that names what it lacks, when no toolchain is installed, when the app holds electron/audiocpp/ (checked before compiling), or when staging again fails, so pnpm dev goes on with every family it has.

    • macOS downloads upstream's archive, checked against its pinned SHA-256.
    • Windows and Linux compile the pinned commit, because upstream's Linux archives need glibc 2.38 and its Windows archive compiles AVX-512 into the executable. The recipe, scripts/audiocpp/recipe.mjs, builds only the CPU backend, since the server runs with --backend cpu, with one ggml library per micro-architecture and only the curated model families.
    • Compiling needs CMake and GCC 13 or newer on Linux, or Visual Studio 2022 or newer with the C++ tools on Windows. On Windows the build names Visual Studio's generator (msvc-runtime.mjs), since a CMake from MinGW or MSYS2 defaults to Ninja, which refuses -A x64 and finds no compiler outside a Visual Studio prompt. Without them the script stages an empty folder and says why, and the app runs without local audio. Release CI and pnpm dist pass --strict, which fails instead, naming everything missing and the command that installs it with the machine's package manager.

  • scripts/opencode/stage.mjs, which build:opencode runs, stages opencode 1.18.34 and ripgrep 15.1.0 into electron/opencode/, each archive checked against its SHA-256 in scripts/opencode/pins.mjs. On x64 it takes opencode's -baseline build, which opencode compiles without AVX2. The release archives carry only the executable, so opencode's licence comes from its tag, also pinned; ripgrep's three licence files ship in opencode/ripgrep/. Before it swaps the folder into place, scripts/opencode/check-stage.mjs checks the folder holds both executables and the licences and runs opencode --version and rg --version from it, and stage.mjs restages when the staged opencode reports another version. The Linux executable needs glibc 2.17 and no libstdc++. scripts/opencode/enabled.mjs turns staging on for every build; SURFSENSE_LOCAL_OPENCODE_ENABLED=0 turns it off for one, and off it removes an earlier stage. An empty value keeps the default, and any value but 1 or 0 stops the build, so a true or a no never ships an installer without the agent. The Windows stage is 184,869,203 B unpacked and 44,286,272 B as an xz -6 tar (measured, 7 Oct 2026); the release archives are 46 to 62 MB.

pnpm dist in electron/ runs every staging step before electron-builder, the native runtimes first and the third-party notices last, so a missing toolchain stops it before the frontend and the Python binaries are built. It stages sd.cpp and audio.cpp with --strict, so an installer never ships without them; predev does not, so dev runs without a toolchain. On Windows, the rename that swaps each stage into place waits up to a minute while another process still has a file in it open, such as a Windows service reading an executable the script just ran.

frontend/ and electron/ each pin their pnpm in packageManager (pnpm@11.27.1), which pnpm switches to locally and the desktop workflows read through package_json_file. The repository root's packageManager is the web app's, pnpm@10.26.0, and does not apply here.

Third-party notices

build:notices in electron/ writes the licence notices the installer ships, after every staging step because it reads what was staged. Four generators each write a fragment into electron/licenses/fragments/, and scripts/notices/merge.mjs merges them into electron/licenses/THIRD_PARTY_NOTICES.json, one entry per component with its name, version, tree, licence id and full text, and a plain .txt of the same, ordered by tree and then by name in code-unit order, so every machine writes the same file. Only those two files ship, as resources/licenses/; the folder is generated and ignored by git.

Generator Tree Reads
electron/scripts/notices/npm.mjs frontend, electron pnpm licenses list --json --prod in each pnpm tree, and every LICENSE, LICENCE, COPYING and UNLICENSE file at the top of each package. A NOTICE file ships after that text but is attribution, not licence terms, so a package carrying only a NOTICE has no text for the gate. A package that ships no text but whose terms a maintainer established takes the text recorded for its exact version, with its source, in electron/scripts/notices/reviewed-texts.json
backend/scripts/write_python_notices.py python the names uv export --frozen --no-dev gives for this platform, since CI's uv sync also installs the dev tools, then importlib.metadata for each: its License-Expression, License or classifiers, and the licence files it installs, the dist-info's first and then those in the package's own folders, such as a vendored library's. A NOTICE file, or a third-party notices file such as onnxruntime's ThirdPartyNotices.txt, ships after that text, as on the npm side, but never counts as it. It adds PyInstaller, whose bootloader runs inside both frozen binaries, and CPython, whose interpreter and standard library they carry, with the LICENSE.txt from the standard library or, failing that, the installation root, where uv's Windows Python keeps it
electron/scripts/notices/native.mjs native the licence files the stage scripts already leave in llamacpp/, sdcpp/, audiocpp/ and opencode/, and Electron's own; versions come from the stage scripts' pins. On Windows it also reads llamacpp/LICENSE-LLVM-OpenMP, for the libomp.dll only that platform's archive carries; a component listed for other platforms is skipped. Every file a staged component lists must hold text, or the generator fails naming it, since ripgrep's COPYING names its licences without their terms. A folder absent or empty, as in a dev checkout or a build staged with SURFSENSE_LOCAL_OPENCODE_ENABLED=0, is reported and left out
backend/scripts/write_model_notices.py model the pinned specs of the embedding, voice, parser and Silero VAD packs: repository, revision and licence id. Silero VAD's text is the LICENSE build:vad stages beside its weights; the others ship no licence file, so each says only the id is known and no text ships

The merge fails the build when a fragment is missing, or when an npm, Python or native entry has no licence text and electron/scripts/notices/allowed-without-text.json has no entry for it at that exact version with a reason, so a version bump is reviewed again. A reviewed entry adds its reason to the notice in place of the text.

Building audio.cpp

build-audiocpp.yml compiles the Windows and Linux builds with stage.mjs --strict, checks them, and hands each staged folder on as an artifact. release-local.yml calls it, and its packaging jobs unpack the artifact before stage.mjs, which then finds the server already staged. A pull request that changes scripts/audiocpp/, scripts/not-staged/, the audio.cpp adapters or engine, the transcription slice, the manifest or either server's test runs it too. It caches the staged folder by the scripts' contents, so a release that does not change them reuses the last checked build, and voices and transcribes with it again. The recogniser's 930 MB file is cached by its pinned sha256, while the voices are downloaded every run.

Why it compiles rather than downloads, in plain words: upstream's ready-made programs for Linux and Windows do not run on many of the computers the app supports, for a different reason on each.

  • Linux: the age of the system. Upstream's program needs a recent Linux, Ubuntu 24.04 or later (glibc 2.38). The app's build also runs on older ones, such as Ubuntu 22.04 and RHEL 9.
  • Windows: the processor, not the Windows version. Upstream's program uses AVX-512 instructions that many common processors lack, so it crashes on them, recent ones included: Intel's 12th to 14th generation, and AMD's before Zen 4. The app's build picks the right code for the processor it runs on.

macOS needs neither, so the app ships upstream's program there. The app's builds are upstream's own code at a pinned commit; only the build settings differ.

It is a stopgap. Once upstream publishes archives that meet both floors, the app downloads those as it does llama.cpp's, and the compile path and the workflow go.

  • Linux builds on ubuntu-22.04, the release's own runner, with GCC 13 from the toolchain PPA. The PPA replaces the runner's libstdc++ with a newer one, so the build runs in its own job, and the frozen Python binaries never bundle it.
  • libstdc++ is linked statically into every file, ggml's CPU modules included, since those take CMake's module flags. libgcc stays dynamic, because GCC 12 and 13's static unwinder on 22.04 calls _dl_find_object, from glibc 2.35.
  • OpenMP is off, so no libgomp ships.
  • Linux gates on no symbol newer than GLIBC_2.34 or GCC_7.0.0, which RHEL 9's glibc and libgcc provide, and on no file that needs libstdc++, libgomp or Vulkan. The runner's own libstdc++ is newer than a user's, so only that gate, not the voicing test, catches a dynamic one.
  • Windows copies the MSVC runtime beside the executable, so a clean Windows needs no redistributable.
  • Staging checks that the build has every curated family, and a change to FAMILIES changes the cache key, so adding a family compiles again and runs both floors' gates on the new build.
  • A third job stages upstream's macOS archive on macos-15 the same way, so a family added to the pin that the archive lacks fails on the pull request; the release's macOS job stages it again.
  • With the four families, the Windows stage is 32.6 MB, and cloning and compiling it took 4.8 minutes on a 16-thread Ryzen 7 5800X; nemotron_asr added 0.24 MB to the executable (measured, 14 Oct 2026).
  • Both run test_audiocpp_voicing.py on the staged folder, cached or not, with each curated model's pinned file downloaded from the URL its manifest entry names (Packaging tests).
  • An artifact drops symlinks and file modes, so the staged folder travels as a tar.

Building sd.cpp

build-sdcpp.yml compiles the Linux and macOS builds with stage.mjs --strict, checks each against the app's floors, runs test_sdcpp_generation.py on the staged folder, cached or not, with Stable Diffusion 1.5's pinned file downloaded from the URL its manifest entry names (Packaging tests), and hands each staged folder on as an artifact, as build-audiocpp.yml does. A pull request that changes scripts/sdcpp/, scripts/not-staged/, the image engine, the local manifest or that test runs it too, so a compile that a pin bump or a new runner image breaks, or a server that starts and cannot diffuse, fails in review rather than in a release. It caches the staged folder by the scripts' contents. Windows is a pinned download, so the release job stages it itself, as it does llama.cpp.

Why it compiles on Linux and macOS, in plain words: upstream's ready-made programs start only on the newest systems there, Ubuntu 24.04 (glibc 2.38) and macOS 26.0, while the app runs on Ubuntu 22.04, RHEL 9 and macOS 13.3, the floor its llama.cpp and audio.cpp builds already set. Upstream's Windows program runs on any x64 processor, since its AVX-512 code sits only in the per-processor ggml libraries ggml picks at start, so Windows takes it as built.

  • Linux builds on ubuntu-22.04 with LunarG's Vulkan SDK, as llama.cpp's own 22.04 release does, because 22.04 packages no glslc to compile ggml's shaders.
  • libstdc++ is linked statically into every file, ggml's modules included, and OpenMP is off, so neither libstdc++ nor libgomp ships. libgcc stays dynamic.
  • Linux gates on no symbol newer than GLIBC_2.34 or GCC_7.0.0, on no file that needs libstdc++ or libgomp, and on only the Vulkan backend needing the system's Vulkan loader, which comes with the GPU driver.
  • macOS gates on every file targeting 13.3 or older and linking only the system's libraries and its own.
  • It is a stopgap, like audio.cpp's: once upstream publishes a Linux archive built on 22.04 and a macOS one for 13.3, the app downloads those and the compile goes.

Release builds

release-local.yml builds on a v* tag, or on workflow_dispatch with a version and publish: never for a dry run:

Runner Artifacts
macos-15 SurfSense-arm64.dmg, SurfSense-arm64.zip
windows-latest SurfSense-Setup.exe, NSIS, x64
ubuntu-22.04 SurfSense.AppImage, SurfSense.deb, x64

The Linux runner is pinned because ubuntu-latest moves to 26.04 and would silently raise the AppImage's glibc floor; llama.cpp's Vulkan build needs 2.34. There is no Intel Mac build, because torch and onnxruntime no longer publish Intel macOS wheels.

Each runner, in order, checks the version (semver; on a tag push it must equal surfsense_local/VERSION; whenever it publishes it must carry no prerelease suffix), refuses to build with the test signing key (license), freezes the binaries, smokes the frozen worker's Docling vision imports (--check-vision-runtime) and the frozen API's /health, stages the model packs, builds the SPA and the Electron bundles, stages llama.cpp, opencode, audio.cpp and sd.cpp, generates the third-party notices, and runs electron-builder. On macOS and Windows, electron-builder's afterSign hook, scripts/opencode/after-sign.mjs, runs check-stage.mjs on the packaged opencode folder once the app is signed, and on macOS notarized, before any installer is made, so a tag's build never uploads one whose opencode signing broke; unsigned builds run it too. Linux signs nothing, so electron-builder skips the hook there with a warning. Every runner then runs check-stage.mjs again on the folder its build packaged: win-unpacked, linux-unpacked or mac-arm64/SurfSense.app. The three runners wait for the audiocpp and sdcpp jobs, and unpack audio.cpp's Windows and Linux builds and sd.cpp's Linux and macOS builds. On Linux it then starts the API from the packaged linux-unpacked resources and runs llama-server --list-devices, audiocpp_server --list-devices and sd-server --help from their packaged directories, because ggml finds its backends only next to the running executable, and checks that eSpeak-ng and its licence are packaged beside audiocpp_server, sd.cpp's Vulkan backend and licence beside sd-server, and the notices in resources/licenses without their fragments.

The macOS build is signed with the hardened runtime and notarized whenever the signing secrets are available, a dry run in this repository included. electron-builder signs every Mach-O file in the bundle, Contents/Resources included, with entitlementsInherit, except opencode: mac.signIgnore leaves it with its vendor's Developer ID signature, hardened runtime, secure timestamp and the five entitlements Bun asks for, among them disable-executable-page-protection, which the app's file lacks (measured on the pinned opencode-darwin-arm64.zip, 7 Oct 2026). The notary service takes it so: v2.2.0's tag build was notarized carrying it (9 Oct 2026). rg arrives only ad-hoc signed, so it is signed with the app. Azure Trusted Signing for the Windows build runs only on tag pushes, and signs every .exe under resources/ as electron-builder copies it, replacing any signature it carries. opencode.exe arrives Authenticode-signed by its vendor and rg.exe unsigned, and a tag's build runs both signed with SurfSense's certificate. A dry run has no Azure credentials and CSC_LINK, the macOS certificate, reaches only the macOS runner, so a dry run's Windows build is unsigned; before that it reached Windows too and electron-builder signed every .exe with it through signtool (the 2.1.0 dry run's log, 3 Oct 2026).

The Apple notarization credentials are checked by their own workflow, notarization-credentials.yml, with notarytool history: weekly on Mondays, on demand, and as a job the release build waits on, dry runs included; the audio and image sidecar builds run alongside it. A lapsed agreement or credential fails there in seconds, not at the end of a build, as the 2.1.0 dry run's did with Apple's 403 on 3 Oct 2026. Without the secrets it passes with a notice, except on a run that publishes, a tag's included, which must notarize.

The entitlements file also carries com.apple.security.device.audio-input, without which the hardened runtime keeps the microphone from the app and from its helpers, which inherit the file, and mac.extendInfo sets NSMicrophoneUsageDescription, the sentence macOS's microphone prompt shows: "SurfSense turns your speech into text on this computer." Electron's systemPreferences.askForMediaAccess, which the main process calls before the page opens the microphone, needs both: without the entitlement it can crash the app, and without the sentence the prompt shows Electron's own. scripts/mac-microphone.test.mjs checks that both stay configured. Which page may use the microphone once macOS allows it is decided at runtime (overview).

The installer's version is the one the workflow resolved, passed as -c.extraMetadata.version. The tree carries it too: surfsense_local/VERSION is the one number, and surfsense_local/scripts/bump-version.sh writes it into the backend's pyproject.toml, both package.json files, the site's lib/app-release.ts and the backend's app/license/release.py, then refreshes uv.lock.

The NSIS size limit

makensis is 32-bit and cannot map a payload over 2 GB, so the Windows installer has a hard ceiling. The Windows build hit it before 2.0.0 through the bundled Ollama's CUDA runners, which were pruned; 2.2.0's SurfSense-Setup.exe, the first to carry opencode and ripgrep, is 1,209,002,840 bytes on the release, about 1.2 GB. No CUDA payload ships.

Packaging tests

All nine carry the packaging marker, which pyproject.toml excludes by default because three of them freeze a binary, three run audio.cpp's server on real models and one runs sd.cpp's on a 3 GB one:

Test Proves
test_frozen_boot.py a minimal frozen entry, sys.frozen true, migrates a real database from the bundled revisions, loads vec0 and round-trips a vector through chunk_vectors
test_real_binaries.py the real API binary passes its retrieval import check and answers /health; the real worker passes its vision import check and both queue consumers stay up; both binaries ship the remote manifest, the worker ships the local one, and the frozen API serves a curated row
test_frozen_document_scripts.py a frozen worker, started by the runner in script mode as a packaged app starts it, places a matplotlib chart in a Word file with no display, saves charts as SVG and PDF and a PdfPages PDF, and writes a deck with a native chart and workbooks with xlsxwriter and openpyxl; an analysis script reads a workbook with pandas, prints a sum and keeps a table and a chart; the decode child, worker --decode-audio, reads an MP3 to 16 kHz mono with miniaudio, whose cffi backend is loaded by name; it freezes the worker from worker.spec itself, or uses the one SURFSENSE_TEST_FROZEN_WORKER names
test_spec_data_files.py every literal datas path in the specs exists, so a renamed file cannot ship missing, and both api.spec and worker.spec bundle the local model manifest
test_license_key.py the compiled license keys exclude the fixture key
test_audiocpp_voicing.py the staged audio.cpp server, started with the sidecar's flags from the server.json the app writes, voices two turns of each curated model through the app's adapter, at the sample rate its entry names, then holds no model loaded; it runs only when SURFSENSE_TEST_AUDIO_MODELS names a folder of the pinned files, each checked against its sha256, and then fails without a staged server; build-audiocpp.yml runs it on both builds
test_audiocpp_transcription_config.py the staged server, started as the transcription sidecar from the transcription.json the app writes, loads the curated recogniser and takes its request options
test_audiocpp_transcription.py Kokoro, on the voices' server, says one sentence each in English, Spanish, French, Hindi, Brazilian Portuguese and Chinese, and the transcription server writes each down through the app's client, tags and cleanup included, within 20% word error (character error for Chinese) and in the language voiced, which a Latin-script sentence Nemotron left untagged may leave unnamed; it records the server's peak memory, gives the model back by id, and, with Silero staged, checks the detector hears every sentence; it runs only when SURFSENSE_TEST_AUDIO_MODELS holds Kokoro's and the recogniser's pinned files
test_recording_transcription.py a recording of three sentences Kokoro voiced in English, Spanish and Portuguese goes through ingest end to end: the decode child, the real Silero, the app's API on a real port with the curated recogniser chosen, and the staged transcription server; the transcript has its header and two windows, each at its time, with words from every sentence, and segments.json points at both paragraphs; it runs only when SURFSENSE_TEST_AUDIO_MODELS holds the recogniser's pinned file
test_sdcpp_generation.py the staged sd.cpp server, started with the sidecar's flags on Stable Diffusion 1.5 as the engine reads an installed build, makes one image of a red apple through the app's image client, and the reply is checked for what such a picture has, not for the apple itself: it decodes, is the 512 pixels square the entry asks for, has more than a thousand colours, is red on average, and its middle is redder than its border, which neither a flat frame nor noise is; it runs at 4 steps and on seed 42, sd-server's default written down, the two flags the app does not pass (and --backend cpu on the hosted macOS runner, whose virtual graphics card Metal cannot use, so Metal itself is not exercised in CI), and only when SURFSENSE_TEST_IMAGE_MODELS names a folder holding the pinned file, checked against its sha256, and then fails without a staged server

Seven run in CI: test_license_key.py inside the release workflow, test_frozen_document_scripts.py there too, on each OS against the worker the job just froze, the four audio.cpp tests inside build-audiocpp.yml, and test_sdcpp_generation.py inside build-sdcpp.yml. Windows' sd-server is a pinned download the release job stages, so that gate does not cover it.

Known gaps

  • No issue on audio.cpp asks for archives that meet the app's floors yet, so build-audiocpp.yml has no end date.
  • On macOS nemotron_asr and canary_asr are checked only for a loader: the macos-archive job stages upstream's archive and transcribes nothing, so a macOS server that cannot transcribe would still ship.
  • No packaging test checks that the frozen API carries the agent's SKILL.md; api.spec collects it, and it was checked by hand.
  • The packaged llama-server, audiocpp_server and sd-server are started only on the Linux runner (#1986).
  • Only a tag's build exercises Windows signing: a dry run's Windows build is unsigned, so a lapsed Azure client secret shows on the tag first.
  • No notarized macOS build has shown the microphone prompt yet: the entitlement and the usage sentence are checked only as configuration, and the Privacy › Microphone address openSettings() opens is unchecked on macOS 13 to 15.
  • The sidecars are signed with entitlementsInherit, so they carry the audio-input entitlement too: once macOS lets SurfSense use the microphone, a sidecar may open it without the page, the Studio worker running a model's analysis script among them, as any process may on Windows and Linux. electron-builder gives every file but the app's own executable that one list, so narrowing it needs a signing hook of the app's own.
  • What matplotlib adds to the installer has not been measured against its size gate. A scratch freeze of the worker put it at about 15 MB unpacked under _internal/matplotlib, plus about 0.6 MB for contourpy and kiwisolver.