Watertown is a file machine.
Watertown is a timeseries database.
Watertown is a site generator.
Watertown is a telemetry system.

Watertown is local-first.
Watertown is transactional.
Watertown is replicated.
Watertown is small.

What is Watertown?

Watertown is a file system tool to help organize process files and file-based processes. Watertown lets you place your CSV, Parquet, and JSON files into application-specific paths, then pattern-match and query them using pre-built file factories or ad-hoc SQL statements.

Watertown is built in Rust, using Apache DataFusion embedded query engine and DeltaLake transaction system. Watertown reads and writes local storage, cloud storage, git repositories, REST APIs, and more.

Watertown's abstract file system has first-class support for tabular, time-series, and multi-version file data. File and directory factory instances can be registered to create dynamic, derivative file content. Watertown includes built-in factories for combining, joining, and reducing timeseries.

Watertown is useful for data collection, analysis, and monitoring in small industrial settings.

Who is building Watertown?

Watertown is built by the Caspar Water System for small water systems everywhere.

The author is Joshua MacDonald, an open-source lead and software engineer at Microsoft, working in telemetry systems. Joshua is a member of the OpenTelemetry technical committee and co-founder of the OpenTelemetry-Arrow project, which is bringing a high-performance telemetry pipeline in Rust to OpenTelemetry.

Who is using Watertown?

Caspar Water uses Watertown for telemetry, monitoring, its public portal, and more.

The Noyo Center for Marine Sciences in Fort Bragg, CA was Watertown's first user, where Watertown gathers water quality data for its public portal.

CLI Reference

See docs/cli-reference.md for the complete command reference. Common commands:

pond init                           # Create a new pond
pond list '/**'                     # List all entries
pond cat /path/to/file              # Read a file
pond cat --sql "SELECT * FROM source WHERE ..." /path  # Query a table
pond copy host:///local/file /pond/path                # Import a file
pond copy host+series:///data.parquet /pond/series      # Import time-series
pond mkdir /dir                     # Create a directory
pond mknod <factory> /path --config-path config.yaml   # Install a factory
pond run /path/to/factory <command>                     # Execute a factory
pond log                            # Transaction history

Host mode (no pond required):

pond cat host+csv:///tmp/data.csv --format=table       # Query a local CSV
pond run host+remote:///config.yaml list-ponds          # Browse S3 backups
pond run host+sitegen:///site.yaml build ./dist         # Generate a site

Integration Tests

Tests live in testsuite/tests/ as numbered shell scripts. Each test runs in a fresh Docker container with the pond binary:

make test-image                     # Build the test Docker image
make integration                    # Run all tests (skips browser tests)
make integration-all                # Run all tests including browser

# Run a single test
cd testsuite && ./run-test.sh 201

# Run interactively (explore in container)
cd testsuite && ./run-test.sh --interactive

License

Apache-2.0 — see LICENSES/ for details.