Skip to content

About

RSS to Mastodon Bridge

Topics

Resources

Stars

13 stars

Watchers

1 watching

Forks

Latest commit

 

History

716 Commits

Folders and files

Repository files navigation

robot.villas

An RSS-to-fediverse bridge. Each feed in feeds.yml becomes a bot account (for example @hackernews@robot.villas) that anyone on Mastodon or another ActivityPub server can follow. Bots post unlisted, so followers see every post and public timelines and search don't.

Configuration

feeds.yml defines the bots. Editors can validate it with feeds.schema.json.

# yaml-language-server: $schema=./feeds.schema.json
bots:
  hackernews:                 # username: lowercase letters, digits, underscores
    feed_url: https://news.ycombinator.com/rss
    display_name: Hacker News # max 100 chars
    summary: Top stories from Hacker News # max 500 chars
    profile_photo: https://news.ycombinator.com/y18.svg # optional
    homepage_url: https://news.ycombinator.com          # optional
    default_hashtags: [Tech, News]                      # optional, up to 3
    webmentions: false                                  # optional; stops webmentions to the site
follows: ["@alice@example.social"]    # every bot follows these
relays: [https://relay.example/actor] # instance relays to subscribe to
relay_subscription_bot: hackernews    # bot that subscribes to relays (default: first bot)
blocked_instances: [spam.example]     # blocked both ways, subdomains included
meta: true                            # adds @meta, which announces new bots

URLs must be HTTP(S). Feeds are polled every 15 minutes with conditional requests, and a 429 pauses a feed until its Retry-After (1 hour if missing, 24 hours at most).

Choosing feeds

Add news sites and blogs, feeds a person would read. Leave out machine-generated streams (package releases, price drops, image boards, status pages, wiki recent changes, listings) and forum or Q&A feeds that post every reply. Every new post becomes one delivery per follower inbox and per relay, and all bots share one delivery queue, so a feed posting thousands of items a day delays every other bot. A bot's first fetch and entries dated more than 7 days back are stored without delivery.

Environment

Variable Purpose Default
DATABASE_URL PostgreSQL connection string required
DOMAIN Public domain for ActivityPub IDs and handles required
WEB_BOT_AUTH_PRIVATE_KEY PEM Ed25519 private key for the signed key directory and HTTPS remote fetches disabled
PORT HTTP port 3000 (8080 in Docker)
BLOCKED_INSTANCES Comma-separated hosts added to blocked_instances
POLL_CONCURRENCY Feeds fetched at once 10
DISABLE_BACKGROUND true serves HTTP only (no queue, jobs, or poller)
SOURCE_COMMIT or GIT_SHA Build ID; startup jobs run once per build and config GIT_SHA is set in published images
GEMINI_API_KEY or GEMINI_PROJECT Enables AI hashtag suggestions (API key or Vertex AI)
GEMINI_LOCATION Vertex AI region us-central1
GEMINI_MODEL Gemini model gemini-2.5-flash-lite

Development

Requires Node 26, pnpm, and PostgreSQL. Migrations run on startup.

pnpm install
DATABASE_URL=postgres://localhost/robot_villas DOMAIN=localhost pnpm dev
Command Does
pnpm test Vitest; database tests run only when DATABASE_URL is set
pnpm lint:check What CI runs: ESLint (including @fedify/lint), YAML format, typecheck
pnpm lint Same, but fixes what it can
pnpm validate-feeds Checks feeds.yml; add --skip-network to skip avatar probes
pnpm db:generate Creates a migration in drizzle/ after editing src/lib/schema.ts

Endpoints

  • /@name: bot profile with its posts; /@name/followers and /@name/following
  • /users/name/posts/id: a post and the people who liked or boosted it; also the webmention source sent to the article's site (see docs/operations.md)
  • /users/name/posts/id/reactions/page_id: one like or boost, the webmention source for that reaction
  • /posts, /tags, /tags/name: all posts, all hashtags, posts with a hashtag
  • /search: find bots by handle, name, or description
  • /about: adding and removing feeds, opting out of webmentions, and how the fetcher behaves
  • /stats: bot, post, and follower counts
  • /status: feed health, publishing backlog, relays, follows, and inbox results
  • /mcp: read-only, authless MCP server (Streamable HTTP) with search_feeds, search_posts, and get_feed_posts tools
  • /healthcheck (liveness) and /readyz (database reachable)
  • ActivityPub, WebFinger, and NodeInfo are served by Fedify through src/proxy.ts

Layout

src/app/         Next.js pages and routes
src/proxy.ts     Sends federation requests to Fedify
src/instrumentation.ts  Startup: migrations, queue workers, jobs, poller
src/lib/
  config.ts      Parses and validates `feeds.yml`
  federation.ts  Actors, objects, collections, inbox listeners
  inbox-state.ts Follow responses and reactions (authorized, idempotent)
  poller.ts      Feed polling with per-feed leases
  rss.ts         Feed fetching and parsing
  publisher.ts   Stores new entries with their delivery records
  publications.ts Hands delivery records to Fedify's queue, with backoff
  inbox-filter.ts Drops relay Announces of other servers' posts
  maintenance.ts Startup jobs as Fedify background tasks
  remote-fetch.ts Public-address-only, size-limited fetching
  robots.ts      robots.txt checks before feed fetches
  webmentions.ts Webmention discovery and sending for posts and reactions
  hashtags.ts    Hashtags from feed categories, defaults, and Gemini
  mcp.ts         MCP server tools
  schema.ts, db.ts  Drizzle schema and queries
drizzle/         SQL migrations
docs/operations.md  How delivery, retries, and limits behave

Deployment

main publishes ghcr.io/icco/robot.villas:main after CI passes and the built image passes a smoke test. feeds.yml is baked into the image; mount a file over /app/feeds.yml to override it.

Cloudflare Botbase

Generate a persistent signing key outside the repository and store it in your deployment's secret manager:

umask 077
openssl genpkey -algorithm ed25519 -out private-key.pem

Set WEB_BOT_AUTH_PRIVATE_KEY to the PEM file's contents, including actual newlines, and DOMAIN=robot.villas. Never commit the private key or bake it into the image. All replicas must use the same key. Without a key, remote fetches remain unsigned and the key directory returns 503; invalid configured keys fail instead of silently disabling signing.

The app serves a signed public JWKS at https://robot.villas/.well-known/http-message-signatures-directory. Responses bind the request authority, content type, and body digest and disable caching to avoid stale signatures. HTTPS remote fetches sign @authority and signature-agent with a fresh nonce and a 60-second lifetime on every redirect hop. HTTP requests remain unsigned. ActivityPub signing is unchanged.

Follow Cloudflare's Web Bot Auth registration instructions: select Request Signature in the Bot Submission Form and use the directory URL above as Validation Instructions. The feed User-Agent is robot.villas RSS poller/1.0 (+https://robot.villas/about). Cloudflare approval is still required before requests become verified.

After deployment, inspect the directory with curl -i https://robot.villas/.well-known/http-message-signatures-directory and validate it with Cloudflare's http-signature-directory tool. Cloudflare's request test endpoint returns 200 for a verified known key, 401 for an unknown key or failed verification, and 400 for malformed requests. Replacing the key also changes its thumbprint; coordinate rotation with Cloudflare before removing the registered key.

License

MIT

About

RSS to Mastodon Bridge

Topics

Resources

Stars

13 stars

Watchers

1 watching

Forks

Sponsor this project

Packages

Used by

Contributors

Languages