Skip to content

fix: use DB_SCHEMA for Oban tables - #3826

Open
ElemTran wants to merge 2 commits into
Logflare:mainfrom
ElemTran:fix/oban-db-schema
Open

fix: use DB_SCHEMA for Oban tables#3826
ElemTran wants to merge 2 commits into
Logflare:mainfrom
ElemTran:fix/oban-db-schema

Conversation

@ElemTran

Copy link
Copy Markdown

Summary

  • configure Oban to use DB_SCHEMA, falling back to public
  • create Oban structures in the configured schema for both fresh and existing installations
  • preserve existing jobs and Oban objects in public during upgrades
  • document the upgrade behavior and cover both migration paths with PostgreSQL integration tests

Fixes #3251

Testing

  • mix test test/logflare/oban_schema_migration_test.exs --trace (2 tests, 0 failures)
  • verified the migration tests fail when the custom-schema prefix is removed
  • mix format --check-formatted
  • mix credo
  • git diff --check
  • verified default, fresh custom-schema, and existing-installation upgrade scenarios against PostgreSQL 15 and PostgreSQL 18.4

@@ -0,0 +1,114 @@
defmodule Logflare.ObanSchemaMigrationTest do

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test repo would already have migrations run against it, so this test module is redundant.

can instead do the following:

  • change application config for db schema in test setup
  • provision the schema and run the migration
  • assert that the oban tables are created in the randomized schema.
  • assert no public tables created.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CI setup runs mix ecto.setup before the test suite, so public.oban_jobs already exists when this test starts. For the "assert no public tables created" step, would you prefer that we snapshot and assert the existing public Oban tables/jobs remain unchanged, or temporarily remove and restore the public Oban structures around this test?

def up, do: Oban.Migration.up(version: 12, prefix: oban_prefix())
def down, do: :ok

defp oban_prefix, do: System.get_env("DB_SCHEMA", "public")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should not use env var directly, should reference the db schem via application config with get_env

def up, do: Oban.Migration.up(version: 12, prefix: oban_prefix())
def down, do: Oban.Migration.down(version: 1, prefix: oban_prefix())

defp oban_prefix, do: System.get_env("DB_SCHEMA", "public")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shoudl not use env var directly

Comment thread config/runtime.exs
enable_alerting? = Env.get_boolean("LOGFLARE_ALERTS_ENABLED", true)

config :logflare, Oban,
prefix: System.get_env("DB_SCHEMA", "public"),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can bind as module variable and reference for both db config and oban config.

| `DB_PASSWORD` | String, defaults to `nil` | Password for Logflare's internal PostgreSQL database connection. |
| `DB_POOL_SIZE` | Integer, defaults to `10` | Overrides the Ecto connection pool size for Logflare's internal PostgreSQL database connection. |
| `DB_SCHEMA` | String, defaults to `nil` | Allows configuration of the database schema to scope Logflare operations. |
| `DB_SCHEMA` | String, defaults to `nil` | Allows configuration of the database schema to scope Logflare operations, including Oban jobs. Defaults to the `public` schema when unset. When upgrading an existing installation, Oban structures are created in the configured schema, but existing jobs and Oban objects in `public` are not copied or removed automatically. Stop all Logflare nodes before manually migrating jobs that must be preserved. |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can move the upgrade note to a an Upgrading section with a version number + date.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Which release version should the new Upgrading section use? I can date the note 2026-08-14, but I don't want to guess the target release version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make Oban table creation configurable to use custom schema (avoid public schema pollution)

2 participants