Skip to content

Add Google Group Management app-group-lifecycle plugin#532

Open
somethingnew2-0 wants to merge 14 commits into
mainfrom
pcollins/pr-498-rebase-plugin-7c13de
Open

Add Google Group Management app-group-lifecycle plugin#532
somethingnew2-0 wants to merge 14 commits into
mainfrom
pcollins/pr-498-rebase-plugin-7c13de

Conversation

@somethingnew2-0

@somethingnew2-0 somethingnew2-0 commented Jul 15, 2026

Copy link
Copy Markdown
Collaborator

Takes over and rebases #498 (originally by @barborico) onto current main. See that PR for fuller description, validation, and screenshots.

What

Adds an opt-in example app-group-lifecycle plugin that mirrors each Access group to a Google group and links them via Okta group push (Okta then owns membership sync), plus the core-Access enablers it needs:

  • OktaService gains group push-mapping create/delete/list helpers.
  • Group-config plugin hooks become app-config aware (so a plugin's group schema/validation can reflect app-level settings, e.g. an email-prefix pattern), and gain an immutable field flag enforced host-side on update and rendered read-only in the UI.
  • New ModifyGroupPluginData operation owns the plugin_data merge and fires group_updated only when a plugin's configuration actually changes — never on status-only writes, which would otherwise re-trigger reconciliation in a loop. Group plugin config is validated up front on create/update (400 instead of a later reconcile failure).
  • The bulk-sync hook sync_all_group_membership is renamed to sync_all_groups (it reconciles all group state, not just membership), with a renamed access sync-app-groups CLI command.
  • The image ships the plugin source under /app/plugins without installing it, so a downstream image opts in by pip-installing it.

Why this is a fresh rebase, not a merge

#498 was branched before three things now on main: the async-SQLAlchemy migration (#480/#481), the uv/ty toolchain (#507), and — most relevant — the async plugin interface (#511). So this isn't a mechanical rebase: every backend surface #498 added had to be reworked for async. The plugin's lifecycle hooks are now async def and receive the request AsyncSession; Okta calls are awaited; the blocking Cloud Identity client runs off the event loop via asyncio.to_thread; and the operation/router/CLI paths were converted to awaited async SQLAlchemy. The 19 original commits are re-expressed as 6 logically-grouped, async-native commits.

The plugin's own behavior (idempotent reconcile, Access-as-source-of-truth with backfill from live Google/Okta state, immutable group email, per-group sync status) is unchanged from #498 — see that PR's description and screenshots for the functional walkthrough.

Rebase and async conversion by Claude; original plugin design and implementation by @barborico in #498.

🤖 Generated with Claude Code

somethingnew2-0 and others added 10 commits July 22, 2026 23:32
Add async create/delete/list helpers for Okta group push mappings on
OktaService -- the link primitive the app group lifecycle plugins need to bind
an Access-managed Okta group to an external group. delete PATCHes the mapping
INACTIVE before DELETE (Okta rejects deleting active mappings), treats a 404 as
success so the operation is idempotent, and defaults deleteTargetGroup to False
so unlinking never removes the external group. list follows Link-header
pagination so callers see every mapping.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…all_groups

Thread the owning app's plugin config into the group-config-properties and
group-config validation hooks so a plugin's group schema and validation can
reflect app-level settings (e.g. an email pattern). The hookspec args are
additive and pluggy passes only the args an impl declares, so existing
implementations keep working. Add an `immutable` group-config flag enforced
generically host-side on update -- a changed immutable field is rejected, while
an error on an *unchanged* one is suppressed (it's locked and can't be actioned
on this update). Add set_config_value and is_plugin_config_changed helpers.

Rename the bulk-sync hook sync_all_group_membership to sync_all_groups (it
reconciles all of an app's groups, not just memberships), keeping the old name
as a deprecated -- still async -- alias so plugins on either name keep running,
and add invoke_sync_all_groups to dispatch to both via run_hooks_to_completion.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… front

Add ModifyGroupPluginData, which owns the group plugin_data merge (preserving
other plugins' top-level entries and per-plugin status the patch omits) and
fires the async group_updated hook only when the configured plugin's
*configuration* actually changes -- never on status-only writes, which the
plugin makes itself and which would otherwise re-trigger reconciliation in a
loop. put_group now routes plugin_data through it.

Validate group plugin config against the plugin's schema on both create
(post_group) and update (put_group), returning 400 up front instead of failing
later during reconcile, and let group-config-props accept ?app_id= so the
returned schema reflects app-level constraints. Rename the periodic CLI command
to `access sync-app-groups`, keeping `sync-app-group-memberships` as a hidden
deprecated alias.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Surface each group-config field's validation.patterns as react-hook-form rules
so the UI rejects out-of-pattern input before submitting (the backend stays
authoritative). Render immutable fields read-only -- not disabled, so their
value is still submitted -- when editing an existing app or group, with a
helper-text note. Thread the owning app's id into the group-config-props lookup
so the schema reflects app-level constraints. Regenerate the API client for the
new app_id query param and the immutable config-property flag.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an opt-in example app-group-lifecycle plugin that creates, updates, and
deletes Google groups for Access groups and links them to the Access-managed
Okta group via Okta group push (which then owns membership sync). A single
idempotent reconcile drives create/adopt/enforce and a periodic sync_all_groups
sweep: Access is the source of truth, with graceful backfill from live
Google/Okta state, an app-level email-prefix pattern, an immutable group email,
and per-group sync status/errors.

Written against the async (Access 2.0) plugin interface: the lifecycle hooks are
async and receive the request's AsyncSession, Okta calls are awaited, and the
blocking Cloud Identity client runs off the event loop via asyncio.to_thread.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… plugin

Read the root log level from LOG_LEVEL (default INFO) so a deployed environment
can raise verbosity (e.g. to trace app-group reconciliation) without a code
change. Add a `make sync-app-groups` target that sources .env before running the
CLI, since the app-group lifecycle plugins need credentials (Google/Okta) the
other management targets don't load. Copy the example plugin source into the
image at /app/plugins without installing it, so a downstream image can opt in by
pip-installing it while the source stays single-homed here.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve four issues raised in review of the Google group manager PR:

- Partial group_config PATCHes no longer 400 on an omitted immutable
  field; a field counts as edited only when present in the patch, so an
  API update of just the mutable fields stops tripping the immutable one.

- The Google plugin can now clear a group's description. Backfill from
  Google only happens during adoption (no Access-side config yet); once
  Access owns the config its description is authoritative, so emptying it
  clears Google instead of being silently backfilled from the stale value.

- A single PUT that changes both name/description and plugin config now
  fires group_updated once instead of twice. ModifyGroupDetails and
  ModifyGroupPluginData gain a fire_lifecycle_hook opt-out (default True
  preserves standalone behavior) and put_group fires one consolidated
  hook, avoiding a double reconcile (Google + Okta round-trips).

- Drop the deprecated sync_all_group_membership hook alias entirely
  rather than guard against dispatching it twice. The #511 async
  migration already broke every plugin implementing it (sync -> async)
  and sync_all_groups has never shipped, so the "backward-compatible"
  alias protected nobody while adding a double-reconcile footgun. Also
  removes the now-thin invoke_sync_all_groups wrapper and the deprecated
  sync-app-group-memberships CLI alias.

Tests live alongside each fix.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
list/create/delete_group_push_mapping hand-rolled request_executor.execute(),
which assumed okta 2.x's 2-tuple (OktaAPIResponse, error) return. The installed
okta 3.4.4 (OpenAPI-generated) execute() returns (response, response_body, error)
of raw aiohttp objects, so listing raised "ValueError: too many values to unpack
(expected 2)" and the get_body()/pagination calls would have failed next.

- Route all three through the inherited GroupPushMappingApi methods on the client
  proxy so they ride the same (data, resp, error) contract as every other list
  method and reuse _paginate for Link-header pagination.
- create-or-link returns HTTP 200, but the SDK only types the spec'd 201, leaving
  resp.data empty; parse the mapping from resp.raw_data as a fallback.
- Deactivate-then-delete now goes through update/delete_group_push_mapping; 404
  idempotency moves to an except ApiException block since the typed client raises
  on HTTP errors rather than returning them in the tuple.
- Add a server-side sourceGroupId filter to list_group_push_mappings so callers
  after one source group's mapping don't page the whole app.
- Convert models to camelCase dicts via model_dump(by_alias=True); the model's
  own to_dict() drops the read-only id/sourceGroupId/targetGroupId fields.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rework the Google group manager so group creation goes through Okta group push
rather than a Google-first create: Okta creates its target group AND the
downstream Google group (named by the email prefix) and links them in one step,
avoiding the manual-fetch import lag. This removes _create_google_group and the
GOOGLE_WORKSPACE_CUSTOMER_ID requirement; the group's Cloud Identity id and
metadata are resolved on a later reconcile once Okta has pushed it.

- Route the adoption-time description backfill through the ModifyGroupDetails
  operation (ORM write + Okta sync + audit log) instead of mutating the group
  directly; suppress the group_updated hook to avoid re-entering this plugin.
- Surface a sync error when an existing push mapping targets a Google group
  other than the group's configured email (a conflict that won't self-heal),
  instead of silently adopting the wrong group.
- Read the googleGroupEmail custom attribute from the profile union's
  actual_instance.additional_properties (the 3.4.4 model shape), and use the
  new server-side sourceGroupId filter in discovery.
- Fix last_synced_at to a timezone-aware datetime.now(timezone.utc).
- Update README and tests to match.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rebase integration onto main's stricter ty: put_group passed
body.plugin_data (typed dict[str, Any] | None) to _validate_group_plugin_config,
whose parameter is dict[str, Any]. Coerce None to {} for the validation call,
matching the sibling create-group call site and the pre-existing behavior; the
None handling for the actual update below is unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Comment thread api/operations/modify_group_plugin_data.py Outdated
Comment thread api/routers/groups.py Outdated
Comment thread api/routers/groups.py
Comment thread api/services/okta_service.py Outdated
Comment thread api/cli.py
click.echo("Completed app group lifecycle plugin sync")


@cli.command("sync-app-groups")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heads up on backward compatibility: sync-app-group-memberships (this command) and the sync_all_group_membership hook both exist on main today, so this is a rename of already-shipped surface, not a net-new name. There is no compatibility alias, so:

  • any external scheduler invoking access sync-app-group-memberships (for example a periodic job defined outside this repo) breaks with "No such command" on deploy; and
  • any third-party lifecycle plugin still implementing sync_all_group_membership silently stops being invoked, since it no longer matches a hookspec.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Accepting this rename without a compatibility alias, intentionally. This ships as part of Access 2.0, which introduces a number of breaking changes (e.g. the async hook specs), so we're deliberately omitting the deprecated command/hook shims that would preserve backward compatibility. External schedulers and third-party plugins will need to move to the new sync-app-groups command and hook name as part of the 2.0 upgrade.

Comment thread examples/plugins/app_group_lifecycle_google/plugin.py Outdated
@barborico
barborico force-pushed the pcollins/pr-498-rebase-plugin-7c13de branch from 69f9ade to 11a5fd2 Compare July 23, 2026 00:23
barborico and others added 4 commits July 23, 2026 00:43
…elds

Add an optional `suffix` to app-group-lifecycle plugin config properties,
rendered as an inline end-adornment on the app/group edit form's text input.
This lets a plugin show static trailing context (e.g. an email domain) while
the stored value stays just what the user types.

- api/plugins + api/schemas: add `suffix` to AppGroupLifecyclePluginConfigProperty
  and its PluginConfigProp wire schema (the router's asdict passthrough needs no
  change).
- src/api: regenerated client exposes the new optional field.
- src/components: ConfigField renders `suffix` as a MUI InputAdornment on text
  fields, omitted when unset.
- google plugin: set suffix="@{domain}" on the email-prefix field so operators
  see the full address; the local part remains the stored value.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…alidation

- ModifyGroupPluginData (#1): compute config_changed against the fully-merged effective
  config, snapshotting the pre-merge config first (the partial-patch merge mutates it in
  place). A partial patch that only omits an unchanged config key (e.g. the immutable email,
  now permitted by the present-only immutable check) no longer misreads as a change and
  fires a redundant reconcile.

- put_group + ModifyGroupType (#2): fire exactly one app group lifecycle hook per PUT, after
  every field is applied. ModifyGroupType gains fire_created_hook so put_group can defer the
  group_created fire to the end (after plugin_data is applied) and choose the event by the
  transition: group_created on convert-to-app-group, group_updated on a same-type edit,
  group_deleted (unchanged, still in ModifyGroupType, which needs the pre-conversion row) on
  convert-away. Previously a convert-to-app-group + rename fired both group_created (before
  the config was applied) and group_updated -- a double reconcile that also hid the new config
  from group_created.

- put_group validation (#3): catch AppGroupLifecyclePluginFilteringError (a plain Exception,
  raised when a plugin doesn't answer a hook with exactly one response) and surface a clean
  500 rather than an unhandled stack trace.

Tests cover the three type transitions (single fire; group_created sees final config), the
partial-patch no-redundant-reconcile case, and the filtering-error 500.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Critical:
- delete_group_push_mapping: the 404-idempotency was dead code. This SDK returns HTTP errors
  in the result tuple (it does not raise ApiException), and delete returns a 2-tuple on error
  vs a 3-tuple on success -- so the old `_, _, error =` unpack raised ValueError and the
  `except ApiException` never fired. Read the trailing error element arity-safely and key
  idempotency on error.status == 404 at both the deactivate and delete steps. Rewrote the
  tests to simulate returned error tuples (they had mocked a raised ApiException, which the
  SDK never does) and added a deactivate-step-404 case.

Important:
- Validate group plugin_data on the convert-to-app_group PUT path (previously skipped because
  the group was still its old type), against the target app; old_plugin_data=None since the
  conversion attaches the plugin fresh.
- Add a parity test asserting the plugin dataclasses and their Pydantic wire models expose
  identical fields, so silent asdict()->model_validate() drift fails loudly.
- Test gaps: first-ever config set fires group_updated; second-pass reconcile skips re-create
  when the mapping is already recorded; _reconcile persists SYNC_ERROR and re-raises on an
  unexpected failure.

Suggestions:
- Freeze the plugin descriptor dataclasses; reject suffix on non-text config properties at
  construction; drop the defensive getattr(prop, "immutable").
- create_group_push_mapping: replace asserts on the external response with explicit errors.
- sync_all_groups: count per-group failures and log an aggregate error line.
- _discover_existing_push_mapping: log un-interpretable mappings at WARNING (duplicate risk).
- Comment fixes: stale get_body()/next-cursor references, the fire_lifecycle_hook wording,
  and the stale "GitHub team" example.

Deliberately not done: strictly typing the `validation` field (a legitimately open,
plugin-defined bag -- the audit-logger example uses `allowed_values`, not `patterns`), and
moving the plugin-data audit into ModifyGroupPluginData (single caller; would change audit
semantics on the convert+plugin_data path).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@barborico
barborico marked this pull request as ready for review July 23, 2026 06:43
Comment on lines +339 to +340
async def _delete_google_group(self, google_group_id: str) -> None:
await asyncio.to_thread(lambda: self._groups_api.delete(name=self._resource_name(google_group_id)).execute())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_delete_google_group is the only Google-side path that isn't tolerant of an already-absent group. Every read path (_lookup_google_group_id, _email_from_status, _owned_group_id) tolerates absence, but this bare delete().execute() does not.

In group_deleted the mapping unlink is wrapped in a best-effort try/except, but this call runs outside it. So if the Google group was already deleted out of band (or on a delete replay/retry, which the reconcile design intends to be safe), delete() raises, the group_deleted hook raises, and invoke_app_group_lifecycle_hook rolls back instead of committing. The "authoritative cleanup" the surrounding comment describes then fails on a group that is already gone.

Note it should NOT reuse _is_group_absent_error here: that treats 403 as absent, whose rationale ("a genuine permission problem surfaces on the subsequent create call") does not hold on the delete path. A 403 on delete most likely means the service account lacks delete permission on a group that still exists; swallowing it would log a phantom successful deletion and leak the group. Only a true 404 is an unambiguous "already gone":

Suggested change
async def _delete_google_group(self, google_group_id: str) -> None:
await asyncio.to_thread(lambda: self._groups_api.delete(name=self._resource_name(google_group_id)).execute())
async def _delete_google_group(self, google_group_id: str) -> None:
try:
await asyncio.to_thread(lambda: self._groups_api.delete(name=self._resource_name(google_group_id)).execute())
except HttpError as e:
# 404 -> already gone, so a delete replay is a no-op. Deliberately 404-only (not
# _is_group_absent_error): a 403 on delete more likely means we lack permission on a
# group that still exists, which must surface rather than masquerade as a deletion.
if getattr(getattr(e, "resp", None), "status", None) != 404:
raise

(One thing to confirm: that Cloud Identity's groups.delete returns 404, not a conflated 403, for an already-deleted group. If it returns 403, a replay would still error here, which is the safe direction.)

Comment thread Dockerfile
# Ship prod-ready plugin source into the image WITHOUT installing it here.
# Downstream images opt in by pip installing the plugins, which keeps the source
# single-homed in this repo while leaving activation to the consumer.
COPY ./examples/plugins/app_group_lifecycle_google ./plugins/app_group_lifecycle_google

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we rebase this off of #552 and add this plugin install as optional, but enable it via a build argument in the docker-image.yml Github Action?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't have any other frontend tests and nothing runs them in CI. Do we want to start including these or should they be removed?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding frontend test framework in #564.

)


class ModifyGroupPluginData:

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should ModifyGroupDetails call into this new operation instead of it running invoke_app_group_lifecycle_hook(AppGroupLifecycleHook.GROUP_UPDATED...) itself?

Comment thread api/plugins/app_group_lifecycle.py
set_config_value,
set_status_value,
)
from api.services import okta # used by the push-mapping/discovery helpers

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how much I like plugins being able to call Access API methods directly. I wonder if there's a way we can expose the Okta group mapping methods via the plugin interface?

if plugin_id is not None and plugin_id != PLUGIN_ID:
return
groups = (
await session.scalars(

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feels weird to allow the plugin to query the Access database directly, should the app groups be passed into this plugin method instead or maybe accessible from the app parameter?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants