Add Google Group Management app-group-lifecycle plugin#532
Add Google Group Management app-group-lifecycle plugin#532somethingnew2-0 wants to merge 14 commits into
Conversation
a2bb9a8 to
67596c6
Compare
Add async create/delete/list helpers for Okta group push mappings on OktaService -- the link primitive the app group lifecycle plugins need to bind an Access-managed Okta group to an external group. delete PATCHes the mapping INACTIVE before DELETE (Okta rejects deleting active mappings), treats a 404 as success so the operation is idempotent, and defaults deleteTargetGroup to False so unlinking never removes the external group. list follows Link-header pagination so callers see every mapping. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…all_groups Thread the owning app's plugin config into the group-config-properties and group-config validation hooks so a plugin's group schema and validation can reflect app-level settings (e.g. an email pattern). The hookspec args are additive and pluggy passes only the args an impl declares, so existing implementations keep working. Add an `immutable` group-config flag enforced generically host-side on update -- a changed immutable field is rejected, while an error on an *unchanged* one is suppressed (it's locked and can't be actioned on this update). Add set_config_value and is_plugin_config_changed helpers. Rename the bulk-sync hook sync_all_group_membership to sync_all_groups (it reconciles all of an app's groups, not just memberships), keeping the old name as a deprecated -- still async -- alias so plugins on either name keep running, and add invoke_sync_all_groups to dispatch to both via run_hooks_to_completion. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… front Add ModifyGroupPluginData, which owns the group plugin_data merge (preserving other plugins' top-level entries and per-plugin status the patch omits) and fires the async group_updated hook only when the configured plugin's *configuration* actually changes -- never on status-only writes, which the plugin makes itself and which would otherwise re-trigger reconciliation in a loop. put_group now routes plugin_data through it. Validate group plugin config against the plugin's schema on both create (post_group) and update (put_group), returning 400 up front instead of failing later during reconcile, and let group-config-props accept ?app_id= so the returned schema reflects app-level constraints. Rename the periodic CLI command to `access sync-app-groups`, keeping `sync-app-group-memberships` as a hidden deprecated alias. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Surface each group-config field's validation.patterns as react-hook-form rules so the UI rejects out-of-pattern input before submitting (the backend stays authoritative). Render immutable fields read-only -- not disabled, so their value is still submitted -- when editing an existing app or group, with a helper-text note. Thread the owning app's id into the group-config-props lookup so the schema reflects app-level constraints. Regenerate the API client for the new app_id query param and the immutable config-property flag. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an opt-in example app-group-lifecycle plugin that creates, updates, and deletes Google groups for Access groups and links them to the Access-managed Okta group via Okta group push (which then owns membership sync). A single idempotent reconcile drives create/adopt/enforce and a periodic sync_all_groups sweep: Access is the source of truth, with graceful backfill from live Google/Okta state, an app-level email-prefix pattern, an immutable group email, and per-group sync status/errors. Written against the async (Access 2.0) plugin interface: the lifecycle hooks are async and receive the request's AsyncSession, Okta calls are awaited, and the blocking Cloud Identity client runs off the event loop via asyncio.to_thread. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… plugin Read the root log level from LOG_LEVEL (default INFO) so a deployed environment can raise verbosity (e.g. to trace app-group reconciliation) without a code change. Add a `make sync-app-groups` target that sources .env before running the CLI, since the app-group lifecycle plugins need credentials (Google/Okta) the other management targets don't load. Copy the example plugin source into the image at /app/plugins without installing it, so a downstream image can opt in by pip-installing it while the source stays single-homed here. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Resolve four issues raised in review of the Google group manager PR: - Partial group_config PATCHes no longer 400 on an omitted immutable field; a field counts as edited only when present in the patch, so an API update of just the mutable fields stops tripping the immutable one. - The Google plugin can now clear a group's description. Backfill from Google only happens during adoption (no Access-side config yet); once Access owns the config its description is authoritative, so emptying it clears Google instead of being silently backfilled from the stale value. - A single PUT that changes both name/description and plugin config now fires group_updated once instead of twice. ModifyGroupDetails and ModifyGroupPluginData gain a fire_lifecycle_hook opt-out (default True preserves standalone behavior) and put_group fires one consolidated hook, avoiding a double reconcile (Google + Okta round-trips). - Drop the deprecated sync_all_group_membership hook alias entirely rather than guard against dispatching it twice. The #511 async migration already broke every plugin implementing it (sync -> async) and sync_all_groups has never shipped, so the "backward-compatible" alias protected nobody while adding a double-reconcile footgun. Also removes the now-thin invoke_sync_all_groups wrapper and the deprecated sync-app-group-memberships CLI alias. Tests live alongside each fix. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
list/create/delete_group_push_mapping hand-rolled request_executor.execute(), which assumed okta 2.x's 2-tuple (OktaAPIResponse, error) return. The installed okta 3.4.4 (OpenAPI-generated) execute() returns (response, response_body, error) of raw aiohttp objects, so listing raised "ValueError: too many values to unpack (expected 2)" and the get_body()/pagination calls would have failed next. - Route all three through the inherited GroupPushMappingApi methods on the client proxy so they ride the same (data, resp, error) contract as every other list method and reuse _paginate for Link-header pagination. - create-or-link returns HTTP 200, but the SDK only types the spec'd 201, leaving resp.data empty; parse the mapping from resp.raw_data as a fallback. - Deactivate-then-delete now goes through update/delete_group_push_mapping; 404 idempotency moves to an except ApiException block since the typed client raises on HTTP errors rather than returning them in the tuple. - Add a server-side sourceGroupId filter to list_group_push_mappings so callers after one source group's mapping don't page the whole app. - Convert models to camelCase dicts via model_dump(by_alias=True); the model's own to_dict() drops the read-only id/sourceGroupId/targetGroupId fields. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rework the Google group manager so group creation goes through Okta group push rather than a Google-first create: Okta creates its target group AND the downstream Google group (named by the email prefix) and links them in one step, avoiding the manual-fetch import lag. This removes _create_google_group and the GOOGLE_WORKSPACE_CUSTOMER_ID requirement; the group's Cloud Identity id and metadata are resolved on a later reconcile once Okta has pushed it. - Route the adoption-time description backfill through the ModifyGroupDetails operation (ORM write + Okta sync + audit log) instead of mutating the group directly; suppress the group_updated hook to avoid re-entering this plugin. - Surface a sync error when an existing push mapping targets a Google group other than the group's configured email (a conflict that won't self-heal), instead of silently adopting the wrong group. - Read the googleGroupEmail custom attribute from the profile union's actual_instance.additional_properties (the 3.4.4 model shape), and use the new server-side sourceGroupId filter in discovery. - Fix last_synced_at to a timezone-aware datetime.now(timezone.utc). - Update README and tests to match. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Rebase integration onto main's stricter ty: put_group passed
body.plugin_data (typed dict[str, Any] | None) to _validate_group_plugin_config,
whose parameter is dict[str, Any]. Coerce None to {} for the validation call,
matching the sibling create-group call site and the pre-existing behavior; the
None handling for the actual update below is unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
| click.echo("Completed app group lifecycle plugin sync") | ||
|
|
||
|
|
||
| @cli.command("sync-app-groups") |
There was a problem hiding this comment.
Heads up on backward compatibility: sync-app-group-memberships (this command) and the sync_all_group_membership hook both exist on main today, so this is a rename of already-shipped surface, not a net-new name. There is no compatibility alias, so:
- any external scheduler invoking
access sync-app-group-memberships(for example a periodic job defined outside this repo) breaks with "No such command" on deploy; and - any third-party lifecycle plugin still implementing
sync_all_group_membershipsilently stops being invoked, since it no longer matches a hookspec.
There was a problem hiding this comment.
Accepting this rename without a compatibility alias, intentionally. This ships as part of Access 2.0, which introduces a number of breaking changes (e.g. the async hook specs), so we're deliberately omitting the deprecated command/hook shims that would preserve backward compatibility. External schedulers and third-party plugins will need to move to the new sync-app-groups command and hook name as part of the 2.0 upgrade.
69f9ade to
11a5fd2
Compare
…elds
Add an optional `suffix` to app-group-lifecycle plugin config properties,
rendered as an inline end-adornment on the app/group edit form's text input.
This lets a plugin show static trailing context (e.g. an email domain) while
the stored value stays just what the user types.
- api/plugins + api/schemas: add `suffix` to AppGroupLifecyclePluginConfigProperty
and its PluginConfigProp wire schema (the router's asdict passthrough needs no
change).
- src/api: regenerated client exposes the new optional field.
- src/components: ConfigField renders `suffix` as a MUI InputAdornment on text
fields, omitted when unset.
- google plugin: set suffix="@{domain}" on the email-prefix field so operators
see the full address; the local part remains the stored value.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…alidation - ModifyGroupPluginData (#1): compute config_changed against the fully-merged effective config, snapshotting the pre-merge config first (the partial-patch merge mutates it in place). A partial patch that only omits an unchanged config key (e.g. the immutable email, now permitted by the present-only immutable check) no longer misreads as a change and fires a redundant reconcile. - put_group + ModifyGroupType (#2): fire exactly one app group lifecycle hook per PUT, after every field is applied. ModifyGroupType gains fire_created_hook so put_group can defer the group_created fire to the end (after plugin_data is applied) and choose the event by the transition: group_created on convert-to-app-group, group_updated on a same-type edit, group_deleted (unchanged, still in ModifyGroupType, which needs the pre-conversion row) on convert-away. Previously a convert-to-app-group + rename fired both group_created (before the config was applied) and group_updated -- a double reconcile that also hid the new config from group_created. - put_group validation (#3): catch AppGroupLifecyclePluginFilteringError (a plain Exception, raised when a plugin doesn't answer a hook with exactly one response) and surface a clean 500 rather than an unhandled stack trace. Tests cover the three type transitions (single fire; group_created sees final config), the partial-patch no-redundant-reconcile case, and the filtering-error 500. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Critical: - delete_group_push_mapping: the 404-idempotency was dead code. This SDK returns HTTP errors in the result tuple (it does not raise ApiException), and delete returns a 2-tuple on error vs a 3-tuple on success -- so the old `_, _, error =` unpack raised ValueError and the `except ApiException` never fired. Read the trailing error element arity-safely and key idempotency on error.status == 404 at both the deactivate and delete steps. Rewrote the tests to simulate returned error tuples (they had mocked a raised ApiException, which the SDK never does) and added a deactivate-step-404 case. Important: - Validate group plugin_data on the convert-to-app_group PUT path (previously skipped because the group was still its old type), against the target app; old_plugin_data=None since the conversion attaches the plugin fresh. - Add a parity test asserting the plugin dataclasses and their Pydantic wire models expose identical fields, so silent asdict()->model_validate() drift fails loudly. - Test gaps: first-ever config set fires group_updated; second-pass reconcile skips re-create when the mapping is already recorded; _reconcile persists SYNC_ERROR and re-raises on an unexpected failure. Suggestions: - Freeze the plugin descriptor dataclasses; reject suffix on non-text config properties at construction; drop the defensive getattr(prop, "immutable"). - create_group_push_mapping: replace asserts on the external response with explicit errors. - sync_all_groups: count per-group failures and log an aggregate error line. - _discover_existing_push_mapping: log un-interpretable mappings at WARNING (duplicate risk). - Comment fixes: stale get_body()/next-cursor references, the fire_lifecycle_hook wording, and the stale "GitHub team" example. Deliberately not done: strictly typing the `validation` field (a legitimately open, plugin-defined bag -- the audit-logger example uses `allowed_values`, not `patterns`), and moving the plugin-data audit into ModifyGroupPluginData (single caller; would change audit semantics on the convert+plugin_data path). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
| async def _delete_google_group(self, google_group_id: str) -> None: | ||
| await asyncio.to_thread(lambda: self._groups_api.delete(name=self._resource_name(google_group_id)).execute()) |
There was a problem hiding this comment.
_delete_google_group is the only Google-side path that isn't tolerant of an already-absent group. Every read path (_lookup_google_group_id, _email_from_status, _owned_group_id) tolerates absence, but this bare delete().execute() does not.
In group_deleted the mapping unlink is wrapped in a best-effort try/except, but this call runs outside it. So if the Google group was already deleted out of band (or on a delete replay/retry, which the reconcile design intends to be safe), delete() raises, the group_deleted hook raises, and invoke_app_group_lifecycle_hook rolls back instead of committing. The "authoritative cleanup" the surrounding comment describes then fails on a group that is already gone.
Note it should NOT reuse _is_group_absent_error here: that treats 403 as absent, whose rationale ("a genuine permission problem surfaces on the subsequent create call") does not hold on the delete path. A 403 on delete most likely means the service account lacks delete permission on a group that still exists; swallowing it would log a phantom successful deletion and leak the group. Only a true 404 is an unambiguous "already gone":
| async def _delete_google_group(self, google_group_id: str) -> None: | |
| await asyncio.to_thread(lambda: self._groups_api.delete(name=self._resource_name(google_group_id)).execute()) | |
| async def _delete_google_group(self, google_group_id: str) -> None: | |
| try: | |
| await asyncio.to_thread(lambda: self._groups_api.delete(name=self._resource_name(google_group_id)).execute()) | |
| except HttpError as e: | |
| # 404 -> already gone, so a delete replay is a no-op. Deliberately 404-only (not | |
| # _is_group_absent_error): a 403 on delete more likely means we lack permission on a | |
| # group that still exists, which must surface rather than masquerade as a deletion. | |
| if getattr(getattr(e, "resp", None), "status", None) != 404: | |
| raise |
(One thing to confirm: that Cloud Identity's groups.delete returns 404, not a conflated 403, for an already-deleted group. If it returns 403, a replay would still error here, which is the safe direction.)
| # Ship prod-ready plugin source into the image WITHOUT installing it here. | ||
| # Downstream images opt in by pip installing the plugins, which keeps the source | ||
| # single-homed in this repo while leaving activation to the consumer. | ||
| COPY ./examples/plugins/app_group_lifecycle_google ./plugins/app_group_lifecycle_google |
There was a problem hiding this comment.
Can we rebase this off of #552 and add this plugin install as optional, but enable it via a build argument in the docker-image.yml Github Action?
There was a problem hiding this comment.
We don't have any other frontend tests and nothing runs them in CI. Do we want to start including these or should they be removed?
| ) | ||
|
|
||
|
|
||
| class ModifyGroupPluginData: |
There was a problem hiding this comment.
Should ModifyGroupDetails call into this new operation instead of it running invoke_app_group_lifecycle_hook(AppGroupLifecycleHook.GROUP_UPDATED...) itself?
| set_config_value, | ||
| set_status_value, | ||
| ) | ||
| from api.services import okta # used by the push-mapping/discovery helpers |
There was a problem hiding this comment.
Not sure how much I like plugins being able to call Access API methods directly. I wonder if there's a way we can expose the Okta group mapping methods via the plugin interface?
| if plugin_id is not None and plugin_id != PLUGIN_ID: | ||
| return | ||
| groups = ( | ||
| await session.scalars( |
There was a problem hiding this comment.
Feels weird to allow the plugin to query the Access database directly, should the app groups be passed into this plugin method instead or maybe accessible from the app parameter?
Takes over and rebases #498 (originally by @barborico) onto current
main. See that PR for fuller description, validation, and screenshots.What
Adds an opt-in example app-group-lifecycle plugin that mirrors each Access group to a Google group and links them via Okta group push (Okta then owns membership sync), plus the core-Access enablers it needs:
OktaServicegains group push-mapping create/delete/list helpers.immutablefield flag enforced host-side on update and rendered read-only in the UI.ModifyGroupPluginDataoperation owns theplugin_datamerge and firesgroup_updatedonly when a plugin's configuration actually changes — never on status-only writes, which would otherwise re-trigger reconciliation in a loop. Group plugin config is validated up front on create/update (400 instead of a later reconcile failure).sync_all_group_membershipis renamed tosync_all_groups(it reconciles all group state, not just membership), with a renamedaccess sync-app-groupsCLI command./app/pluginswithout installing it, so a downstream image opts in by pip-installing it.Why this is a fresh rebase, not a merge
#498 was branched before three things now on
main: the async-SQLAlchemy migration (#480/#481), theuv/tytoolchain (#507), and — most relevant — the async plugin interface (#511). So this isn't a mechanical rebase: every backend surface #498 added had to be reworked for async. The plugin's lifecycle hooks are nowasync defand receive the requestAsyncSession; Okta calls are awaited; the blocking Cloud Identity client runs off the event loop viaasyncio.to_thread; and the operation/router/CLI paths were converted to awaited async SQLAlchemy. The 19 original commits are re-expressed as 6 logically-grouped, async-native commits.The plugin's own behavior (idempotent reconcile, Access-as-source-of-truth with backfill from live Google/Okta state, immutable group email, per-group sync status) is unchanged from #498 — see that PR's description and screenshots for the functional walkthrough.
Rebase and async conversion by Claude; original plugin design and implementation by @barborico in #498.
🤖 Generated with Claude Code