feat(merge-users) Scalar user references and public sharing user rewrite (#17495) - #17577
Draft
Ludovic Bouges (ludovic) wants to merge 7 commits into
Draft
feat(merge-users) Scalar user references and public sharing user rewrite (#17495)#17577Ludovic Bouges (ludovic) wants to merge 7 commits into
Ludovic Bouges (ludovic) wants to merge 7 commits into
Conversation
The schema declares which attributes point at a User, but not what the merge should do with them: it says nothing about the register, about the lifecycle states splitting one attribute into two register rows, and it is silent on a few fields whose declarations are missing or wrong. Discovery walks the types one by one rather than through getIdAttributes(), which deduplicates by name and would collapse the several entities declaring user_id into one entry. A table alongside gives every discovered attribute a disposition, and a discovered attribute without one is reported as unassigned rather than silently rewritten. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Both the count and the update run off the same query builder, so what the dry run reported is what the real pass writes. A term filter goes to the keyword sub-field, booleans stay on the bare path, and a negated condition goes to must_not so documents missing the field are also selected. On a multiple field the script removes before appending, which is what makes a replay a no-op. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Some differences a handler reports change the security posture and should not be applied on the operator's behalf. The refusal sits in the engine, between the two passes: raising it in apply() would stop the merge after earlier handlers already wrote, and raising it in compute() would keep the difference out of the dry run report, which is the one place the operator can read it before deciding. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
One count per target, and a change emitted even at zero so the report names what was examined and not only what moved. Targets flagged as unexpected on an idle platform raise an alert when they match: a work still running means something was left on. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A feed, a taxii collection and a stream published under the source user become readable with the target user's markings and organizations. The handler reports the difference per endpoint and blocks the real pass when the target sees more than the source did. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The scalar handler reads the schema to build its targets, so registering it from the userMerge module would only see the modules imported before it. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## issue/17494 #17577 +/- ##
===============================================
- Coverage 34.18% 26.16% -8.02%
===============================================
Files 3386 3394 +8
Lines 137814 138034 +220
Branches 37256 37310 +54
===============================================
- Hits 47109 36114 -10995
- Misses 90705 101920 +11215
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Proposed changes
idand pointing at User. Discovery walks the entity types one by one rather than throughgetIdAttributes(), which deduplicates by attribute name and would collapse the several entities declaringuser_idinto a single entry.History,ActivityandPirHistorybelong to the history chunk,platform_ip_whitelist_exclusion_idsis invalidated rather than transferred, andUser-Account.user_idis the STIX observable property holding an account identifier on the observed system, declared as a user reference by mistake. Lifecycle splits:work.status,BackgroundTask.completedandNotification.is_readeach split one attribute into two register rows with different dispositions. Complements: nine paths the declarations do not expose, each recording why it is missing.creator_idismultiple, so the script removes the source before appending the target, which makes a replay a no-op.feed_public_user_id,taxii_public_user_idandstream_public_user_idare transferred, and the handler reports what each endpoint exposes anonymously before and after the transfer.blockingflag onUserMergeRightsAlert, raised by the public sharing handler when the target user carries markings or organizations the source did not have. The engine checks it between the two passes and refuses the real pass untilacknowledgeExposureChangeis set.modules/index.ts, since the scalar handler reads the schema and userMerge is imported before the workflow and custom field modules.The nine complements exist because
connector_user_id,NewsFeedItem.user_idandinitiator_idare declared with formatshort,PublicDashboard.user_idis not declared at all,InternalFile.metaData.creator_idsits under an object with no attribute definitions, and the workflow versioncreatedByfields are nested children declared withoutentityTypes. Fixing those declarations is out of scope:format: 'id'drives filter representative resolution, the filter key schema and history diff rendering, so changing it has effects well beyond this feature. A test fails once the schema starts yielding a complement, so the entry gets dropped rather than silently duplicating a discovered target.Filters, JSON manifests, STIX relations and RBAC have their own chunks. Hardening
resolveValidUseragainst a disabled account serving a public endpoint is tracked separately; transferring*_public_user_idmeans no endpoint points at the merged-away source, so that check is defence in depth.Related issues
How to test this PR
Stacked on #17535, so review the last 7 commits only. Enable the flag and start the platform:
Create some data owned by a source user — a few entities, an ingestion, a public feed — then run a dry merge. Each target reports a count, including the ones at zero, so the report names what was examined and not only what moved:
With a target user holding markings the source does not have, the public sharing handler must raise a
blockingalert, and the real run must refuse:The same call with
acknowledgeExposureChange: trueproceeds. Replaying it must report zero updates, since the rewrite is idempotent — worth checking specifically oncreator_id, which is a multi-valued field.Automated coverage:
Checklist
Further comments
72 unit tests and 27 integration tests. Discovery resolves to 17 targets, 12 motivated exclusions and no unassigned attribute — the same coverage an enumerated list would give, derived rather than written down, so a properly declared attribute added later is covered without touching this code.
The blocking refusal sits in the engine rather than in the handler. Raising it in
apply()would stop the merge after earlier handlers already wrote, and raising it incompute()would keep the difference out of the dry run report, which is the one place the operator can read it before deciding.