Repository navigation
feat: detect sensitive text with privmask's DetectionPipeline - #43
Merged
Merged
Conversation
NLTagger has no Japanese named-entity model, so Japanese personal names were effectively never found, and the broad Apple Intelligence prompt produced false positives such as port numbers read as addresses. Run the OCR text through privmask instead: deterministic detectors for phone numbers (with Japanese plausibility rules), emails, postal codes, addresses, My Number and credentials, and the on-device model for Japanese personal names only, verified against a surname list. Model findings go through the same pipeline so its precedence rules drop a name that wraps a phone number. Usernames/handles and card numbers are no longer covered; tracked in snaka/privmask#25 and #26. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
SensitiveDetectors+ the broad Apple Intelligence prompt) with privmask'sDetectionPipeline, pinned to0.4.x.JoinedLinesmaps each finding's UTF-16 range back to its line so the existing glyph-rect lookup still applies.Why
NLTaggerhas no Japanese NER, so Japanese names were effectively never detected; the broad AI prompt produced false positives (port numbers and IPs as addresses). privmask measured 9/9 Japanese names on text input and filters implausible phone numbers (order numbers, timestamps).Coverage changes
NLTagger).Test plan
swift test --package-path Packages/BokashiCore(39 tests, incl. newJoinedLinesTests)🤖 Generated with Claude Code