Repository navigation
Decode git output and read prompts as UTF-8 - #33
Merged
Merged
Conversation
The git helpers relied on text=True, which decodes subprocess output using the locale's preferred encoding. Under a non-UTF-8 locale that mangles or crashes on non-ASCII bytes in commit messages, filenames, or diffs. Decode explicitly as UTF-8, and use errors="replace" for the porcelain status and diff, where arbitrary file content can appear. Split out from #30.
Contributor
There was a problem hiding this comment.
Pull request overview
This PR makes git metadata collection in tiptop/recording.py decode subprocess output explicitly as UTF-8, avoiding dependence on the system locale encoding when recording run metadata (including dirty status and diffs).
Changes:
- Use
encoding="utf-8"for git subprocess output decoding in_get_git_root,_collect_git_info, and_get_git_diff. - Add
errors="replace"forgit status --porcelainandgit diffoutput to prevent decode failures when arbitrary bytes appear in status/diff content.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Path.read_text() decodes with the locale's preferred encoding, which mangles or crashes on non-ASCII prompt text under a non-UTF-8 locale. Decode explicitly as UTF-8, matching the git-output fix.
williamshen-nz
changed the title
ryanlindeborg
approved these changes
Jul 30, 2026
6 tasks done
2 tasks done
2 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Splits the UTF-8 file/output-decoding fixes out of #30 into their own PR. Both are the same class of issue: relying on the locale's preferred encoding, which mangles or crashes on non-ASCII bytes under a non-UTF-8 locale.
Changes
recording.py— the git helpers usedtext=True, which decodes subprocess output with the locale encoding. Decode git output explicitly asencoding="utf-8"in_get_git_root,_collect_git_info, and_get_git_diff, and adderrors="replace"for thegit status --porcelainandgit diffcalls where arbitrary file content can appear.gemini.py—load_promptusedPath.read_text()(locale encoding); read prompt templates explicitly asencoding="utf-8".