Skip to content

fix(rename): read and write source files as UTF-8 - #6761

Open
anxkhn wants to merge 3 commits into
reflex-dev:mainfrom
anxkhn:patch-13
Open

fix(rename): read and write source files as UTF-8#6761
anxkhn wants to merge 3 commits into
reflex-dev:mainfrom
anxkhn:patch-13

Conversation

@anxkhn

@anxkhn anxkhn commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

All Submissions:

  • Have you followed the guidelines stated in CONTRIBUTING.md file?
  • Have you checked to ensure there aren't any other open Pull Requests for the desired changed?

Type of change

  • Bug fix (non-breaking change which fixes an issue)

What and why

reflex rename <new_name> walks the user's app and rewrites imports and the app
name in every .py and .md file. The read/modify/write in
rename_imports_and_app_name (reflex/utils/rename.py) used
Path.read_text() and Path.write_text() with no encoding= argument, so both
default to the platform's locale encoding. On a Western Windows locale that is
cp1252, not UTF-8.

The consequences on non-UTF-8 locales:

  • Silent corruption (Western Windows, cp1252): a UTF-8-authored file that
    contains non-ASCII bytes (curly quotes in docstrings, accented identifiers or
    string literals, emoji) is decoded as cp1252, rewritten, and written back as
    cp1252, mangling the user's own source.
  • Hard failure (CJK codepages): read_text() raises UnicodeDecodeError,
    aborting the rename partway through the tree and leaving some files already
    rewritten and others not, i.e. an inconsistent app.

On a POSIX UTF-8 locale it round-trips, so this is effectively Windows-specific.

The fix passes encoding="utf-8" on both calls. This mirrors the repo's own
reflex/utils/path_ops.py find_replace, which already reads and writes the same
read/replace/write pattern with encoding="utf-8"; rename.py was the outlier.

Changes

  • reflex/utils/rename.py: add encoding="utf-8" to the read_text() and
    write_text() calls in rename_imports_and_app_name (+2 / -2).
  • tests/units/test_prerequisites.py: add regression test
    test_rename_imports_and_app_name_preserves_utf8, colocated with the existing
    rename tests. It simulates a cp1252 default (monkeypatching Path.read_text /
    write_text so a missing encoding falls back to cp1252, while an explicit
    encoding= still passes through), writes a UTF-8 file with realistic non-ASCII
    content, runs the rename, and asserts the bytes are preserved exactly and the
    rename still happened. The test fails before the fix (UnicodeDecodeError) and
    passes after.
  • news/<pr>.bugfix.md: towncrier changelog fragment.

Changes To Core Features:

  • Have you added an explanation of what your changes do and why you'd like us to include them? (above)
  • Have you written new tests for your core changes, as applicable?
  • Have you successfully ran tests with your changes locally?

Locally (all green):

  • uv run pytest tests/units/test_prerequisites.py -k rename -> 6 passed
  • uv run pytest tests/units/test_prerequisites.py tests/units/utils -> 677 passed, 4 skipped
  • uv run ruff check . (changed files) -> All checks passed
  • uv run ruff format --check . (changed files) -> already formatted
  • uv run pyright reflex/utils/rename.py tests/units/test_prerequisites.py -> 0 errors
  • uv run towncrier check --compare-with origin/main -> fragment found

@anxkhn
anxkhn requested a review from a team as a code owner July 14, 2026 07:50
@greptile-apps

greptile-apps Bot commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR makes reflex rename preserve source-file encodings across platforms. The main changes are:

  • Detect Python source encodings with tokenize.detect_encoding.
  • Read and write each Python file with its detected encoding.
  • Use UTF-8 explicitly for supported non-Python text files.
  • Add byte-level tests for UTF-8 and cp1252 source files.
  • Add a bug-fix changelog entry.

Confidence Score: 5/5

This looks safe to merge.

  • The declared-encoding failure is addressed with Python's standard source-encoding detection.
  • Reads and writes now use the same detected encoding.
  • Tests cover both UTF-8 files on non-UTF-8 locales and cp1252 encoding declarations.
  • No blocking issues were found in the updated code.

Important Files Changed

Filename Overview
reflex/utils/rename.py Detects declared Python source encodings and reuses them for reading and writing during rename.
tests/units/test_prerequisites.py Adds byte-level tests for UTF-8 under a cp1252 locale and Python source declared as cp1252.
news/6714.bugfix.md Documents the cross-platform source-encoding fix.

Reviews (4): Last reviewed commit: "test(rename): handle platform newlines" | Re-trigger Greptile

Comment thread reflex/utils/rename.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 657b04cfd5

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread reflex/utils/rename.py Outdated
@Alek99 Alek99 closed this Jul 15, 2026
@Alek99 Alek99 reopened this Jul 15, 2026
@codspeed-hq

codspeed-hq Bot commented Jul 15, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 26 untouched benchmarks
⏩ 8 skipped benchmarks1


Comparing anxkhn:patch-13 (4f7d658) with main (1358e00)

Open in CodSpeed

Footnotes

  1. 8 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

anxkhn added 3 commits July 28, 2026 00:04
reflex rename reads every .py/.md file in the user's app with
Path.read_text() and writes it back with Path.write_text(), neither
of which passed encoding=. Both default to the platform locale
encoding, which on Western Windows is cp1252 rather than UTF-8.

UTF-8-authored source containing non-ASCII bytes (curly quotes, emoji,
accented identifiers or docstrings) is therefore silently corrupted on
a Western Windows locale, or aborts the rename partway with
UnicodeDecodeError on a CJK codepage, leaving the tree in an
inconsistent state. Pass encoding="utf-8" on both calls, matching
path_ops.find_replace which already does this.

Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
Signed-off-by: Anas Khan <83116240+anxkhn@users.noreply.github.com>
@anxkhn

anxkhn commented Jul 27, 2026

Copy link
Copy Markdown
Contributor Author

fixed the windows ci failures in 4f7d658. the encoding assertions remain byte-exact while accounting for the platform newline emitted by the existing text-write behavior. focused tests, ruff, and pyright pass locally. @reflex-dev please take another look when ci completes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants