Skip to content

feat: Optimize linting by checking changed files - #5939

Open
another-rex wants to merge 10 commits into
google:masterfrom
another-rex:optimize_linting_incremental_ci
Open

feat: Optimize linting by checking changed files#5939
another-rex wants to merge 10 commits into
google:masterfrom
another-rex:optimize_linting_incremental_ci

Conversation

@another-rex

Copy link
Copy Markdown
Contributor

Trying to do quick fixes really make you find out where the pain points in the CI.

Turned the lint and format script into a python script (with no dependencies). We now tune a few settings to make it run a lot faster.

  1. Make use of the parallel processing flags for python lints
  2. Only run it on files that changed (In CI it's against the master branch).

@another-rex another-rex changed the title Optimize linting incremental ci feat: Optimize linting incremental ci Aug 31, 2026
@another-rex another-rex changed the title feat: Optimize linting incremental ci feat: Optimize linting by checking changed files Aug 31, 2026

@michaelkedar michaelkedar left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems marginally over-engineered.

(also increasing the amount of Python in the repo 😔)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is related 🤔

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh... shouldn't this not be a problem since it's already merged in? Maybe merging master into this might fix this.

Comment thread tools/lint_and_format.py Outdated
Comment on lines +258 to +264
# Dependency propagation for Go
if "bindings/go" in affected_mods and "go" in all_go_mods:
affected_mods.add("go")
if "go" in affected_mods:
for dep in ("go/cmd/tools/reimport-tui", "tools/repo-allowlist-sync"):
if dep in all_go_mods:
affected_mods.add(dep)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This hard-coded propagation is annoying to maintain, and I don't really understand its purpose - is it saying that if a dependency changes, then also lint the thing depending on it?

I don't really see why that would be necessary?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah idk, removed now.

Comment thread tools/lint_and_format.py
- If 'bindings/go' changes -> both 'bindings/go' and 'go' are linted.
- If 'go' changes -> 'go', 'go/cmd/tools/reimport-tui', and
'tools/repo-allowlist-sync' are linted.
* Skipped (0.0s) when no in-scope Go modules changed.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why?

@michaelkedar

Copy link
Copy Markdown
Member

Also, generally you're using double quotes for strings, when the python style guide says single quotes should be used

@another-rex

Copy link
Copy Markdown
Contributor Author

Simplified it a bit, now just finds all the targets (e.g. go.mod files), figures out the diffs and which belongs to which target, and then run the linter against the appropriate target.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants