Skip to content

Validate scroll deltas and live-screen input on the Bot computer - #488

Merged
davidmckayv merged 2 commits into
CopilotKit:mainfrom
Ayush7614:fix/agent-computer-input-validation
Sep 11, 2026
Merged

Validate scroll deltas and live-screen input on the Bot computer#488
davidmckayv merged 2 commits into
CopilotKit:mainfrom
Ayush7614:fix/agent-computer-input-validation

Conversation

@Ayush7614

Copy link
Copy Markdown
Contributor

What this changes

Two related holes on the Bot computer, both the same bug class already fixed for navigate URLs and exec timeouts (400 naming the field, not a 502 reading as a broken computer):

  1. Scroll deltas: /scroll, /human/scroll (and the internal fallbacks) accepted any typeof === number. Infinity (valid JSON: 1e999) travelled into mouse.wheel and came back as a 502. New parseScrollDelta (mirroring parseExecTimeout) requires a finite number; absent still means the endpoint default (600 Bot / 400 human). Edge returns 400 with the same message the server gateway uses; the performers also refuse defensively.
  2. Live-screen socket: any JSON object passed the cast and fell through to Input.insertText({ text: undefined }), or forwarded wrong-typed x/y/key/code into CDP. New parseInputMessage discriminates mouse/wheel/key/text, requires events, finite coordinates/deltas, and non-empty key/code/text; malformed frames get an error reply and never reach CDP.

Where it runs

The computer process holding one Bot browser. No shared state: each computer validates its own requests.

  • New state that outlives a request? None. Per-request validation.
  • What happens on the second replica? Same 400/error on every computer: pure function of the request/frame. No cross-computer coordination.
  • Anything serialised? No.
  • Anything fanned out to a browser? Validation happens before Playwright/CDP calls, on the computer that owns that browser.
  • New listener, port, or schedule? No.

Postgres is already there and is the default answer to all of the above: nothing here needs it.

Boundary and audit

  • Every acting call still goes through the gateway: resolve, decide, audit, then act. The computer holds no policy engine; this checks well-formedness only, same as the existing navigate/timeout checks.
  • New refusals and new failures each write a row. N/A on this process: 400s here are caller errors answered at the edge, matching the existing navigate/timeout behaviour; gateway audit ordering is untouched.
  • Nothing new is trusted from the client that the server can resolve itself. Stricter: fewer shapes reach Playwright/CDP.

Changelog

  • A line in CHANGELOG.md under Unreleased: scroll/input validation entry.

Proof

  • bun test agent-computer/tests/request-validation.test.ts: 44 pass (scroll absent/finite/infinite/string/null/boolean/object; mouse/wheel/key/text well-formed and malformed cases).
  • bun run format:check: clean. bun run lint: clean.
  • Server/worker typecheck pass; app has the pre-existing embla-carousel-react failure identical on clean upstream main.

@davidmckayv davidmckayv left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deep-reviewed clean (validation, no secret leak, fail-closed, agrees with existing layers). CI green.

@davidmckayv
davidmckayv merged commit e197209 into CopilotKit:main Sep 11, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants