Skip to content

Commit 54227cb

Browse files
committed
Merge main into a Bot's browser
Brings in #879: a Bot's browser now also closes when the computer's memory runs high and its idle background tabs close, sparing any browser whose wheel a person holds, so a recording is never cut off by either.
2 parents b9a270b + f4083dd commit 54227cb

48 files changed

Lines changed: 4853 additions & 425 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.env.example‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -359,6 +359,20 @@ COMPUTER_TOKEN=
359359
# COMPUTER_BROWSER_BACKEND=managed
360360
# COMPUTER_BROWSER_MODE=headed
361361

362+
# When a computer closes a browser besides the cap (COMPUTER_MAX_BROWSERS) and the idle timeout
363+
# (COMPUTER_BROWSER_IDLE_MS). At or above this share of its memory limit, read as the kubelet reads it
364+
# (the working set, so the page cache does not count), it closes browsers nobody has used for a
365+
# minute, least recently used first, until it is ten points under. Memory is read before each launch
366+
# and once a minute, so a page that grows between two checks can still run it out of memory. A
367+
# container with no memory limit is measured against the machine's memory; a computer outside a
368+
# container closes nothing for memory. In the all-in-one image the share counts the database and the
369+
# API server too, on purpose: the out-of-memory kill counts them as well. 0 switches it off.
370+
# COMPUTER_MEMORY_HIGH_PERCENT=85
371+
# How long a Bot's background tab may sit untouched before it is closed, in ms: a tab the Bot left
372+
# behind, such as the page a sign-in popup opened from. The page the Bot is on, a person's own
373+
# browser, and any tab of a Bot whose wheel a person holds are never closed. 0 keeps them.
374+
# COMPUTER_TAB_IDLE_MS=600000
375+
362376
# Web Bot Auth: every request a Bot's browser makes is signed as coming from this deployment
363377
# (Ed25519, HTTP Message Signatures), and the public key is published at
364378
# <OPENBOT_PUBLIC_URL>/.well-known/http-message-signatures-directory. Off unless set to on, because

‎.github/workflows/ci.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -142,7 +142,7 @@ jobs:
142142
tests/webrtc-proxy.test.ts tests/web-bot-auth-browser.test.ts
143143
tests/handoff-page-restore.test.ts tests/enter-text.test.ts tests/person-browser.test.ts tests/person-browser-computer.test.ts
144144
tests/bot-browser.test.ts tests/bot-browser-computer.test.ts
145-
tests/screencast-keys-browser.test.ts
145+
tests/screencast-keys-browser.test.ts tests/browser-memory.test.ts
146146
working-directory: agent-computer
147147
env:
148148
OPENBOT_COMPUTER_BROWSER: '1'

‎CHANGELOG.md‎

Lines changed: 73 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ Newest first. `Unreleased` is what is on `main` and not yet tagged.
88

99
## Unreleased
1010

11-
**Before upgrading.** Fourteen things change for an existing deployment:
11+
**Before upgrading.** Fifteen things change for an existing deployment:
1212
- A hosted computer's browser is now headed by default. It uses a little more memory than headless
1313
did; set `COMPUTER_BROWSER_MODE=headless` (Helm: `computers.browserMode: headless`) to keep the old
1414
behaviour.
@@ -61,6 +61,55 @@ Newest first. `Unreleased` is what is on `main` and not yet tagged.
6161
to keep the old behaviour.
6262
- Interfaces a Bot drew before the upgrade (A2UI 0.9 surfaces) no longer draw when a conversation is
6363
reopened; the messages around them stay. New ones use A2UI 1.0 (below).
64+
- A computer now closes browsers nobody is using, least recently used first, when its memory
65+
reaches 85% of its limit, and closes a Bot's background tabs after ten minutes untouched (below).
66+
In the all-in-one image that share counts the database and the API server too.
67+
`COMPUTER_MEMORY_HIGH_PERCENT=0` and `COMPUTER_TAB_IDLE_MS=0` (Helm: `computers.memoryHighPercent`
68+
and `computers.tabIdleMs`) keep the old behaviour.
69+
70+
### A computer closes browsers nobody is using when its memory runs high
71+
72+
A computer runs one browser for each Bot that is browsing, and the only limits were a count and half
73+
an hour untouched. Neither looked at memory, so a few heavy pages could take a container past its
74+
memory limit well under the count, and the kernel then killed processes without regard to which Bot
75+
was busy. On Kubernetes it kills everything in the container at once, so every Bot's browser closed
76+
together and nothing could reach the computer until it came back. A computer now reads its own
77+
memory the way the kubelet does (the working set, which leaves out the file cache the kernel can
78+
drop), before it starts a browser and once a minute. At or above 85% of its limit it closes browsers
79+
one at a time, least recently used first, until it is under 75%, so the next launch does not cross
80+
the line again straight away. Launches that arrive together share one pass rather than closing a
81+
browser each. A Bot whose browser was closed starts again where it left off on its next step, still
82+
signed in, because its profile is on disk.
83+
84+
Only a browser nobody is using is closed. Never closed for memory: the browser being started, the
85+
most recently used one, any browser used in the last minute (a Bot between two steps asks for
86+
nothing while its model works out the next), a person's own browser, cloud or native, whose Recorder
87+
session would end, and the browser of a Bot whose wheel a person holds or has been asked to take.
88+
When every browser is in use, none is closed.
89+
90+
This is a check, not a guarantee. Memory is read before each launch and once a minute, so a page
91+
that grows between two checks can still run the container out of memory. A container with no memory
92+
limit is measured against the machine's memory, and a computer outside a container, such as OpenBot
93+
Desktop's, closes nothing for memory. In the all-in-one image the database and the API server run in
94+
the same container as the browsers, and the share counts them too. That is deliberate: the kernel's
95+
out-of-memory kill counts everything in the container and can take the database down, and closing a
96+
browser nobody is using first is how the database is protected. Each close is logged as
97+
`computer-browser-closed-for-memory` with the working set and the limit in bytes.
98+
`COMPUTER_MEMORY_HIGH_PERCENT` (Helm: `computers.memoryHighPercent`) sets the share where closing
99+
starts, and `0` switches it off.
100+
101+
The tabs a Bot leaves behind are closed too. A Bot always works in its newest page, so an older tab,
102+
such as the page a sign-in popup was opened from or a link a site opened in a new tab, is one it
103+
cannot go back to, and it held memory for as long as the browser stayed up. A background tab that
104+
nothing has used, navigated or brought to the front for ten minutes is now closed. The page the Bot
105+
is on is never closed, and neither is any tab of a person's own browser or of a Bot whose wheel a
106+
person holds. `COMPUTER_TAB_IDLE_MS` (Helm: `computers.tabIdleMs`) sets the time, and `0` keeps them.
107+
108+
Through Helm, `computers.browserIdleMs` and `computers.maxBrowsers` now reach the computer as
109+
written. An hour (`browserIdleMs: 3600000`) used to render as `3.6e+06`, which the computer ignored
110+
for its default of thirty minutes, and `browserIdleMs: 0` (keep browsers open) was dropped. All four
111+
of these settings now also reach the computers `computers.mode: sandbox` makes, which had none of
112+
them.
64113

65114
### Generated interfaces use A2UI 1.0
66115

@@ -100,6 +149,12 @@ The newest come first, and a press on When turns the order round. A search box a
100149
Started by buttons narrow the events already loaded, and the count says how many of them match. A
101150
row about a Bot (a message routed to it, say) now names it under Bot.
102151

152+
### The desktop setup page does not freeze when it loads during an install
153+
154+
If the setup page loaded again while the desktop app downloaded local software, the window showed a
155+
spinning cursor until the download finished. Its checks for a running OpenBot and for the last
156+
failure now wait in the background, so the window keeps answering.
157+
103158
### Each Bot on a shared computer has its own files
104159

105160
A computer that serves many Bots kept one folder of files for all of them, so a note one Bot saved
@@ -385,6 +440,13 @@ apps, which nobody could connect there. An app's own page, reached from an older
385440
available yet and that an administrator has to finish setting it up, instead of naming the setting.
386441
The administrator's page for the app still names `COMPOSIO_API_KEY`.
387442

443+
### A new desktop install of v0.2.0 gets past Install again
444+
445+
A desktop app built from `main` stopped every new install at "Installing OpenBot" with
446+
`shared/package.json: No such file or directory`. It had started reading `shared`, `toolkit` and
447+
`extension` as packages, which v0.2.0 does not have: there `shared` is plain source and there is no
448+
`toolkit` or `extension`. It now reads them only when the release includes them.
449+
388450
### A built-in Bot behind an OpenAI proxy answers again
389451

390452
A built-in Bot behind a gateway or proxy in front of OpenAI (an `OPENAI_BASE_URL` that is not OpenAI or
@@ -646,6 +708,16 @@ close it, and the reason is in the server's and the computer's logs.
646708
Clicking or focusing the desktop window while it downloaded local software showed a spinning cursor
647709
until the download finished. The window now comes forward at once and keeps the setup screen.
648710

711+
### A reinstall no longer picks up the previous installation's Bot files and sign-ins
712+
713+
A desktop installation set up again after its folder was deleted reused whatever the previous one
714+
left in the container engine: Bot workspace files, and each Bot's browser profile with its website
715+
sign-ins, under the same Bot ids. Only the database was checked. Start now finds every volume and
716+
stopped container the previous installation left and offers Start fresh, which deletes exactly those
717+
and then starts; nothing is reused without asking. To keep the data, restore the previous folder
718+
instead. A container from it that is still running stops the deletion until it is stopped. An
719+
installation with its saved key keeps its data as before.
720+
649721
### A new desktop install starts its server again
650722

651723
A desktop install of v0.2.0 stopped at start with `server stopped straight away (exit status: 1)` and

‎agent-computer/src/browser-eviction.ts‎

Lines changed: 122 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
/**
2-
* Which Bots' browsers to close, and when.
2+
* Which Bots' browsers to close, and when, and which of a Bot's tabs.
33
*
44
* Its own file, with no Playwright import, for the reason `authorisation.ts` and `bot-id.ts` are:
55
* `profiles.ts` imports Playwright at module scope, so anything living there needs a browser merely
@@ -62,21 +62,96 @@ export function chooseEvictions(
6262
const entries = [...running];
6363
if (entries.length <= max) return [];
6464
// A cap of zero keeps nothing, the newest included; that is the mistake reading the setting guards.
65-
const newest =
66-
max > 0
67-
? entries.reduce((latest, entry) =>
68-
entry[1].usedAt >= latest[1].usedAt ? entry : latest,
69-
)
70-
: undefined;
65+
const newest = max > 0 ? newestOf(entries) : undefined;
7166

67+
return inClosingOrder(entries, newest, goesLast)
68+
.slice(0, entries.length - max)
69+
.map(([botId]) => botId);
70+
}
71+
72+
/** The most recently used, which is the one a launch has just added. Ties go to the later entry. */
73+
function newestOf(
74+
entries: [string, Evictable][],
75+
): [string, Evictable] | undefined {
76+
if (entries.length === 0) return undefined;
77+
return entries.reduce((latest, entry) =>
78+
entry[1].usedAt >= latest[1].usedAt ? entry : latest,
79+
);
80+
}
81+
82+
/**
83+
* The order browsers are closed in when some have to go: every Bot's before a person's native window
84+
* (`keptOpen`), and least recently used first within each. `spare` is left out, which is the newest.
85+
*/
86+
function inClosingOrder(
87+
entries: [string, Evictable][],
88+
spare: [string, Evictable] | undefined,
89+
goesLast: (botId: string) => boolean,
90+
): [string, Evictable][] {
7291
return entries
73-
.filter((entry) => entry !== newest)
92+
.filter((entry) => entry !== spare)
7493
.sort(
7594
([aId, a], [bId, b]) =>
7695
Number(goesLast(aId)) - Number(goesLast(bId)) || a.usedAt - b.usedAt,
96+
);
97+
}
98+
99+
/**
100+
* How long after its last use a browser still counts as in use, and so is never closed for memory.
101+
*
102+
* A browser's use is recorded when something asks for its page: a Bot's step, a person's input, the
103+
* live screen. Between two steps a Bot asks for nothing while its model works out the next one, which
104+
* can take tens of seconds, and nothing here can tell that pause from a Bot that has finished.
105+
* Closing the browser then closes it in the middle of a run: the page the Bot was on is gone, and its
106+
* next step starts a cold browser, which is the very jump in memory the close was meant to head off.
107+
* A minute covers a model's turn between steps. It also covers a browser that has just launched, so
108+
* the next launch never closes the one before it, which is what would otherwise turn a full computer
109+
* into a loop of each Bot's launch closing the last Bot's browser.
110+
*/
111+
export const RECENTLY_USED_MS = 60_000;
112+
113+
/**
114+
* Which to close next because the computer's memory runs high (`container-memory.ts`).
115+
*
116+
* One at a time, because the caller reads memory again after each close and stops as soon as it is
117+
* enough: a count says in advance how many have to go, and bytes do not. Least recently used first,
118+
* the cap's order, so the two never disagree about who goes first.
119+
*
120+
* Never a browser in use, which here means four things:
121+
* - The one being launched. The caller runs this before a launch and passes only the browsers
122+
* already running, so the Bot that asked is not among them.
123+
* - The most recently used of those, however long ago that was. So at least one browser is always
124+
* left, because one browser that is on its own too big for the computer is the limit's problem,
125+
* and closing it would only start it again on its next step.
126+
* - Any used within `recentlyUsedMs` ({@link RECENTLY_USED_MS}), which is a Bot in the middle of a
127+
* run or a browser that has just started.
128+
* - Any `spared` names, which is a Bot whose wheel a person holds or has been asked to take: closing
129+
* it closes it under somebody's hands, or under a page the Bot is waiting for somebody to work.
130+
*
131+
* And never a person's own browser, cloud or native. Their Recorder runs in it, and closing it ends
132+
* what they were recording without a word to them. It still counts toward the cap, which closes it
133+
* last; memory pressure is not a reason to end somebody's session.
134+
*/
135+
export function chooseForMemory(
136+
running: Iterable<[string, Evictable]>,
137+
now: number,
138+
{
139+
recentlyUsedMs = RECENTLY_USED_MS,
140+
spared = () => false,
141+
}: {
142+
recentlyUsedMs?: number;
143+
spared?: (botId: string) => boolean;
144+
} = {},
145+
): string | undefined {
146+
const entries = [...running];
147+
const cutoff = now - recentlyUsedMs;
148+
return inClosingOrder(entries, newestOf(entries), () => false)
149+
.filter(
150+
([botId, entry]) =>
151+
entry.usedAt <= cutoff && !isPersonBrowserId(botId) && !spared(botId),
77152
)
78-
.slice(0, entries.length - max)
79-
.map(([botId]) => botId);
153+
.map(([botId]) => botId)
154+
.at(0);
80155
}
81156

82157
/**
@@ -104,3 +179,40 @@ export function chooseIdle(
104179
.filter(([botId, entry]) => entry.usedAt <= cutoff && !stays(botId))
105180
.map(([botId]) => botId);
106181
}
182+
183+
/** Enough of a tab for the choice. Keeps this file independent of what else is on a page. */
184+
export type IdleTab = {
185+
/** When the Bot or a person last acted on it, it navigated, or it was the page in front. */
186+
usedAt: number;
187+
/** The page the Bot is on, which is the one the live screen shows and its next action lands in. */
188+
front: boolean;
189+
/** A page that is never the Bot's to lose: an extension's own (the Recorder's) or DevTools. */
190+
kept?: boolean;
191+
};
192+
193+
/**
194+
* Which of a Bot's background tabs to close because nothing has touched them.
195+
*
196+
* A Bot is always on its newest page (`chooseLivePage`): a window a site opens is where its work
197+
* went, and it has no way back to an older tab except that newer ones close. So a tab left behind
198+
* (the page a sign-in popup was opened from, a link that opened in a new tab and was never closed)
199+
* holds a renderer and its memory for nothing, for as long as the browser stays up. Closed past a
200+
* timeout rather than at once, because an opener whose popup is still mid sign-in is waiting for
201+
* that popup to come back to it.
202+
*
203+
* Never the page in front, which is the one the Bot is on, so closing tabs never moves it: it stays
204+
* the newest open page, which is what the Bot follows. And never anything when which page is in
205+
* front is not exactly one, because then any tab could be the one the Bot is on. That also means the
206+
* last tab is never closed, since a single tab is the one in front. A timeout of zero or less
207+
* switches this off.
208+
*/
209+
export function chooseIdleTabs<T extends IdleTab>(
210+
tabs: readonly T[],
211+
idleTimeoutMs: number,
212+
now: number,
213+
): T[] {
214+
if (idleTimeoutMs <= 0) return [];
215+
if (tabs.filter((tab) => tab.front).length !== 1) return [];
216+
const cutoff = now - idleTimeoutMs;
217+
return tabs.filter((tab) => !tab.front && !tab.kept && tab.usedAt <= cutoff);
218+
}

0 commit comments

Comments
 (0)