[Bug]: container exec/stop hang indefinitely on SSH reconnect, requiring full macOS restart to recover

[wooii]

I have done the following

• I have searched the existing issues
• If possible, I've reproduced the issue using the 'main' branch of this project

Steps to reproduce

1. Start a container with a bind mount: container run -d --name test0 -v :/code ubuntu:latest
2. Install openssh-server inside it and configure key-based SSH access.
3. Connect via SSH (e.g. VS Code Remote-SSH, or plain ssh). This first connection works fine.
4. Disconnect (e.g. restart VS Code, or point Remote-SSH at a different folder within the same container).
5. Attempt to reconnect via SSH, or run container exec -it test0 bash directly.
6. The exec/SSH session hangs indefinitely — no prompt, no error, no timeout.
7. container stop test0 also hangs with no output.
8. container list --all continues to report the container as "running" throughout.

In a second occurrence the same session, container system stop also hung (log showed stopTimeoutSeconds=5 ... stopping containers then nothing further — the stated timeout was not enforced), and launchctl kickstart -k <container-apiserver-label> did not recover it either. A full macOS restart was the only thing that restored a working state.

Separately, ps aux showed multiple stuck container exec client processes accumulated from repeated attempts (none had exited on their own), while container-runtime-linux and container-apiserver processes appeared healthy — suggesting the hang is in the exec/PTY session layer, not the VM or daemon itself. ping to the affected container's IP succeeded instantly while exec was hung, ruling out a network-layer cause.

Problem description

container exec (and subsequently container stop / container system stop) can hang indefinitely, specifically triggered by reconnecting to a container via SSH/exec — the first connection after starting/configuring the container works fine, but a second or later connection attempt reliably triggers the hang. This suggests something isn't being cleaned up when an exec/SSH session ends, and the leftover state blocks the next session.

Separately, container list --all reported a container as "running" while container exec on that same container immediately failed with cannot exec: container is not running — a state-tracking inconsistency between the two commands.

Expected behavior: exec/SSH sessions should be able to reconnect repeatedly without hanging, and container stop should always respect its stated timeout rather than hanging indefinitely and requiring a full OS restart to recover.

Environment

* OS: macOS 26.5.2 (Tahoe)
* Hardware: Apple Silicon (M4)
* Container: Container CLI version 1.1.0 (installed via Homebrew)
* Image: docker.io/library/ubuntu:latest (Ubuntu 26.04 LTS, arm64)

Code of Conduct

• I agree to follow this project's Code of Conduct

[lakshitsoni26]

Opened PR #1926 with a targeted fix for this — PTY cleanup on XPC disconnect + stop timeout enforcement.

[wooii]

@lakshitsoni26 Hi, I found that the original hang in #1916 is no longer reproducible on my machine (likely due to an update of VS Code to 1.128.0). So I believe #1916 can be closed.

However, I did discover a separate container cp hang problem, which I've filed as issue #1927. I tested your PR #1922 (commit 7645ca7) against the container cp hang, but it does not fix it — so it's likely a separate issue from the one your PR addresses.

[lakshitsoni26]

@wooii Thanks for testing! Even though the VS Code update stopped triggering this specific symptom on your end by closing connections more cleanly, the underlying daemon bug was still very real. If a user's terminal crashed or an SSH connection dropped abruptly, the orphaned PTY process would still hang the daemon infinitely, and container stop would fail to enforce its timeout. I'll leave this PR open for the maintainers to review since it adds necessary resource cleanup and timeout enforcement to the daemon.

[wooii]

@lakshitsoni26 Thanks for the clarification! Good to know the underlying daemon bug is real even if my specific trigger stopped firing. Appreciate you digging into it.

[lakshitsoni26]

Absolutely! Actually, the reason I started digging into this so quickly is because I've been running into a very closely related hang issue myself.

Since you mentioned you just found a new container cp hang in #1927, would you mind testing this PR (#1926) against it when you have a moment? The cp command also streams data through the daemon, so I strongly suspect your cp hang might be getting caught in the exact same underlying deadlock that this PR fixes. Let me know if it helps!

[wooii]

Hey @lakshitsoni26, update: a full uninstall of container 1.1.0 followed by a clean install of PR #1926 resolves issue #1916 for me.

My earlier "fixed by VS Code update" theory was wrong. Earlier I built with make cli and compared container vs ./bin/container side by side — the PR's effect was inconsistent then. After removing that build and restarting macOS, the hang returned. With a clean uninstall of 1.1.0 + fresh install of PR #1926, it's gone — VS Code connects reliably and stays responsive. So this PR does fix issue #1916 for me. Thank you!

I also tested PR #1926 against the container cp hang (issue #1927) as you suggested. No luck — still hangs the same way. So that one looks like a separate deadlock.

[wooii]

Correction: PR #1926 did not actually fix this — I've now traced the real trigger to the Pylance extension. With it enabled in a VS Code Remote-SSH session, the connection hangs and keeps retrying (container exec is not responsive either); disabling it makes the connection stable and responsive immediately.

This also explains my earlier false conclusions — both the VS Code update and the clean install of PR #1926 happened to remove Pylance from the container I was testing, not fix the underlying issue.

Hope a maintainer can take a look, or @lakshitsoni26 if you get a moment.

[lakshitsoni26]

Ah, that makes perfect sense! Pylance is a very heavy language server that spawns tons of file watchers and background I/O. It's highly likely that it's completely saturating the container's PTY buffer or SSH stream, freezing the session alive.

Because Pylance keeps the socket open (even though it's frozen), the daemon doesn't detect a "disconnect," so it just sits there waiting.

Quick question about your test with PR #1926 installed: When Pylance freezes the exec session, if you open a new local terminal and run container stop test0, does the stop command successfully force-kill the container after 5 seconds now? Or does the stop command still hang forever requiring a macOS reboot like it used to?

[wooii]

Hey @lakshitsoni26 ,

When Pylance is enabled, the container becomes unresponsive — the SSH session disconnects and keeps retrying. stop can still stop the container in this state.

The real trigger for the indefinite hang is a second exec -it test0 bash into an already-unresponsive session. On 1.1.0 that wedges the daemon forever, requiring kill -9 on container-apiserver (previously a full macOS restart). With PR #1926 installed, that same exec-on-frozen-session trigger no longer hangs — stop recovers in ~5.5s. So #1926 does fix the underlying daemon hang; my earlier "fixed by #1926" conclusion was actually right.

So to summarize: #1926 fixes stop/system stop for the second-exec-triggered indefinite hang. Pylance causing the container to become unresponsive in the first place is a separate problem that #1926 doesn't address.

Timing data:

Scenario	time container stop test0
1.1.0, Pylance disabled	5.46s, 5.34s
1.1.0, Pylance enabled, no exec	5.38s, 5.41s
1.1.0, Pylance enabled + second exec into frozen session	hangs forever (kill -9 on container-apiserver required)
PR #1926, Pylance enabled, no exec	5.38s, 5.37s
PR #1926, Pylance enabled + second exec into frozen session	5.66s, 5.45s

container system stop is slightly slower than stop in all cases, roughly 0.5–1s more.

[lakshitsoni26]

hey @wooii,
Thank you so much for the detailed testing and timing data! I'm really glad to hear that PR #1926 successfully fixes the daemon wedge and allows the system to recover gracefully when a second exec hits the frozen session.

Regarding Pylance causing the container to freeze in the first place: this is unfortunately a very common issue with Pylance in remote/containerized environments. Because it spawns so many file watchers and uses a lot of memory, it can completely exhaust the VM's resources and lock up the I/O.

To fix or mitigate the Pylance freezing, try these workarounds:

Increase Container Resources: When you start your container, try allocating more memory and CPU to it (e.g., if you are using a config file or CLI flags, make sure the VM has at least 4GB+ of RAM, as Pylance is very memory-hungry).
Exclude Folders from Watcher: In your VS Code settings.json, add large directories (like .venv, node_modules, or data folders) to files.watcherExclude. This stops Pylance from saturating the I/O stream.
Switch to a lighter Language Server: If it still freezes, you might want to try switching your VS Code Python language server to Jedi instead of Pylance, which is much lighter on I/O.
We can definitely look into optimizing how the daemon handles extreme I/O saturation in the future, but I'm thrilled that #1926 prevents the permanent daemon hang! I will let the maintainers know this PR is ready.