refactor: atomic installation by lima-limon-inc · Pull Request #201 · 0xMiden/midenup

lima-limon-inc · 2026-05-07T23:39:54Z

Closes #198
Closes #195
Blocked-by: #209

This PR changes the way installs are performed in midenup. After this PR, installs in midenup are made atomically, meaning that the entire toolchain is either fully installed onto the filesystem or not.

Installs are performed on a separate directory that's stored inside the installed_toolchains/<channel-version>-<hash>. This directory is then atomically symlinked into the corresponding toolchains/<channel-version> directory. If a toolchain gets re-installed with newer components (such as in the case=
This structure roughly looks like:

midenup
├── installed_toolchains
│   └── 0.14.0-fb9b2252a909ddb9
│       ├── bin
│       │   └── (...)
│       ├── install.rs
│       ├── lib
│       │   └── (...)
│       ├── install.rs
│       └── opt
│           └── (...)
└── toolchains
    ├── 0.14.0 -> ../installed_toolchains/0.14.0-fb9b2252a909ddb9
    └── stable -> 0.14.0

Notes:

A couple of trait implementations were removed in favor of the default derived ones.
- Hash, PartialEq and Eq for Component. These custom definition were only used in update.rs when performing set operations. Removed in 5597a1e.
- Now, in order to compare Components by names, the wrapper struct ComponentByName can be used.
All components on the Channel now derive Hash, which is then used to generate the directory name in installed_toolchains.
The installation-successful and .installation-in-progress install-logs have been removed. Since toolchains are atomically installed, if the toolchain is symlinked, then it was definitely installed.
The channel.json snapshot is no longer saved during installs. Now, uninstall relies on the information saved on the local manifest to uninstall the component. This got changed on 2c39912
- This change is what fixes bug: midenup fails to uninstall components that were installed from pre-compiled artifacts #195.
There's one limitation with this approach: if a channel requires a name change when updating, the install is no longer "atomic".
- This particularly affects channel migrations. However, as pointed out in the code, were a migration to be stopped right before installing the previous channel, re-issuing a midenup update would finish the process accordingly.

…q impls

…ally used

lima-limon-inc · 2026-05-13T22:39:46Z

@bitwalker like mentioned on #194, this PR is also ready for review. The main set of changes come from install.rs + channel.rs.

Thanks in advance ^_^

bitwalker

Looks good overall! I found a couple of issues that we should address, but this is pretty close

bitwalker · 2026-05-20T15:46:40Z

+    // We now rename tmp_link to toolchain_dir. When renamed, it will still be
+    // pointing to relative_install_target. If the channel direcotry existed, it
+    // will overwrite the file. This is what marks the install as completed.
+    std::fs::rename(&temp_symlink, &toolchain_dir).with_context(|| {


It isn't clear to me whether there will be an issue here with existing installations. Those may already have real directories at toolchains<version>, so this will fail if toolchain_dir already exists.

Great point, addressed it here: https://github.com/0xmiden/midenup/compare/35b49ac..970b91d

I added a migration check and associated function. Were more migrations to be needed in the future, we could just add more checks and associated fuctions on the corresponding mod.rs file

bitwalker · 2026-05-20T15:47:53Z


-    local_manifest.remove_channel(channel.name.clone());
+    // We remove the symlink, thus making the channel unaccesible.
+    if toolchain_symlink.exists() {


Similar to my comment in install.rs, if there is an existing install, then remove_file here may be getting called on a real directory, not a symlink. Existing installs will need a one-time migration or compatibility path.

bitwalker · 2026-05-20T15:50:34Z

-        .unwrap_or(false);
+    let installed_toolchains_dir = config.midenup_home.join("installed_toolchains");
+    let install_dir_name = format!("{}-{}", &channel.name, channel.content_hash());
+    let install_dir = installed_toolchains_dir.join(&install_dir_name);


It appears updates that have the same hash can be marked complete without reinstalling. Same-hash updates can be marked complete without reinstalling. install_dir is based on channel.content_hash() here, and changed branch/path components can reuse the same upstream channel hash. If the hashed install directory already exists, component removal is skipped entirely (see line 44), the generated install script skips existing binaries, and the manifest is refreshed as if the update happened. This affects branch-tip and local-path updates.

Great catch, I've since adressed this issue here: https://github.com/0xmiden/midenup/compare/2402558..725ac8f

Now, when performing an update, the channels might mutate (for instance, channels that are tracking branches will get the fetched latest commit added onto their latest_commit field. Same with Path tracked components.

Update: This is no longer the case, the is_up_to_date function no longer changes the component. Now, when the UpstreamChannel is created, we sync the channel to get the latest changes.

bitwalker · 2026-05-20T15:51:56Z

+                Authority::Cargo { .. } | Authority::Git { .. } => true,
+                // Since uninstalling a component from the filesystem is potentially
+                // irreversible, we take special precautions before uninstalling them.
+                Authority::Path { .. } => match options.path_update {


--path-update=off no longer preserves skipped components. Update excludes path components here, but installation still saves the upstream channel and refreshes path metadata for all path components (see line 223 of install.rs), so a skipped rebuild can then look current.

Noted. This has seen been fixed here: f29ac1e

This same issue also caused a bug where partial channels would install all the remaining components.

bitwalker · 2026-05-20T15:54:24Z

+    // We now rename tmp_link to toolchain_dir. When renamed, it will still be
+    // pointing to relative_install_target. If the channel direcotry existed, it
+    // will overwrite the file. This is what marks the install as completed.
+    std::fs::rename(&temp_symlink, &toolchain_dir).with_context(|| {


It looks like installs can be published before metadata is committed. The runnable toolchains/<version> symlink is published before stable and manifest.json are written here and on line 176. A failure in between will leave the filesystem and local manifest state inconsistent.

This has since been fixed here: d58c9dc

Updating the manifest should be the last operation we do IIUC

bitwalker · 2026-05-20T15:55:54Z


-    local_manifest.remove_channel(channel.name.clone());
+    // We remove the symlink, thus making the channel unaccesible.
+    if toolchain_symlink.exists() {


It appears uninstallation is not retry-safe after cleanup failure. This removes the symlink and manifest entry before removing components/install directory. If cleanup fails, rerunning uninstall bails because the manifest entry is gone.

Good point, changed here: f1a3b5d

Now the symlink is removed after removing the components.

…on' into fabrizioorsi/atomic-installation

This reverts commit 2bfa932.

… files

lima-limon-inc · 2026-06-12T20:26:03Z

The latest commits updated the way updates were computed. The main change is the following:
https://github.com/0xmiden/midenup/compare/fd96e357560fc4a3f33b94db1de4df69c2dee28b..f1a3b5dff88878b4a7de6a6c298538515eca049f#diff-4bf1ef1a8771c506c71d2ef69d5d59e623f54b2cc09f2c297496e30d5f7da25eR487-R508

The old channel gets cloned as "base", in order to inherit tags
The components are taken from the components_to_install variable, which includes both the components that need updating and, notably, the components that are up to date
If the channel got migrated, we apply the changes (i.e. the name change), on the migrate_channel function

The entire change set can be seen on this diff: https://github.com/0xmiden/midenup/compare/fd96e357560fc4a3f33b94db1de4df69c2dee28b..aab07d5c0c32eb76c113d61f39960f943dacc840

lima-limon-inc added the check:install PRs only: runs workflows that perform additional end-to-end integration testing label May 8, 2026

lima-limon-inc force-pushed the fabrizioorsi/atomic-installation branch from 5cdcf71 to 66cd69c Compare May 11, 2026 20:41

lima-limon-inc changed the title ~~(WIP): atomic installation~~ refactor: atomic installation May 11, 2026

lima-limon-inc force-pushed the fabrizioorsi/atomic-installation branch 3 times, most recently from d4a962e to f9967ef Compare May 13, 2026 20:32

lima-limon-inc marked this pull request as ready for review May 13, 2026 22:23

lima-limon-inc added 15 commits May 13, 2026 19:27

refactor: add ComponentByName struct, avoiding Component's Hash and E…

77a1b18

…q impls

feat: add Channel hashing

29b88f5

feat: atomic install with symlink + rename

ec08d63

enhacement: re-use components

7ef5f5e

refactor: remove installation log logic

565c4f4

refactor: only store artifacts in the LocalManifest if they were actu…

c7270ee

…ally used

feat: remove install-log logic from uninstall

a10631c

feat: update midenup update

55c6e3c

feat: update midenup set

75f0ed9

feat: update override command + Toolchain::current

7696485

fix: integration tests

fe120a5

fix: update channel

c446f4e

fix: remove migrated channel from directory

aace779

refactor: component filter

4e08b8c

refactor: remove is_entirely_removed check

d1f6aad

lima-limon-inc changed the base branch from main to fabrizioorsi/i193-fix-migration May 13, 2026 22:32

lima-limon-inc force-pushed the fabrizioorsi/i193-fix-migration branch from d1f6aad to 1d53ddf Compare May 13, 2026 22:36

lima-limon-inc force-pushed the fabrizioorsi/atomic-installation branch from f9967ef to d1f6aad Compare May 13, 2026 22:37

lima-limon-inc mentioned this pull request May 13, 2026

fix: migration bug #194

Merged

lima-limon-inc requested a review from bitwalker May 13, 2026 22:38

Base automatically changed from fabrizioorsi/i193-fix-migration to main May 20, 2026 14:15

bitwalker requested changes May 20, 2026

View reviewed changes

lima-limon-inc added 6 commits June 5, 2026 16:36

refactor: is_up_to_date now returns UpdateStatus

2bfa932

bug: don't install missing components from partially installed channels

32147f8

fix: remove installed channel after removing it from the manifest

b0498ec

QoL: display the serde error when parsing URI

a6ad61f

fix: update manifest test data

e2d14f5

refactor: split manifest parsing into stages

d288af2

lima-limon-inc force-pushed the fabrizioorsi/atomic-installation branch from a826083 to d288af2 Compare June 5, 2026 19:37

fix: move stable alias before writing to the manifest

d58c9dc

lima-limon-inc force-pushed the fabrizioorsi/atomic-installation branch from fa817e0 to d58c9dc Compare June 5, 2026 20:49

lima-limon-inc added 5 commits June 5, 2026 18:53

Merge branch 'main' into fabrizioorsi/atomic-installation

f97632f

fix: update installed_executable field on tests json

d3ad849

fix: copy new element when updating

1ab72eb

Merge remote-tracking branch 'upstream/fabrizioorsi/atomic-installati…

d84b1be

…on' into fabrizioorsi/atomic-installation

Revert "refactor: is_up_to_date now returns UpdateStatus"

88acfbf

This reverts commit 2bfa932.

lima-limon-inc force-pushed the fabrizioorsi/atomic-installation branch from 03ca629 to 452b5ee Compare June 9, 2026 15:41

feat: return new component when updating

fd96e35

lima-limon-inc force-pushed the fabrizioorsi/atomic-installation branch 2 times, most recently from 0e97928 to 100f6a4 Compare June 9, 2026 21:27

lima-limon-inc added 3 commits June 9, 2026 18:55

feat: add sync method and make UpstreamChannel

4a4240c

refactor: up to date no longer mutates component

b7fce3e

refactor: update function

6c1f491

lima-limon-inc force-pushed the fabrizioorsi/atomic-installation branch from 100f6a4 to 6c1f491 Compare June 9, 2026 21:56

lima-limon-inc added 3 commits June 10, 2026 16:34

refactor: components_to_update -> compute_update

5a7d108

refactor: update_channel refactor with new compute_update function

3ff2a90

doc: expand update documentation and explain the logic behind copying…

8cc8e8f

… files

lima-limon-inc force-pushed the fabrizioorsi/atomic-installation branch from c51597a to 8cc8e8f Compare June 11, 2026 19:01

lima-limon-inc added 2 commits June 12, 2026 16:04

nit: remove symlink after uninstalling components

f1a3b5d

feat: restore migration warning

aab07d5

lima-limon-inc requested a review from bitwalker June 12, 2026 20:30

Conversation

lima-limon-inc commented May 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lima-limon-inc commented May 13, 2026

Uh oh!

bitwalker left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lima-limon-inc commented Jun 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

lima-limon-inc commented May 7, 2026 •

edited

Loading