Mutable assets

[palapapa]

Would it be possible to add a feature that lets you mutate an asset? Bevy's asset system seems to support both mutating assets and hot-reloading. The documentation of Handle says that hot-reloading is implemented using RwLock, so would it be possible to implement mutable assets by exposing this lock to the user?

If someone needs mutable assets, here is a possible workaround (I haven't actually implemented this yet, so this may not be the exact pattern):

struct MyAsset {
    // Anything you want to put in this asset
}

impl Asset for MyAsset {    
    fn load(cache: &AssetCache, id: &SharedString) -> Result<Self, BoxedError> {
        /* Any logic to load your asset. If you need to read a file to
        construct `MyAssetInner`, read it using `cache.source().read()` so
        that the `AssetCache` registers that file as the dependency of this
        asset and calls this method automatically when that file changes. */
    }
}

struct MutableAsset<T>(RwLock<T>);

impl<A: Asset> Asset for MutableAsset<A> {
    const HOT_RELOADED: bool = A::HOT_RELOADED;

    fn load(cache: &AssetCache, id: &SharedString) -> Result<Self, BoxedError> {
        Ok(Self(RwLock::new(A::load(cache, id)?)))
    }
}

You can then use AssetCache::load to load MutableAsset<MyAsset>.

[a1phyr]

This would be possible indeed.

Adding a Handle::write method would be difficult for now, because the integration with hot-reloading would require significant refactoring to do properly.

I was thinking of adding AssetCache::insert_or_update, which would work like AssetCache::get_or_insert but would allow updating the value. Would it work for you?

Your workaround works, but there is a footgun here: if your Mutable is reloaded through hot-reloading, are your modifications are lost. This is why RwLock and Mutex don't implement Asset directly. You can mitigate this issue using a pattern like this:

impl Asset for MutableAsset {
    fn load(cache: &AssetCache, id: &SharedString) -> Result<Self, BoxedError> {
        if let Some(old_value) = assets_manager::no_record(|| cache.get::<Self>(id)) {
            // The asset is being reloaded
        } else {
            // This is the first time the asset is loaded
        }
    }
}

Anyway, what is your use case here? Knowing this would help me design a good API.

[palapapa]

In my project, the pattern actually looks like this:

/// A handle to an asset so that one can be shared by multiple entities.
/// `AssetCache` assets are loaded from a file on the disk and are stored in an
/// [`AssetCache`]. `Procedural` assets are those generated by code at runtime,
/// and are simply stored on the heap and referenced using [`Arc`]s.
/// 
/// In both cases, `T` is wrapped in [`MutableAsset`] to allow them to be
/// modified.
#[derive(From)]
pub enum AssetHandle<T> {
    /// A handle to a static asset that is stored in an [`AssetCache`].
    AssetCache(ArcHandle<MutableAsset<T>>),
    /// A handle to a dynamic asset that is stored on the heap.
    Procedural(Arc<MutableAsset<T>>)
}

/// A newtype pattern that wraps a [`RwLock`] so that [`Asset`] can be
/// implemented on it. This is used to allow [`Asset`]s to be modified.
pub struct MutableAsset<T>(pub RwLock<T>);

impl<A: Asset> Asset for MutableAsset<A> {
    const HOT_RELOADED: bool = A::HOT_RELOADED;

    fn load(cache: &AssetCache, id: &SharedString) -> Result<Self, BoxedError> {
        Ok(Self(RwLock::new(A::load(cache, id)?)))
    }
}

I should describe the thought process behind why I came up with it. I am making a very simple game "engine" from scratch. I use wgpu and bevy_ecs for its ECS implementation. A problem I encountered was that I sometimes needed to have assets generated from code at runtime. For example, I may generate a unit cube mesh to test if my mesh renderer component is working correctly. The mesh renderer component initially had an ArcHandle, but since the mesh might be loaded from a file or generated from code, I couldn't directly use it. I have thought about implementing Asset for the mesh struct in a special way, such that when the ID starts with dynamic., for example, I could encode information in the ID string, and return a dynamically generated mesh from Asset::load without reading any file. For example, dynamic.cube-1-1-1 could have generated a 1x1x1 cube mesh.

Doing so has a problem, though. Since the AssetCache itself always holds an Arc to the asset, any dynamically generated asset generated this way will never be freed, unless I somehow use a potentially complicated AssetCache hierarchy where a child AssetCache holds only dynamically generated assets, so I can drop that AssetCache to free the memory used by dynamic assets.

This is why I wrote the AssetHandle struct, so that a dynamic asset isn't stored in any AssetCache and can be freed when the last Arc to it is dropped. At this point, AssetHandle looked like this:

pub enum AssetHandle<T> {
    AssetCache(ArcHandle<T>),
    Procedural(Arc<T>)
}

Then, I had another issue. Just like assets loaded from an AssetCache can be modified after they have been loaded by modifying the file they are loaded from, I want to be able to modify dynamic assets, too. Because if I can't, then what if there are multiple components referencing the same dynamic asset, and I want to modify the asset? I would have to recreate an AssetHandle::Procedural, replace the old one in a component, and somehow notify all other components that are still holding the old Arc to reference this one instead. So I thought of wrapping it in an RwLock like Procedural(Arc<RwLock<T>>).

Doing so would create another issue. The whole point of AssetHandle is so that components don't have to care about whether an asset is loaded from a file or generated dynamically. If Procedural can be modified through code but AssetCache can't, that creates an asymmetry. If I want to write a method on AssetHandle that lets you write-lock it, that method would need to fail if the variant is not Procedural, and that would just make it so ugly. I would have to handle that error everywhere. So there needed to be a way to modify an AssetCache asset as well, hence this issue.

(I think I have typed too much context that I don't know if it will be useful to you.)

I am not sure how the load implementation would stop file changes from overwriting changes from code, though. But that fact doesn't really matter to me, anyway, because if a file change comes after a code change, then the file change should obviously overwrite the previous change, because I am treating these two kinds of change as equal. To "mitigate" this would mean that I would prevent the asset from ever being hot-reloaded again if it has been modified through code?

[a1phyr]

Woah this is quite the context!

If I understand, something like AssetCache::insert_or_update would probably work for you, except maybe that mutating the asset would require the AssetCache. If

Side note, you may over-complicate things with procedurally generated assets. You can create them directly then insert them with AssetCache::get_or_insert, so you don't need id hacks to pass data to Asset::load. Then you can use ArcHandle<MutableAsset<T>> everywhere.

I am not sure how the load implementation would stop file changes from overwriting changes from code, though.

It wouldn't, it would be just for hot-reloading.

To "mitigate" this would mean that I would prevent the asset from ever being hot-reloaded again if it has been modified through code?

With the code sample I gave, you can do whatever you want when Asset::load is called a second time: keeping the old value/error out, mix both modifications,...

But that fact doesn't really matter to me, anyway

That's great then!

[palapapa]

You can create them directly then insert them with AssetCache::get_or_insert

Yes, but wouldn't that create the same issue where it would never be freed, causing something similar to a memory leak? Would it be possible to make it so that the AssetCache doesn't hold a reference to the dynamically inserted asset, so that when the last ArcHandle I have to it is dropped, its memory is freed?

[a1phyr]

Yes, but wouldn't that create the same issue where it would never be freed, causing something similar to a memory leak?

It wouldn't be freed until the AssetCache is dropped, yes. But if you update it, old versions are dropped.

Or maybe you only need generated assets for a while, after which you only use loaded ones?

Would it be possible to make it so that the AssetCache doesn't hold a reference to the dynamically inserted asset, so that when the last ArcHandle I have to it is dropped, its memory is freed?

Theoretically, something like this could indeed be created. What I'm starting to fear is some kind of API size explosion, with a lot of combinations: settings/default settings * load/get/contains * store strong/weak ref * typed/untyped * ...