-
-
Notifications
You must be signed in to change notification settings - Fork 4.3k
Retain asset without data for RENDER_WORLD-only assets #21732
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
I wonder if it would be better having an enum like this just to have some more information enum RenderAssetData {
Available(Vec<u8>),
Unavailable,
SentToRenderWorld
} |
|
maybe ... i tried to avoid changing the api of the main-world types (particularly |
crates/bevy_mesh/src/mesh.rs
Outdated
| #[inline] | ||
| pub fn indices(&self) -> Option<&Indices> { | ||
| self.indices.as_ref() | ||
| self.indices.as_ref().expect("Mesh has been extracted to RenderWorld. To access vertex attributes, the mesh must have RenderAssetUsages::MAIN_WORLD").as_ref() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could we swap these methods to return Result and avoid the expects here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We could … it is a usage error though, rather than “something that can happen and you should deal with”, like result usually implies.
I don’t feel strongly, will change if you prefer it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think the intent is for a user to "deal with it", but rather "let's not crash the program because one asset has the wrong flags set". So I am in favour of making these Results.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
making these return results will add literally hundreds of unwraps to the code base, e.g. there are 104 from primitive mesh builder functions calling with_inserted_attribute which are all guaranteed to succeed since they operate on meshes they've just constructed themselves.
perhaps i can add try_xxx variants, so that if we find situations where there are panics that we don't want we have a way to manage them in future.
crates/bevy_mesh/src/mesh.rs
Outdated
| #[inline] | ||
| pub fn indices(&self) -> Option<&Indices> { | ||
| self.indices.as_ref() | ||
| self.indices.as_ref().expect("Mesh has been extracted to RenderWorld. To access vertex attributes, the mesh must have RenderAssetUsages::MAIN_WORLD").as_ref() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think the intent is for a user to "deal with it", but rather "let's not crash the program because one asset has the wrong flags set". So I am in favour of making these Results.
crates/bevy_mesh/src/mesh.rs
Outdated
| } | ||
| } | ||
|
|
||
| pub fn take_gpu_data(&mut self) -> Option<Self> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Doc comment please! Also linking to RenderAsset::take_gpu_data would be good.
| fn take_gpu_data( | ||
| source: &mut Self::SourceAsset, | ||
| _previous_gpu_asset: Option<&Self>, | ||
| ) -> Option<Self::SourceAsset> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we even provide a default impl here? Cloning generally seems like the wrong behaviour to me, since I'd expect pretty much every impl to want to transfer the data rather than doing a possibly expensive clone.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hm. i'm not sure the defaul impl gets called anywhere currently (i think the remaining RenderAsset types don't allow the user to set the usages), but then, what should we do here? panic in the trait impl or implement it explicitly for all the RenderAsset types with a panic or a clone, and require users to implement it explicitly as well even if they won't use it?
that doesn't seem helpful to me.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i changed it so the default returns AssetExtractionError::NoImplementation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I much prefer making users implement this explicitly - pretty much every render asset should want to handle this correctly I think, especially since the default is just breaking.
|
|
||
| /// Retrieves the vertex `indices` of the mesh mutably. | ||
| #[inline] | ||
| pub fn indices_mut(&mut self) -> Option<&mut Indices> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All these mutation methods essentially stop functioning when the asset is extracted. What about runtime generated (and mutated) assets? Is the intent that users will like fully replace the Mesh with like *mesh_mut = Mesh::new()?
If so, this probably needs a migration guide.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the pattern would be either to fully initialize in one step, or to initialize with RenderAssetUsages::MAIN_WORLD, make any modifications, then when it's finished change the usage to RenderAssetUsages::RENDER_WORLD. after that no more modifications are possible since the data is only on GPU and we can't get it back.
if users want to make continuous runtime modifications, then they should use MAIN_WORLD || RENDER_WORLD, or fully replacing will also work.
this PR doesn't change anything in those scenarios, modifying an asset with RENDER_WORLD and not MAIN_WORLD is not possible currently either since the asset gets removed from the Assets collection. i can write something if you like, but the usage doesn't change.
| ) -> Option<Self::SourceAsset> { | ||
| let data = source.data.take(); | ||
|
|
||
| let valid_upload = data.is_some() || previous_gpu_asset.is_none_or(|prev| !prev.had_data); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why can't we just reupload here? We're losing a capability here I think, since previously hot-reloading an image would just send the data to the GPU, which would unconditionally upload. Am I missing something here?
Could you also test that hot reloading still works? I believe the alter_sprite example is probably sufficient with the file_watcher feature (and then messing with the loaded sprite).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so the purpose of RenderAssetUsages::RENDER_WORLD is to reduce CPU memory by not keeping data on the CPU that is only required on the GPU. we clearly can't re-upload data from the CPU that we have chosen to purge.
as i described in the top comment, this check is specifically to catch an error which was not previously possible: where a user modifies some metadata on a RENDER_WORLD-only image after it has been transferred. that's not possible currently because currently the image gets removed from the CPU-side Assets collection so cannot be accidentally modified.
keeping access to it CPU side has the benefits i listed right at the top, but introduces this new way to break things, so i added this check to help anyone who does so accidentally. without this check, the image would be treated as an uninitialized texture -- for images with data: None we create an uninitialized GPU-texture which is much faster -- and the GPU pixel data would be zeros, which will never be what the user wants if the image had data when first transferred.
we're not losing any capabilities. default is MAIN_WORLD | RENDER_WORLD, in which case you can modify and (re)upload as much as you like because we keep the pixel data CPU side.
hot reloading still works for MAIN_WORLD | RENDER_WORLD and for RENDER_WORLD-only as reloading an asset reloads the data too.
i still need to do a pass through to fix up docs for all the copy/pasted functions, but the logic should be complete now i think. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code seems reasonable - just got a couple of requests to avoid some corner cases. I also tested a variety of examples, and hacked in a few extra tests (including glTFs with RENDER_WORLD only, and making some examples deliberately trigger errors).
I don't like how the Mesh interface has ended up, but I'm not sure that's a problem with this PR. My hot take is that it's a problem with Mesh trying to be three things at once:
- A finished, immutable asset that can be efficiently shared and rendered.
- A way to incrementally build a mesh by adding/modifying attributes and suchlike.
- A way to animate meshes by mutating an asset in-place and re-extracting.
I think the solution could be:
- Add a new
MeshBuilderstruct and move all theMeshmutation methods toMeshBuilder.- This struct wouldn't have to deal with
MeshAccesserrors since it's entirely CPU side, so a load oftry_xxxmethods disappear. - A
finishfunction would consume the builder and return aMesh.
- This struct wouldn't have to deal with
- Change
Meshto be immutable, except bytake_gpu_data.- Animation that wants to mutate the mesh now has to create a fresh
Meshand overwrite the entire asset.
- Animation that wants to mutate the mesh now has to create a fresh
That gives as clear separation between 1) and 2), but makes 3) more awkward and inefficient. I'd argue that's ok in the short-term since it still basically works and mutating the asset is always going to be somewhat inefficient. In the long-term there could be a separate way of efficiently sending fine-grained mesh updates to the GPU.
The Mesh/MeshBuilder separation might also make compressed and derived data (like AABBs) more robust and efficient. So MeshBuilder::finish would automatically calculate AABBs and optionally apply compression.
EDIT: I should have said that I don't think this PR needs to explore stuff like MeshBuilder. What the PR does right now - make the Mesh interface more awkward in the short-term - seems like a reasonable trade-off for solving RENDER_WORLD issues now.
crates/bevy_mesh/src/mesh.rs
Outdated
| }) = attributes.as_ref()?.get(&Self::ATTRIBUTE_POSITION.id) | ||
| { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| }) = attributes.as_ref()?.get(&Self::ATTRIBUTE_POSITION.id) | |
| { | |
| }) = attributes.as_ref()?.get(&Self::ATTRIBUTE_POSITION.id) | |
| && !position_values.is_empty() | |
| { |
Not 100% sure it's actually possible to make and extract a mesh with an empty position attribute, but seems reasonable to avoid a panic here just in case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i'll change this to not bubble the error here, just fail to calc the aabb. then the only "?" error will be AlreadyExtracted. that should solve this and the below issue.
it's legitimate to extract a mesh with no positions, i guess, if you are doing some custom processing that doesn't use positions? maybe?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sorry i misunderstood - i see now why this is required and added it
| fn take_gpu_data( | ||
| source: &mut Self::SourceAsset, | ||
| _previous_gpu_asset: Option<&Self>, | ||
| ) -> Result<Self::SourceAsset, AssetExtractionError> { | ||
| source | ||
| .take_gpu_data() | ||
| .map_err(|_| AssetExtractionError::AlreadyExtracted) | ||
| } | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If Mesh::take_gpu_data returns MeshAccessError::NotFound (e.g. because the mesh has no attributes), then this logic will wrongly report AlreadyExtracted. I repro'd this by changing the generate_custom_mesh example to make a mesh with no attributes.
I'm not sure what the solution is - maybe AssetExtractionError should also have a discriminant for missing data.
|
it would be nice to have a MeshBuilder with infallible access methods. then all the Mesh accessors could be result-based without scattering 100s of unwraps about. i don't think i want to do it as part of this PR though. |
|
Ah, my bad, I should have been clearer that I think this PR is fine as it is. It would have been simpler if |
…hat bevyengine#21732 or similar will land, so we don't lose the bounds if the mesh doesn't have `RenderAssetUsages::MAIN_WORLD`.
17f22ad to
b11ae9d
Compare
crates/bevy_mesh/src/mesh.rs
Outdated
| pub enable_raytracing: bool, | ||
| /// Precomputed min and max extents of the mesh position data. Used mainly for constructing `Aabb`s for frustum culling. | ||
| /// This data will be set if/when a mesh is extracted to the GPU | ||
| pub final_aabb_extents: Option<(Vec3, Vec3)>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this should be a Option<Aabb3d> which is more clearer than a tuple. And could it be private?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
changed the type, thanks.
it's used in bevy_camera so can't be private unless we add an accessor ... that might be better since it would stop users setting it wrongly, but would prevent users who have a legitimate reason to modify it ... not sure if there is such a reason.
greeble-dev
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Noticed one more issue while testing meshes with missing attributes.
Co-authored-by: Greeble <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm clicking approve because the PR solves an immediate problem that I've seen a few people trip over. It will also unblock #21837. The Mesh interface ends up a bit awkward right now, but there's room to improve it in future (see #21986). I think the PR still needs a migration guide though?
EDIT: Minor correction - maybe this PR can skip the migration guide, assuming #21986 lands in the same release and has a migration that covers both? Would keep things simpler.
I totally agree with @greeble-dev. So copying vertex and indices to MeshAllocator is fast and no longer needs repacking data. This opens a door to direct constructing Mesh and mutating Mesh in-place if you have the raw vertex and indices data and info and want to avoid repacking. I think the current approach of extracting vertex attributes is suboptimal. And Edit: Oh, think again, I can see the reason to store attributes on the Mesh since |
|
Regarding the Mesh interface, I‘d prefer to remove these |
since the change to results was at request of @andriyDev (and @alice-i-cecile) i'll defer to them. i agree fwiw |
| attributes: BTreeMap<MeshVertexAttributeId, MeshAttributeData>, | ||
| indices: Option<Indices>, | ||
| attributes: MeshExtractableData<BTreeMap<MeshVertexAttributeId, MeshAttributeData>>, | ||
| indices: MeshExtractableData<Indices>, | ||
| #[cfg(feature = "morph")] | ||
| morph_targets: Option<Handle<Image>>, | ||
| morph_targets: MeshExtractableData<Handle<Image>>, | ||
| #[cfg(feature = "morph")] | ||
| morph_target_names: Option<Vec<String>>, | ||
| morph_target_names: MeshExtractableData<Vec<String>>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These extractable data can be consolidated. This ensures you always extract them together and that they are either all with data or without data simultaneously. Like this:
struct MeshExtractableData {
attributes: BTreeMap<MeshVertexAttributeId, MeshAttributeData>,
indices: Option<Indices>,
#[cfg(feature = "morph")]
morph_targets: Option<Handle<Image>>,
#[cfg(feature = "morph")]
morph_target_names: Option<Vec<String>>,
}
struct Mesh {
...
extractable_data: Option<MeshExtractableData>,
...
}There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
could be. i don't think it improves anything but i can change it if others disagree.
IceSentry
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The new _option function feel a bit weird but I don't really have an alternative solution. I mean it could either always return the option or always map the option to a result by it's not a blocker.
LGTM
|
And yeah, I prefer keeping the |
| } | ||
|
|
||
| impl<T> MeshExtractableData<T> { | ||
| // get a reference to internal data. returns error if data has been extracted, or if no |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should make these all doc comments.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we typically use doc comments for private methods on private structs? i will change if so but i didn't think it was standard.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We typically do, as we have documentation built for private items on dev-docs.bevy.org.
| fn take_gpu_data( | ||
| source: &mut Self::SourceAsset, | ||
| _previous_gpu_asset: Option<&Self>, | ||
| ) -> Option<Self::SourceAsset> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I much prefer making users implement this explicitly - pretty much every render asset should want to handle this correctly I think, especially since the default is just breaking.
| if let Some(asset) = assets.remove(id) { | ||
| extracted_assets.push((id, asset)); | ||
| added.insert(id); | ||
| if let Some(asset) = assets.get_mut_untracked(id) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm a little worried that we're silently mutating the asset. Is this cause for concern?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would argue it's kinda part of the contract of using the render world asset usage but I can see why some people wouldn't like it. I'm not sure what the alternative is unless we can communicate more details about why an asset changed. If a user could react to this change right now there would be no way to know if it was a user event or the render world extraction. I guess if they look at the asset and see missing data they would know that's what hapenned.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
i used get_mut initially, but it ended up in a broken feedback loop since re-extraction is triggered by asset modifications (which is required behaviour for intentional user modifications), including this one.
the alternative would be to use a get_mut variant that fires a different event i guess. but the behaviour of extraction is predictable enough -- the data will be removed in the extract stage -- that i don't think it's warranted.
there are several |
My concern is that if we add more extractable resources (like RawMesh, etc.), we will also follow a similar pattern of providing |
you've seen the |
|
Another possible approach I consider is to consolidate the extractable data and access it through something similar to entry API, thereby avoiding multiple unwrap operations. |
That is unrelated. I mean if more extractable assets are added in the future, or if crate authors want to add their extractable assets, should the interfaces also follow the |
Objective
when
RenderAssetswithRenderAssetUsages::RENDER_WORLDand withoutRenderAssetUsages::MAIN_WORLDare extracted, the asset is removed from the assets collection. this causes some issues:AssetServer::get_handleSolution
extraction:
take_gpu_datato theRenderAssettrait. use it to pull the data out of the asset for transfer, and leave the empty asset in the collection. default implementation justclones the asset.panic. this follows from modifying an asset after extraction, which is always a code error, so i think panic here makes senselog an errorMesh/RenderMesh:
Mesh::attributesandMesh::indicesoptionsexpectoperations which access or modify the vertex data or indices if it has been extracted. accessing the vertex data after extraction is always a code error. fixes Somehow prevent confusion caused by Assets being removed due to not having RenderAssetUsages::MAIN_WORLD #19737 by resulting in the errorMesh has been extracted to RenderWorld. To access vertex attributes, the mesh must have RenderAssetUsages::MAIN_WORLDtry_xxxoperations which allow users to handle the access error gracefully if required (no usages as part of this pr, but provided for future)Aabbwhen gpu data is taken and store the result. this allows extracted meshes to still use frustum culling (otherwise using multiple copies of an extracted mesh now panics ascompute_aabbrelied on vertex positions). there's a bit of a tradeoff here: users may not need the Aabb and we needlessly compute it. but i think users almost always do want them, and computing once (for extracted meshes) is cheaper than the alternative, keeping position data and computing a freshAabbevery time the mesh is used on a new entity.Image/GpuImage:
images are a little more complex because the data can be deliberately
Nonefor render-targets / GPU-written textures where we only want an uninitialized gpu-side texture.Image::dataon extractionGpuImagewhether any data was found initiallycorner case / issue: when used with
RenderAssetBytesPerFrameLimiterthere may be no previous gpu asset if it is still queued pending upload due to the bandwidth limit. this can result in a modified image with initial data skipping thehad_datacheck, resulting in a blank texture. i think this is sufficiently rare that it's not a real problem, users would still hit the panic if the asset is transferred in time and the problem/solution should be clear when they do hit it.ShaderStorageBuffer/GpuShaderStorageBuffer
follows the same pattern as Image/GpuImage:
ShaderStorageBuffer::dataon extractionGpuShaderStorageBufferwhether any data was found initiallywe don't have the queue issue here because
GpuShaderStorageBufferdoesn't implementbyte_lenso we can't end up queueing them.other RenderAssets
i didn't modify the other
RenderAssettypes (GpuAutoExposureCompensationCurve,GpuLineGizmo,RenderWireframeMaterial,PreparedMaterial,PreparedMaterial2d,PreparedUiMaterial) on the assumption thatcloning these is cheap enough anywaythe asset usages are not exposed so we should never calltake_gpu_data. the default implementation panics with a message directing users to implement the method if requiredTesting
only really tested within my work project. i can add some explicit tests if required.