-
- Nir Hasson
- Posts: 71
- Joined:
- Contact:
Unity 2019 - Frequent texture updates
I'm using a 3rd party video solution that updates the video data frequently.
The video data is available as a raw bytes array, simple 4 bytes per pixel RGBA format.
On Unity 2018 we simply used the LoadRawTextureData to update a native unity Texture2D object and then used Noesis.TextureSource to wrap this texture and use it as part of our UI.
This solution doesn't work well on Unity 2019 since the LoadRawTextureData and any other API call that updates the pixel changes the internal API native pointer so the Texture.GetNativeTexturePtr returns a new resource.
See unity doc:
https://docs.unity3d.com/ScriptReferenc ... rePtr.html
Release the current used Noesis.TextureSource doesn't seem to release the reference as Noesis.Texture keeps a reference to the wrapped texture at the Texture static Dictionary.
What's the best way to have frequently updated textures available to the UI system via a subclass of Noesis.ImageSource?
The video data is available as a raw bytes array, simple 4 bytes per pixel RGBA format.
On Unity 2018 we simply used the LoadRawTextureData to update a native unity Texture2D object and then used Noesis.TextureSource to wrap this texture and use it as part of our UI.
This solution doesn't work well on Unity 2019 since the LoadRawTextureData and any other API call that updates the pixel changes the internal API native pointer so the Texture.GetNativeTexturePtr returns a new resource.
See unity doc:
https://docs.unity3d.com/ScriptReferenc ... rePtr.html
Release the current used Noesis.TextureSource doesn't seem to release the reference as Noesis.Texture keeps a reference to the wrapped texture at the Texture static Dictionary.
What's the best way to have frequently updated textures available to the UI system via a subclass of Noesis.ImageSource?
-
-
sfernandez
Site Admin
- Posts: 3364
- Joined:
Re: Unity 2019 - Frequent texture updates
Are you sure native pointer is changing? what platform shows that behavior?
I made a simple test of a texture being updated on every frame using LoadRawTextureData and logging the native pointer always returns the same address (tested on both Unity 2019 and 2020).
I made a simple test of a texture being updated on every frame using LoadRawTextureData and logging the native pointer always returns the same address (tested on both Unity 2019 and 2020).
We keep that reference while texture is being used by the render thread, but we release it as soon as it is no longer used.Release the current used Noesis.TextureSource doesn't seem to release the reference as Noesis.Texture keeps a reference to the wrapped texture at the Texture static Dictionary.
ImageSource cannot be subclassed. One of the goals of a TextureSource is to allow rendering dynamic textures, but it relies on the fact that texture native resource doesn't change, it won't be very efficient to create a new texture each time you need to change its contents.What's the best way to have frequently updated textures available to the UI system via a subclass of Noesis.ImageSource?
-
- Nir Hasson
- Posts: 71
- Joined:
- Contact:
Re: Unity 2019 - Frequent texture updates
It happens on iOS, Android and macOS.
Windows keep the same underlying pointer.
Using unity 2019.4.13f1 LTD
Windows keep the same underlying pointer.
Using unity 2019.4.13f1 LTD
-
-
sfernandez
Site Admin
- Posts: 3364
- Joined:
Re: Unity 2019 - Frequent texture updates
It is strange, I tested it on Android and I'm getting always the same native pointer every frame too.
Here is the sample I'm using:
Here is the sample I'm using:
-
- Nir Hasson
- Posts: 71
- Joined:
- Contact:
Re: Unity 2019 - Frequent texture updates
Maybe it is related to the actual device.
I’m using iPad Pro and iPhone 11 both change the native pointer after each load data and apply sequence.
For Android I’m using Galaxy Tab S5e T720 Samsung.
For macOS it is a Mac mini from 2015.
I’m using iPad Pro and iPhone 11 both change the native pointer after each load data and apply sequence.
For Android I’m using Galaxy Tab S5e T720 Samsung.
For macOS it is a Mac mini from 2015.
-
- Nir Hasson
- Posts: 71
- Joined:
- Contact:
Re: Unity 2019 - Frequent texture updates
Anyway, if there isn’t a way to upload this raw data to a Noesis texture object I’ll try using a tender target instead.
-
- Nir Hasson
- Posts: 71
- Joined:
- Contact:
Re: Unity 2019 - Frequent texture updates
I can confirm that this issue doesn't affect Android, it was my mistake.
However, on iOS and macOS, the native pointer is changed every few calls to LoadRawTextureData.
The workaround for me now is to upload the data to a Texture2D object and then blit it to a RenderTexture that is wrapped by a Noesis.TextureSource.
However, on iOS and macOS, the native pointer is changed every few calls to LoadRawTextureData.
The workaround for me now is to upload the data to a Texture2D object and then blit it to a RenderTexture that is wrapped by a Noesis.TextureSource.
Re: Unity 2019 - Frequent texture updates
Blitting is adding an inefficiency, we should expose a method to update the internal handle of the texture. As there is no way to detect when this happens, this would be a function manually invoked. But at the same time I read this
So calling this per-frame to detect if the handle changed is also going to be inefficient.Texture.GetNativeTexturePtr
Note that calling this function when using multi-threaded rendering will synchronize with the rendering thread (a slow operation), so best practice is to set up needed texture pointers only at initialization time.
-
- Nir Hasson
- Posts: 71
- Joined:
- Contact:
Re: Unity 2019 - Frequent texture updates
Of course, it is inefficient to blit the same data to a render texture, but that's the current workaround.
What about adding an update method to the BitmapImageSource, where we can simply upload the texture data from the system memory directly to a Noesis object?
What about adding an update method to the BitmapImageSource, where we can simply upload the texture data from the system memory directly to a Noesis object?
Re: Unity 2019 - Frequent texture updates
We can only do efficient uploads to textures internally created by us. We could expose the API for this but I think this is not very correct in Unity, we should find a more 'native' way. In 3.1 we are bringing support for video (the same way we are compatible with audio right now) in Unity. I wonder if this would be a valid approach for you.
Who is online
Users browsing this forum: No registered users and 1 guest