Nir Hasson
Topic Author
Posts: 66
Joined: 10 Nov 2013, 21:20
Contact:

Unity 2019 - Frequent texture updates

02 Nov 2020, 16:19

I'm using a 3rd party video solution that updates the video data frequently.
The video data is available as a raw bytes array, simple 4 bytes per pixel RGBA format.
On Unity 2018 we simply used the LoadRawTextureData to update a native unity Texture2D object and then used Noesis.TextureSource to wrap this texture and use it as part of our UI.

This solution doesn't work well on Unity 2019 since the LoadRawTextureData and any other API call that updates the pixel changes the internal API native pointer so the Texture.GetNativeTexturePtr returns a new resource.
See unity doc:
https://docs.unity3d.com/ScriptReferenc ... rePtr.html
Release the current used Noesis.TextureSource doesn't seem to release the reference as Noesis.Texture keeps a reference to the wrapped texture at the Texture static Dictionary.

What's the best way to have frequently updated textures available to the UI system via a subclass of Noesis.ImageSource?

Tags:
 
User avatar
sfernandez
Site Admin
Posts: 1974
Joined: 22 Dec 2011, 19:20

Re: Unity 2019 - Frequent texture updates

02 Nov 2020, 19:23

Are you sure native pointer is changing? what platform shows that behavior?
I made a simple test of a texture being updated on every frame using LoadRawTextureData and logging the native pointer always returns the same address (tested on both Unity 2019 and 2020).
Release the current used Noesis.TextureSource doesn't seem to release the reference as Noesis.Texture keeps a reference to the wrapped texture at the Texture static Dictionary.
We keep that reference while texture is being used by the render thread, but we release it as soon as it is no longer used.
What's the best way to have frequently updated textures available to the UI system via a subclass of Noesis.ImageSource?
ImageSource cannot be subclassed. One of the goals of a TextureSource is to allow rendering dynamic textures, but it relies on the fact that texture native resource doesn't change, it won't be very efficient to create a new texture each time you need to change its contents.
 
Nir Hasson
Topic Author
Posts: 66
Joined: 10 Nov 2013, 21:20
Contact:

Re: Unity 2019 - Frequent texture updates

02 Nov 2020, 19:35

It happens on iOS, Android and macOS.
Windows keep the same underlying pointer.
Using unity 2019.4.13f1 LTD
 
User avatar
sfernandez
Site Admin
Posts: 1974
Joined: 22 Dec 2011, 19:20

Re: Unity 2019 - Frequent texture updates

02 Nov 2020, 20:09

It is strange, I tested it on Android and I'm getting always the same native pointer every frame too.

Here is the sample I'm using:
TextureUpdate.unitypackage
(4.54 KiB) Downloaded 9 times
 
Nir Hasson
Topic Author
Posts: 66
Joined: 10 Nov 2013, 21:20
Contact:

Re: Unity 2019 - Frequent texture updates

02 Nov 2020, 20:18

Maybe it is related to the actual device.
I’m using iPad Pro and iPhone 11 both change the native pointer after each load data and apply sequence.
For Android I’m using Galaxy Tab S5e T720 Samsung.
For macOS it is a Mac mini from 2015.
 
Nir Hasson
Topic Author
Posts: 66
Joined: 10 Nov 2013, 21:20
Contact:

Re: Unity 2019 - Frequent texture updates

02 Nov 2020, 20:19

Anyway, if there isn’t a way to upload this raw data to a Noesis texture object I’ll try using a tender target instead.
 
Nir Hasson
Topic Author
Posts: 66
Joined: 10 Nov 2013, 21:20
Contact:

Re: Unity 2019 - Frequent texture updates

03 Nov 2020, 11:08

I can confirm that this issue doesn't affect Android, it was my mistake.
However, on iOS and macOS, the native pointer is changed every few calls to LoadRawTextureData.
The workaround for me now is to upload the data to a Texture2D object and then blit it to a RenderTexture that is wrapped by a Noesis.TextureSource.
 
User avatar
jsantos
Site Admin
Posts: 2959
Joined: 20 Jan 2012, 17:18
Contact:

Re: Unity 2019 - Frequent texture updates

03 Nov 2020, 12:44

Blitting is adding an inefficiency, we should expose a method to update the internal handle of the texture. As there is no way to detect when this happens, this would be a function manually invoked. But at the same time I read this
Texture.GetNativeTexturePtr

Note that calling this function when using multi-threaded rendering will synchronize with the rendering thread (a slow operation), so best practice is to set up needed texture pointers only at initialization time.
So calling this per-frame to detect if the handle changed is also going to be inefficient.
 
Nir Hasson
Topic Author
Posts: 66
Joined: 10 Nov 2013, 21:20
Contact:

Re: Unity 2019 - Frequent texture updates

03 Nov 2020, 13:22

Of course, it is inefficient to blit the same data to a render texture, but that's the current workaround.
What about adding an update method to the BitmapImageSource, where we can simply upload the texture data from the system memory directly to a Noesis object?
 
User avatar
jsantos
Site Admin
Posts: 2959
Joined: 20 Jan 2012, 17:18
Contact:

Re: Unity 2019 - Frequent texture updates

03 Nov 2020, 13:25

We can only do efficient uploads to textures internally created by us. We could expose the API for this but I think this is not very correct in Unity, we should find a more 'native' way. In 3.1 we are bringing support for video (the same way we are compatible with audio right now) in Unity. I wonder if this would be a valid approach for you.

Who is online

Users browsing this forum: Bing [Bot], Google [Bot] and 7 guests