Forums | NoesisGUI

Context:
Our app is an image editor and is very UI-heavy. We noticed that it crashes during stress testing doing the following repeatedly:
1. Open an image
2. Going through few XAML screens (just to select effect below)
3. Run simple "convert to grayscale" effect on GPU
4. go to 1

After a week of investigation about the source of the crash and more than 50 runs with a variety of code changes and debug logging, we figured out it's GPU memory fragmentation.
[Very] Long story short - after many other changes, in the end, we turned off Noesis completely and saw this:
With Noesis: app crashes after 54 open-effect-close iterations
Without Noesis: app keeps running after 330 open-effect-close iterations

We think something like this is happening in the GPU memory, where "A" means "app allocated memory" and "N" means "Noesis allocated memory" GPU (and maybe CPU) memory "creeps up" on every iteration, thus causing an early crash. This is on Android phone.

Question
How can we better manage Noesis memory to ensure no fragmentation and crash happens? e.g. maybe a trick to specify to Noesis "allocate this much and use this space as your own?"

I remember before there used to be pre-determined off-screen surface size. I think that was helpful in managing fragmentation. Having something like "minimum pre-determined off-screen surface size" would be great to avoid the crash. We could set during app start to what we think is the most Noesis would use and thus prevent crashes due to GPU memory fragmentation.

Are you sure this is GPU fragmentation? Our GPU allocations are kept to a minimum:

• Just a single fixed buffer is a allocated for streaming geometry. This is allocated at the beginning and never reallocated again.
• Offscreen textures are dynamically recreated each time you need bigger a bigger offscreens, but this stabilizes soon. Once it is stabilized we don't reallocate again. As you mention, you can preallocate everything to improve the performance. You can adjust the max number of surfaces and resolution in the Noesis settings panel. Could you try it?
• GPU resources like shaders, geometry descriptors and state objects are all allocated at the beginning and never reallocated again.
• Textures: we use the handle provided by Unity, so we don't allocate textures directly. Are you using textures in this scenario? Could you test with and without textures to see what happens?

Thanks for your feedback.

Thanks for the suggestions - trying! This is very helpful info!
Yes, I suspect it's GPU since after some test iterations we get a "GL_OUT_OF_MEMORY: Not enough memory left to execute command" error. This said, in several cases we'd get the GL_OUT_OF_MEMORY, but the app wouldn't crash immediately - just the screen would be blank as if the RenderTexture was not allocated. The app would crash few more iterations after with CPU (malloc) error. We're trying to get the GPU side improved first.

Few more questions:
1. "Offscreen Texture Size" in settings: Is it a "minimum size" or maximum (set in stone?) Will Noesis allocate above that value in case it's needed?
2. What exactly are "init surfaces" in settings? What is the best practice of determining the # of surfaces I need?
3. If I choose say Texture Size = 2048x2048 and 5 init surfaces, will Noesis allocate memory 2048*2048*5 pixels?
2. re: textures

Are you using textures in this scenario?

Do you mean "imgLinePicker.Source = new TextureSource(...)?
If that's the case, yes we do - sometimes several. I'll remove all such calls and see what happens.

Anyway, this is very good guidance. I'm excited to try it out.

Hi!

1. It is the start size. We will allocate more if needed (releasing all offscreen textures and realocating again)
2. It is the number of offscreen surfaces that we create at the beginning. You need as many textures as the maximum level of nested opacities used. So if you use opacity in a parent and a child use opacity again, that's two levels. We don't recommend using more than two levels so, two textures is fine here. But you can try with more. Unfortunately there is no way, right now (in Unity), to see how many surfaces are being used. This is something we need to improve.
3. 2048*2048*5 "RGBA" pixels yes.

Regarding textures, yes, Unity always create them (we just use the gpu handle). If this part is the one giving you trouble I think you will need to find a Unity solution for this.

Do you already have any clues?

Thanks for the info! Clues - somewhat.
We did (a lot) more tests and found out that:
1. When we reload XAML views and synthetic test (important,) the app crashes after 105-130 iterations. Black screen on iteration 5 due to rendertexture alloc failure (GPU memory fragmentation.)
2. Without XAML views cached, app doesn't crash even after 1000 iterations. Black screen happens around iteration 500, but "recovers". The XAML views are just 3 UserControl-s: main view, selection view, edit view, all of them spanning the full screen.

3. Also noticed that when we remove all Image.Source values from XAML, the black screen never appears.

Will send more info once we have some. In the meantime I welcome any pointers! Does noesis load images every time it sees <Image Source="imagepath.jpg" /> in XAML or is there some caching? e.g. if I load a UserControl, then unload, then load the same again -> use the same image? (I'd suspect the render textures are re-created which is a good thing imo - just checking.)

Will send more info once we have some. In the meantime I welcome any pointers! Does noesis load images every time it sees <Image Source="imagepath.jpg" /> in XAML or is there some caching? e.g. if I load a UserControl, then unload, then load the same again -> use the same image? (I'd suspect the render textures are re-created which is a good thing imo - just checking.)

Noesis does not load images. We just delegate that part to Unity. If you open any .asset belonging to a XAML you will see in the Inspector all the dependencies, including the images being used by it. So I am not sure about the algorithm that Unity is using for loading or unloading images. I will try to figure it out.

Thanks! We're experimenting by assigning the images as TextureSource after the XAML loads, then we can see if that part of the load/unload causes fragmentation

Quick update: We changed the way we load XAML - instead of opening each of our four .xaml views whenever we need it and releasing it after, we never release any views and just cache them in memory once loaded.
With our new IsCache == true, our app crashes on 873-880 test case iterations, compared to 60-90 iterations until crash when we had Noesis reload each of the 4 XAML views. It's a huge improvement.
I *think* we worked around whatever Noesis issue was causing the crash and may be facing issues with our app. Will update this thread again when we have more info.

That's definitely an improvement! Great work!

One question, when you say "...we never release any views and just cache them...", a view here is a NoesisView or you mean the XAMLs themselves? Because NoesisView is a heavy object, in normal circumstances you only need one that is create at the beginning. Each time you need to load a new XAML you just put it at the root of the view.