View Issue Details

IDProjectCategoryView StatusLast Update
0003296NoesisGUIUnity3Dpublic2024-06-13 18:12
ReporterOtter Assigned Tojsantos  
PrioritynormalSeveritymajorReproducibilityalways
Status resolvedResolutionfixed 
Product Version3.2.3 
Target Version3.2.4Fixed in Version3.2.4 
Summary0003296: [Quest 2] App freezes for half a second when enabling UI
DescriptionReproduction 1:
- Download Noesis WorldSpaceUI sample and start the app, observe how the app freezes for half a second after splash screen

Reproduction 2:
- Download Noesis WorldSpaceUI sample, disable NoesisView, Object1 and Object2. Add a script that enables NoesisView and both objects after some period of time - when NoesisView is enabled, app freezes (view attached picture)

Additional information:
- Oculus Quest 2
- il2cpp
- Vulkan
- Unity 2022.3.27
TagsNo tags attached.
PlatformAny

Activities

Otter

Otter

2024-05-03 10:51

reporter  

spike.png (117,039 bytes)   
spike.png (117,039 bytes)   
jsantos

jsantos

2024-05-03 14:12

manager   ~0009460

XAML is loaded firs time OnEnable is invoked. We should probably move this to Awake
Otter

Otter

2024-05-03 14:45

reporter   ~0009461

Is this spike caused because its loading XAMLs? Moving the loading to Awake will only fix the second issue (Reproduction 2) if I'm not mistaken.

Is there any way to load them asynchronously, without freezing the main thread? I could show a loading screen and when everything is loaded, show the UI
Otter

Otter

2024-05-06 10:14

reporter   ~0009466

I've also played around with _visible inside NoesisView, if I set _visible to false and then after 10 seconds to true it also freezes for half a second, no matter if OnEnable was already executed and resources loaded
jsantos

jsantos

2024-05-06 13:03

manager   ~0009469

Hello! This is not related to loading XAMLs, sorry about that. This is happening in the render thread, so the spike is related to first time creation of GPU resources. I assume you are only observing the spike once right?
Otter

Otter

2024-05-06 13:45

reporter   ~0009470

Yeah, only first time, then it works perfectly. In VR it gets weird when halts if you move your head
Otter

Otter

2024-05-13 14:12

reporter   ~0009524

Hello, any updates on this?
jsantos

jsantos

2024-05-13 17:40

manager   ~0009525

I haven't had time for this yet.

Just to understand the priority of this, this is just 500ms after the splash right? Or it is more than that in your game?

You don't need to enable the UI at a later time, do you?
Otter

Otter

2024-05-13 18:40

reporter   ~0009526

It depends, the range is between 500ms to 1000ms.

I'm showing the UI immediately. Probably for now I will try to put a black screen until this fully loads.

Is there anything else I can do? Or is the issue more deep down?
jsantos

jsantos

2024-05-14 18:59

manager   ~0009527

I will be working tomorrow on this and hope to bring more information.
jsantos

jsantos

2024-05-16 13:54

manager   ~0009531

The spike is caused by the creation of the pipeline objects needed by Noesis. This can only be done (at least with the current Unity API) within the first draw, because only at that point we have the active renderpass.

PSO creation should be very fast because we are caching from disk, but even with that, Quest is extremely low creating PSOs (the driver doesn't support derivative pipelines) and right now this is taking around 500-700ms.

This can be improved a bit (maybe something like 100-200ms) because there are a few extensions in Quest that can help. I will keep you posted about this.

But at the end, even if we can optimize this a lot, this is always going to be a visible spike (the alternative is calculating the PSOs on demand, whenever they are needed, but this will introduce very small spikes (10ms) in subsequent frames and I think this is less desirable).

I will work on reducing the spike, but what do you think is the best solution on your side to hide it? Maybe rendering the splash itself with Noesis?
Otter

Otter

2024-05-21 13:05

reporter   ~0009546

Last edited: 2024-05-21 13:06

We do not have a splash screen and currently I introduced black screen until Noesis renders the first frames so the user does not get this stuttering feeling - used camera post render function.

I've also discovered if I move NoesisUnity.InitCore and NoesisUnity.Init function into RuntimeInitializeOnLoadMethod (before scene load or splash screen) I get an improvement on scene load, but the downside here is that delay is introduced before and that I cannot use Xaml ScriptableObjects, so I hardcoded some of the xamls inside the static function (maybe codegen would be a solution here, but did not dig more into it).

Would be nice to have an event from Noesis that would tell us that the resources have been uploaded and we can hide loading/splash screen.
jsantos

jsantos

2024-05-21 18:13

manager   ~0009548

I have been working on this, optimizing our PSO cache, and with the following library you should get a spike of only 50-100ms.

https://drive.google.com/file/d/1WGaMLszuKMSvAdHaGnjXU0fSrfu7AVdb/view?usp=sharing

RuntimeInitializeOnLoadMethod should not save you any GPU time. But indeed this is something we want to explore because right now, for some users Noesis is initialized too late.

I think a possible solution for totally hiding the spike, is after Noesis init, create a render texture with a super simple XAML, render to it and wait. This is something we could even implement by default.
Otter

Otter

2024-05-22 10:09

reporter   ~0009552

I have checked it and its a huge improvement! I don't know if this is connected, but in our case this new build broke scroll view and anything else that hides part of UI, basically the content is now overflowing.

I posted RuntimeInitializeOnLoadMethod suggestion out of context, but I was more thinking to use it for preloading all the XAMLs, to get rid of NoesisXaml.Load spike on the main thread. This would make scene loading more smooth, if you load a lot of XAMLs it takes quite some time (also depends how you structure your views, but if you include all the views inside one main XAML as collapsed, then loading would take quite some time)

So if I understand correctly, after NoesisUnity.Init() is called I would call NoesisRenderer.RenderOnScreen (command buffer render target set to render texture)? Does the size of RenderTexture matter and would something like HelloWorld XAML be enough or even LayoutRoot.xaml?
jsantos

jsantos

2024-05-22 10:41

manager   ~0009553

Last edited: 2024-05-22 11:33

Masking is not working right? The content displays outside the scroll viewer. The binaries I sent you are not aligned with 3.2.3 (it is more a 3.2.4). Could you please attach a screenshot?

You need to do something similar to what NoesisXamlEditor.RenderPreview is doing but just with 1x1 and a trivial XAML. I haven't experimented with this yet.
Otter

Otter

2024-05-22 11:27

reporter   ~0009554

Yeah, its showing elements that should be masked
before.jpg (71,558 bytes)   
before.jpg (71,558 bytes)   
after.mp4 (2,954,778 bytes)   
jsantos

jsantos

2024-05-24 02:00

manager   ~0009564

This version fixes the clipping issues

https://drive.google.com/file/d/1bMaovniYmALu9m88tM_J1v92NPGzX0JV/view?usp=sharing
Otter

Otter

2024-05-27 11:00

reporter   ~0009577

I have tested it and it works as expected!
jsantos

jsantos

2024-06-13 18:12

manager   ~0009705

This has been already committed for 3.2.4

Also, Noesis is now initialized using RuntimeInitializeOnLoadMethod (BeforeSplashScreen)

Thanks for your feedback!

Issue History

Date Modified Username Field Change
2024-05-03 10:51 Otter New Issue
2024-05-03 10:51 Otter File Added: spike.png
2024-05-03 14:03 jsantos Assigned To => jsantos
2024-05-03 14:03 jsantos Status new => assigned
2024-05-03 14:12 jsantos Note Added: 0009460
2024-05-03 14:12 jsantos Target Version => 3.2.4
2024-05-03 14:45 Otter Note Added: 0009461
2024-05-06 10:14 Otter Note Added: 0009466
2024-05-06 13:03 jsantos Note Added: 0009469
2024-05-06 13:03 jsantos Status assigned => feedback
2024-05-06 13:45 Otter Note Added: 0009470
2024-05-06 13:45 Otter Status feedback => assigned
2024-05-13 14:12 Otter Note Added: 0009524
2024-05-13 17:40 jsantos Note Added: 0009525
2024-05-13 17:40 jsantos Status assigned => feedback
2024-05-13 18:40 Otter Note Added: 0009526
2024-05-13 18:40 Otter Status feedback => assigned
2024-05-14 18:59 jsantos Note Added: 0009527
2024-05-16 13:54 jsantos Note Added: 0009531
2024-05-16 13:54 jsantos Status assigned => feedback
2024-05-21 13:05 Otter Note Added: 0009546
2024-05-21 13:05 Otter Status feedback => assigned
2024-05-21 13:06 Otter Note Edited: 0009546
2024-05-21 18:13 jsantos Note Added: 0009548
2024-05-21 18:13 jsantos Status assigned => feedback
2024-05-22 10:09 Otter Note Added: 0009552
2024-05-22 10:09 Otter Status feedback => assigned
2024-05-22 10:41 jsantos Note Added: 0009553
2024-05-22 10:41 jsantos Status assigned => feedback
2024-05-22 11:27 Otter Note Added: 0009554
2024-05-22 11:27 Otter File Added: before.jpg
2024-05-22 11:27 Otter File Added: after.mp4
2024-05-22 11:27 Otter Status feedback => assigned
2024-05-22 11:33 jsantos Note Edited: 0009553
2024-05-24 02:00 jsantos Note Added: 0009564
2024-05-24 02:00 jsantos Status assigned => feedback
2024-05-27 11:00 Otter Note Added: 0009577
2024-05-27 11:00 Otter Status feedback => assigned
2024-06-13 18:12 jsantos Status assigned => resolved
2024-06-13 18:12 jsantos Resolution open => fixed
2024-06-13 18:12 jsantos Fixed in Version => 3.2.4
2024-06-13 18:12 jsantos Note Added: 0009705