View Issue Details

IDProjectCategoryView StatusLast Update
0001427NoesisGUIUnity3Dpublic2019-04-30 06:18
ReporternokolaAssigned Tojsantos 
PrioritynormalSeverityblockReproducibilityalways
Status closedResolutionreopened 
Product Version2.2.0 
Target Version2.2.2Fixed in Version2.2.2 
Summary0001427: Noesis renders to wrong camera in multi-camera case (Android - ISSUE, iOS - unknown, Editor - OK)
DescriptionMarking this as "block" because I can't deploy my app with Noesis 2.2 - I'm not sure what will be affected in multi-camera case, and we found this issue by accident.

I have a setup with 3 cameras:
- Camera.main rendering noesisUI as a HUD on top of the screen. Camera.main only renders Noesis UI.
- "From" camera rendering image in top half of screen (Red color in the example project)
- "To" camera rendering image in bottom half of screen (Green color in the example project)

I need the UI from camera.main to be transparent (render on top of the rest), thus setting ClearFlags to None. When this happens, Noesis starts rendering in the "To" camera (sometimes in the "From" camera) instead of MainCamera
Steps To Reproduce1. Open attached Noesis22Multicam_REPRO2.zip and open Buttons sample
https://1drv.ms/u/s!AspGCVZdWgTBu5kqBF8-av9QFpNEHA
Side note: can you please increase the max upload file size to at least 40MB from 5MB - "hello world" Noesis project is .zipped as 35MB
2. Run in editor. Note everything is OK
3. Deploy to Android phone.
Expected: All OK
Actual: After frame 200 when the main camera switches to ClearFlags.None Noesis will render in the wrong camera.

See NewScriptBehavior.cs and attached video.
TagsNo tags attached.
PlatformAny

Relationships

has duplicate 0001446 closedjsantos Noesis Views Fail to Render 

Activities

nokola

nokola

2019-03-17 19:38

reporter   ~0005534

Adding screenshot
nokola

nokola

2019-03-17 19:48

reporter   ~0005535

Note: this is issue in 2.2 only. Works OK in 2.1. Also OK in editor, but fails on Android phone.
jsantos

jsantos

2019-03-21 17:12

manager   ~0005549

This is an untested workaround. I don't have the time to properly solve this right now (will do next week), but as this seems to be critical for you, could you try the following patch?

NoesisView.cs.patch (1,267 bytes)
Index: NoesisView.cs
===================================================================
--- NoesisView.cs	(revision 7956)
+++ NoesisView.cs	(working copy)
@@ -443,10 +443,13 @@
         }
 
         LoadXaml(false);
+
+        Camera.onPreRender += PreRender;
     }
 
     void OnDisable()
     {
+        Camera.onPreRender -= PreRender;
     }
 
     void OnDestroy()
@@ -749,8 +752,21 @@
         }
     }
 
-    private void OnPreRender()
+    private bool _updatePending = true;
+
+    private void PreRender(Camera cam)
     {
+        // We need the offscreen phase to happen before any camera rendering. This is critical for tiled architectures.
+        // This method is invoked for each camera in the scene. We enqueue the offscreen phase in the first camera and disable for rest of cameras
+        if (_updatePending)
+        {
+            RenderOffscreen();
+            _updatePending = false;
+        }
+    }
+
+    void RenderOffscreen()
+    {
         if (_uiView != null && _visible)
         {
             _commands.Clear();
@@ -819,6 +835,8 @@
             {
                 _texture.DiscardContents(false, true);
             }
+
+            _updatePending = true;
         }
     }
 
NoesisView.cs.patch (1,267 bytes)
nokola

nokola

2019-03-21 17:51

reporter   ~0005551

Thanks! Seems to work on first try - I'll test it and release (temporary) update of my app this week until the proper fix.
jsantos

jsantos

2019-03-26 21:51

manager   ~0005560

Could you please try with the attached NoesisView.cs? This is the right solution, it is a workaround for Unity bug and also an optimization for current 2.2 behavior when you mix cameras to the same render target in tiled architectures (mobiles).
nokola

nokola

2019-03-27 06:00

reporter   ~0005561

Unfortunately it doesn't work. The cameras are activated later (based on what the user does) and the NoesisView.cs code is reading them on onEnable()
During onEnable() there's only Camera.main typically. Why isn't the offscreen phase applied to the camera Noesis is attached to (regardless of any other cameras?) Perhaps it has something with depth sorting?
nokola

nokola

2019-03-27 06:34

reporter   ~0005562

I thought more about it - in my opinion, Noesis offscreen phase should be synchronized the the camera the Noesis script is attached to (unless I'm missing something) - will work as expected in such case.
Perhaps I'm missing something about offscreen rendering?
jsantos

jsantos

2019-03-27 14:20

manager   ~0005563

So, yes. That's what is happening for normal cameras (like a camera for the HUD, a camera for a render texture,...) the offscreen phase is rendered at PreRender time of that camera.

Problem is with cameras that renders to the same texture. In those cameras, only the first camera (by depth order) changes the active render target, the following ones reuse it.

So, if noesis is being used in one of those cameras (not the first one) and we do the offscreen in that camera, a render target change is happening with the following implications:

1. In Android, Unity is bugged and if the render target is changed it is not properly restored.
2. Changing the active render target temporally to render a few things in a different texture (like offscreens) and then going back to the original render target is extremely inefficient in Tiled based architectures (all mobiles).

So we need a way to render the offscreens at the first camera (by depth order) just *before* the render target is bound. My new code, was trying to do that, but I think that the selection of the first camera must be done by frame. I am not sure if I like this approach.

Another approach, is not trying to be smart and expose a property in the component to indicate that camera. I don't like this option neither because it is a manual process and a high percentage of users won't set it up correctly.

So I need to think more about it.
nokola

nokola

2019-03-27 16:04

reporter   ~0005564

Hmm, yes - I see. Would the previous workaround with _updatePending work? It does look more "hacky" but may be the right one.
jsantos

jsantos

2019-03-27 16:32

manager   ~0005565

That workaround is the solution we had in 2.1 but it is inefficient for "normal" cameras because all offscreen from all cameras are done first, before any camera. So, if each offscreen phase needs one extra texture, and you have 4 cameras. you need 4x the number of render targets. If you do the offscreen when each camera needs then we can resuse the offscreen textures.

Not sure, if I am making myself clear here :)
nokola

nokola

2019-03-28 02:21

reporter   ~0005566

Yes I got it - thanks!

Interesting problem.
jsantos

jsantos

2019-03-28 16:29

manager   ~0005567

I am working on a new solution, will send you later
jsantos

jsantos

2019-03-28 16:31

manager   ~0005568

By the way, this way to use cameras (Stacked cameras in Unity parlance) is deprecated in Scriptable Render Pipeline because it is inefficient and hard to implement (buggy) for Unity.

It is a pity they didn't deprecated in the fixed pipeline. :)
nokola

nokola

2019-03-28 16:33

reporter   ~0005569

re: deprecated: Is there another option to render "Noesis UI on top of two cameras"?
jsantos

jsantos

2019-03-28 17:32

manager   ~0005570

https://forum.unity.com/threads/glitching-with-multiple-cameras-lwrp.592477/

They are deprecating multicamera layered rendering in LWRP and HDRP. For us it is going to be easier because we just do the offscreen in the attached camera, exactly what we are doing in 2.2.0. For you, I think you need to use render passes, but I am not sure about it, they are in beta and many things are changing...

But you can stay in fixed pipeline where layered cameras will be supported and we need to fix this issue.
jsantos

jsantos

2019-03-28 22:36

manager   ~0005571

Please, try this new version
nokola

nokola

2019-03-29 07:46

reporter   ~0005574

Works! I changed the code a bit to make it clearer and avoid calling GetComponent<Camera> in Update() - a small thing, I know but just a matter of principle. Attached file with changes.
Very unlikely myCamera will change anyway.

Thanks for the fix!
jsantos

jsantos

2019-03-29 09:57

manager   ~0005575

Thanks! I took your changes, did a few more changes and uploaded to our repository.

NoesisView-4.cs (41,918 bytes)
jsantos

jsantos

2019-03-30 16:36

manager   ~0005578

I will try it later. When is the iOS error happening? when the camera changes to "Do not clear" ?
nokola

nokola

2019-03-30 19:25

reporter   ~0005579

Yes, when camera changes to "do not clear" (or perhaps next render after)
This is the approximate timeline:
1. app start (1 camera)
2. some rendering
3. second and third camera enabled, each taking half screen, top camera set to "do not clear"
4. exception

Hope that helps! If you need any further specific info please let me know! Otherwise I'm going to work on my other tasks. So far I've spent 2+ weeks upgrading to 2.2 in total. *Not* frustrated just making sure to communicate the timeline so that Noesis is aware. Would be nice if upgrade time was less than 1 week - wondering what's the best way to achieve it. I'm not sure if I'm an "edge case" customer (since I have somewhat complex UI, and thus upgrade time is expected) or not.
jsantos

jsantos

2019-03-31 20:24

manager   ~0005580

Last edited: 2019-03-31 20:26

View 4 revisions

I am not sure if this is still related to the same bug in Unity you were hitting on Android but this is another bug in Unity, they are sending us a null pointer when they shouldn't. In the scene I am testing (just two cameras, one rendering skybox and the second one rendering the UI) I am able to crash even with the GDC patch I sent you.

I am able to workaround by setting the Camera Culling Mask to: Nothing, this workaround is a minor optimization in fact. Please let me know if this works for you. If it does we need to document it.

Regarding the time for the integration. Well, problem is we took a lot of time between 2.1 and 2.2, more than a year. This version brought many many things and it is normal to have longer integration times. Honestly, apart from this bug I think you didn't hit more bugs (apart from deviation from WPF we fixed, that yes, they took you time also). We have more clients still integrating, and also clients that did finish the integration. You are not a edge case customer, just you were unlucky hitting this bug.

Our solution to avoid this: release often. That's what we want to achieve with 2.3...

nokola

nokola

2019-04-01 06:31

reporter   ~0005581

Thanks for the update! Yes, the more frequent update plan sounds good.

Unfortunately, the problem with the crash is still here with Camera Culling Mask to: Nothing
I'm using the GDC patch, which works for now... (will have to test on many devices to make sure)
jsantos

jsantos

2019-04-01 07:05

manager   ~0005582

:( I don't know what's going on. I was able to reproduce the same error on macOS, with two cameras (one for Noesis) and the GDC patch didn't help. So I will try later today reproducing exactly your same scenario (3 cameras) and on iOS.
nokola

nokola

2019-04-01 07:52

reporter   ~0005583

Perhaps some weird Unity bug... :(
Thanks! Just to clarify: for the moment, I'm unblocked (GDC patch works and performance is good for my use case.)
Worst case, I'll change my render code to manually clip and adapt all functions to use single camera only (could take several days) instead of using multiple cameras.
On Android so far in the last 4 days since release we haven't heard of complaints, which is good. iOS is still released with Noesis 2.1. I plan to test on more devices in the following week before getting iOS on Noesis 2.2 as well.
jsantos

jsantos

2019-04-01 13:56

manager   ~0005584

I tried to recreate your same scene: 1 camera rendering red up, 1 camera rendering blue bottom and a third-camera (Don't clear) with NoesisGUI rendering a UI using offscreen. If I set the culling mask to 'Nothing' it works on macOS and iOS (Unity2018.3.11). Could you please try it? If it works for you then I need to understand what's different in your project.

Thanks!
jsantos

jsantos

2019-04-01 14:05

manager   ~0005585

In fact, on iOS I don't need the culling mask workaround... that's only needed for macOS
nokola

nokola

2019-04-05 16:23

reporter   ~0005596

I got the project - didn't try it, but it will work. The cameras are set-up at project start. Did you try your latest fix with the 3-camera project I sent when I opened the bug? It's in description. Does 2.2.1 work with it?
nokola

nokola

2019-04-05 16:24

reporter   ~0005597

I think the difference is that I'm setting cameras at runtime (as in the repro project I sent), not at project start.
nokola

nokola

2019-04-05 16:26

reporter   ~0005598

I manually modified 2.2.1 NoesisView.xaml with the initial patch that is not ideal, but works - attached
jsantos

jsantos

2019-04-05 21:14

manager   ~0005599

You are right, I didn't try delaying the change to 'do not clear'. I will try later today. Thanks!
stonstad

stonstad

2019-04-06 17:01

reporter   ~0005606

NoesisView4.cs does not resolve the behavior for me on PC/default render pipeline.
jsantos

jsantos

2019-04-06 17:03

manager   ~0005607

Please, try with the last patch from nokola (0001427:0005598 )
nokola

nokola

2019-04-06 17:19

reporter   ~0005608

btw, for my use case the NoesisView-2.2.1-modified-working.cs is ideal! I mentioned "not ideal" because jsantos mentioned it has some performance implications on mobile. I've tested NoesisView-2.2.1-modified-working.cs on mobile - for me no performance issue at all (although I only render 2 quads + Noesis.)

Just mentioning that I'm happy with NoesisView-2.2.1-modified-working.cs as current solution
stonstad

stonstad

2019-04-06 17:37

reporter   ~0005609

Thank you, I'm back and running!
nokola

nokola

2019-04-06 19:11

reporter   ~0005611

Which patch (of the 4 or 5 above) fixed it for you stonstad?
jsantos

jsantos

2019-04-06 19:43

manager   ~0005612

I think he is using your last patch.

@stontad could you please describe how you are setting up your cameras (how many, what depth order, clearing options etc) and if you are dynamically changing properties (clearing, depth).

I need to understand all the scenario to properly find a solution to this mess.
jsantos

jsantos

2019-04-24 22:12

manager   ~0005640

Last edited: 2019-04-24 22:13

View 3 revisions

Ok, so I found a much better approach (aligned with the idea in 2.1) and also doing the needed optimizations for tiles architectures. I think this is the best solution.

Note that you still need to use Culling Mask se to None in cameras rendering only NoesisGUI (not rendering 3D). This is to avoid a bug in Unity that I am going to report later today. I was able to reproduce this issue even with 2.1

Could you please try it?

jsantos

jsantos

2019-04-26 11:06

manager   ~0005645

Last edited: 2019-04-26 11:07

View 4 revisions

We are going to include this patch in the upcoming 2.2.2, we also added the following mention in the documentation

   UNITY BUG
   When using Camera Stacking (several cameras rendering to the same render target)
   make sure the camera rendering the UI has the property **'Culling Mask'** set to **'Nothing'**.
   This will avoid a bug in the latest versions of Unity.


stonstad

stonstad

2019-04-26 17:31

reporter   ~0005647

NoesisView-5.cs works without issue using my two camera/view setup!
jsantos

jsantos

2019-04-29 03:05

manager   ~0005648

Please, reopen if needed
nokola

nokola

2019-04-30 06:18

reporter   ~0005652

Thanks for the fix! Verified OK on Android and iOS

Issue History

Date Modified Username Field Change
2019-03-17 19:31 nokola New Issue
2019-03-17 19:31 nokola File Added: Noesis_RenderIssue.mp4
2019-03-17 19:38 nokola File Added: Noesis_Multicam_Issue.png
2019-03-17 19:38 nokola Note Added: 0005534
2019-03-17 19:48 nokola Note Added: 0005535
2019-03-18 20:03 jsantos Assigned To => jsantos
2019-03-18 20:03 jsantos Status new => assigned
2019-03-21 17:12 jsantos File Added: NoesisView.cs.patch
2019-03-21 17:12 jsantos Note Added: 0005549
2019-03-21 17:12 jsantos Status assigned => feedback
2019-03-21 17:12 jsantos Target Version => 2.2.1
2019-03-21 17:12 jsantos Description Updated View Revisions
2019-03-21 17:12 jsantos Steps to Reproduce Updated View Revisions
2019-03-21 17:51 nokola Note Added: 0005551
2019-03-21 17:51 nokola Status feedback => assigned
2019-03-26 21:51 jsantos File Added: NoesisView.cs
2019-03-26 21:51 jsantos Note Added: 0005560
2019-03-26 21:52 jsantos Status assigned => feedback
2019-03-27 06:00 nokola Note Added: 0005561
2019-03-27 06:00 nokola Status feedback => assigned
2019-03-27 06:34 nokola Note Added: 0005562
2019-03-27 14:20 jsantos Note Added: 0005563
2019-03-27 16:04 nokola Note Added: 0005564
2019-03-27 16:32 jsantos Note Added: 0005565
2019-03-28 02:21 nokola Note Added: 0005566
2019-03-28 16:29 jsantos Note Added: 0005567
2019-03-28 16:31 jsantos Note Added: 0005568
2019-03-28 16:33 nokola Note Added: 0005569
2019-03-28 17:32 jsantos Note Added: 0005570
2019-03-28 22:36 jsantos File Added: NoesisView-2.cs
2019-03-28 22:36 jsantos Note Added: 0005571
2019-03-28 22:40 jsantos Status assigned => feedback
2019-03-29 07:46 nokola File Added: NoesisView-3.cs
2019-03-29 07:46 nokola Note Added: 0005574
2019-03-29 07:46 nokola Status feedback => assigned
2019-03-29 09:57 jsantos File Added: NoesisView-4.cs
2019-03-29 09:57 jsantos Note Added: 0005575
2019-03-29 09:57 jsantos Status assigned => resolved
2019-03-29 09:57 jsantos Resolution open => fixed
2019-03-29 09:57 jsantos Fixed in Version => 2.2.1
2019-03-30 07:34 jsantos Status resolved => feedback
2019-03-30 07:34 jsantos Resolution fixed => reopened
2019-03-30 16:36 jsantos Note Added: 0005578
2019-03-30 19:25 nokola Note Added: 0005579
2019-03-30 19:25 nokola Status feedback => assigned
2019-03-31 20:24 jsantos Note Added: 0005580
2019-03-31 20:24 jsantos Note Edited: 0005580 View Revisions
2019-03-31 20:24 jsantos Note Edited: 0005580 View Revisions
2019-03-31 20:26 jsantos Note Edited: 0005580 View Revisions
2019-04-01 06:31 nokola Note Added: 0005581
2019-04-01 07:05 jsantos Note Added: 0005582
2019-04-01 07:52 nokola Note Added: 0005583
2019-04-01 13:56 jsantos File Added: 3cameras.unitypackage
2019-04-01 13:56 jsantos Note Added: 0005584
2019-04-01 13:56 jsantos Status assigned => feedback
2019-04-01 14:05 jsantos Note Added: 0005585
2019-04-05 12:38 sfernandez Target Version 2.2.1 => 2.2.2
2019-04-05 16:23 nokola Note Added: 0005596
2019-04-05 16:23 nokola Status feedback => assigned
2019-04-05 16:24 nokola Note Added: 0005597
2019-04-05 16:26 nokola File Added: NoesisView-2.2.1-modified-working.cs
2019-04-05 16:26 nokola Note Added: 0005598
2019-04-05 21:14 jsantos Note Added: 0005599
2019-04-06 17:01 stonstad Note Added: 0005606
2019-04-06 17:03 jsantos Note Added: 0005607
2019-04-06 17:19 nokola Note Added: 0005608
2019-04-06 17:37 stonstad Note Added: 0005609
2019-04-06 19:11 nokola Note Added: 0005611
2019-04-06 19:43 jsantos Note Added: 0005612
2019-04-08 11:06 sfernandez Relationship added has duplicate 0001446
2019-04-24 22:12 jsantos File Added: NoesisView-5.cs
2019-04-24 22:12 jsantos Note Added: 0005640
2019-04-24 22:13 jsantos Note Edited: 0005640 View Revisions
2019-04-24 22:13 jsantos Note Edited: 0005640 View Revisions
2019-04-24 22:13 jsantos Status assigned => feedback
2019-04-26 11:06 jsantos Note Added: 0005645
2019-04-26 11:06 jsantos Note Edited: 0005645 View Revisions
2019-04-26 11:07 jsantos Note Edited: 0005645 View Revisions
2019-04-26 11:07 jsantos Note Edited: 0005645 View Revisions
2019-04-26 17:31 stonstad Note Added: 0005647
2019-04-27 00:40 jsantos Fixed in Version 2.2.1 => 2.2.2
2019-04-29 03:05 jsantos Status feedback => resolved
2019-04-29 03:05 jsantos Note Added: 0005648
2019-04-30 06:18 nokola Status resolved => closed
2019-04-30 06:18 nokola Note Added: 0005652
2019-04-30 18:14 jsantos File Deleted: NoesisView.cs
2019-04-30 18:14 jsantos File Deleted: NoesisView-2.cs
2019-04-30 18:14 jsantos File Deleted: NoesisView-3.cs
2019-04-30 18:15 jsantos File Deleted: 3cameras.unitypackage
2019-04-30 18:15 jsantos File Deleted: Noesis_Multicam_Issue.png
2019-04-30 18:15 jsantos File Deleted: NoesisView-5.cs
2019-04-30 18:15 jsantos File Deleted: NoesisView-2.2.1-modified-working.cs
2019-04-30 18:16 jsantos File Deleted: Noesis_RenderIssue.mp4