Page 1 of 2

Unity 2019.2.x OnGUI Heap Allocation

Posted: 06 Dec 2019, 17:01
by stonstad
The Unity MonoBehavior function OnGUI has a 0.7 kb per-frame gc allocation. Unity does not intend to fix this, as OnGUI is deprecated, per this thread: https://forum.unity.com/threads/gc-coll ... ui.642229/

NoesisView.cs uses OnGUI as follows:
    void OnGUI()
    {
        if (_uiView != null)
        {
            UnityEngine.GUI.depth = -(int)_myCamera.depth;
            ProcessEvent(UnityEngine.Event.current, _enableKeyboard, _enableMouse, _emulateTouch);
        }
    }
I can confirm the following:
1. OnGUI leaks memory without the internal Noesis logic.
2. Commenting out OnGUI completely removes the 0.7 kb per frame allocation.

To eliminate this allocation, is there a different location this Noesis logic may be moved to?

Image

Re: Unity 2019.2.x OnGUI Heap Allocation

Posted: 12 Dec 2019, 01:02
by jsantos
Thanks for taking the time to report this (#1596). This is definitely something we need to investigate because I am not sure how we should get the input events if we are not using OnGUI. I assume this would be using the new UI, but the fact that there is a new UI coming soon makes me wonder if this is the right moment to do this change. Also, using OnGUI we needed to workaround many things to avoid weird bugs in Unity. I assume if we change to something new this is going to happen again taking more time than expected. But yes, thanks!

Re: Unity 2019.2.x OnGUI Heap Allocation

Posted: 12 Dec 2019, 04:18
by stonstad
A few thoughts around this ... I'm honestly not keen to release my game to production with a 60 KB per second GC allocation. I believe it will cause issues on XBOX. I totally agree that this is a Unity bug... and unfortunately, the Unity team isn't likely to fix these IMGUI GC allocations, not now, not ever. IMGUI was replaced in 2015 by Unity UI and five years is ancient history to Unity developers Moreover, OnGUI() is broken by design because it allocates heap objects for eventing -- several hundred or thousand a second in some scenarios.

So what's the viable alternative to OnGUI? Since you asked... One approach is to use Input.GetMouseDown and Input.GetKeyboard. It is doable -- I'm sharing a working proof of concept here. I need to modify it further so that it handles multiple input events simultaneously, but for now, this expresses the concept and it works in my game.

* I edited my post to be more agreeable. I was tired and my initial thoughts didn't translate well.

1. Comment Out
//void OnGUI()
    //{
    //    if (_uiView != null)
    //    {
    //        UnityEngine.GUI.depth = -(int)_myCamera.depth;
    //        ProcessEvent(UnityEngine.Event.current, _enableKeyboard, _enableMouse, _emulateTouch);
    //    }
    //}
2. Add:
   
   private void Update()
    {
        if (_uiView != null)
            ProcessEvent2(_enableKeyboard, _enableMouse, _emulateTouch);
    }
3. Add
private void ProcessEvent2(bool enableKeyboard, bool enableMouse, bool emulateTouch)
    {
        // Process keyboard modifiers
        if (enableKeyboard)
        {
            EventModifiers currentModifiers = EventModifiers.None;
            if (Input.GetKey(KeyCode.LeftShift))
                currentModifiers |= EventModifiers.Shift;
            if (Input.GetKey(KeyCode.LeftControl))
                currentModifiers |= EventModifiers.Control;

            EventModifiers delta = currentModifiers ^ _modifiers;
            if (delta > 0)
            {
                _modifiers = currentModifiers;

                ProcessModifierKey(currentModifiers, delta, EventModifiers.Shift, Key.LeftShift);
                ProcessModifierKey(currentModifiers, delta, EventModifiers.Control, Key.LeftCtrl);
                ProcessModifierKey(currentModifiers, delta, EventModifiers.Command, Key.LeftCtrl);
                ProcessModifierKey(currentModifiers, delta, EventModifiers.Alt, Key.LeftAlt);
            }
        }

        EventType eventType = EventType.Ignore;

        if (Input.GetMouseButtonDown(0) || Input.GetMouseButtonDown(1) || Input.GetMouseButtonDown(2))
            eventType = EventType.MouseDown;
        else if (Input.GetMouseButtonUp(0) || Input.GetMouseButtonUp(1) || Input.GetMouseButtonUp(2)) // naieve approach given multiple inputs may fire simultaneously
            eventType = EventType.MouseUp;
        else if (Input.anyKeyDown)
            eventType = EventType.KeyDown;

        switch (eventType)
        {
            case UnityEngine.EventType.MouseDown:
                {
                    if (enableMouse)
                    {
                        //UnityEngine.Vector2 mouse = ProjectPointer(Input.mousePosition.x, UnityEngine.Screen.height - Input.mousePosition.y);
                        UnityEngine.Vector2 mouse = new Vector2(Input.mousePosition.x, UnityEngine.Screen.height - Input.mousePosition.y);

                        //if (HitTest(mouse.x, mouse.y))
                        //{
                        //    ev.Use();
                        //}

                        if (emulateTouch)
                        {
                            _uiView.TouchDown((int)mouse.x, (int)mouse.y, 0);
                            _touchEmulated = true;
                        }
                        else
                        {
                            // Ignore events generated by Unity to simulate a mouse down when a touch event occurs
                            bool mouseEmulated = Input.simulateMouseWithTouches && Input.touchCount > 0;
                            if (!mouseEmulated)
                            {
                                //if (ev.clickCount == 1)
                                //{
                                if (Input.GetMouseButtonDown(0))
                                    _uiView.MouseButtonDown((int)mouse.x, (int)mouse.y, MouseButton.Left);
                                else if (Input.GetMouseButtonDown(1))
                                    _uiView.MouseButtonDown((int)mouse.x, (int)mouse.y, MouseButton.Right);
                                else if (Input.GetMouseButtonDown(2))
                                    _uiView.MouseButtonDown((int)mouse.x, (int)mouse.y, MouseButton.Middle);
                                //}
                                
                                // TODO: store click count, if truly needed
                                //else 
                                //{
                                //_uiView.MouseDoubleClick((int)mouse.x, (int)mouse.y, (Noesis.MouseButton)ev.button);
                                //}
                            }
                        }
                    }
                    break;
                }
            case UnityEngine.EventType.MouseUp:
                {
                    if (enableMouse)
                    {
                        //UnityEngine.Vector2 mouse = ProjectPointer(Input.mousePosition.x, UnityEngine.Screen.height - Input.mousePosition.y);
                        UnityEngine.Vector2 mouse = new Vector2(Input.mousePosition.x, UnityEngine.Screen.height - Input.mousePosition.y);

                        //if (HitTest(mouse.x, mouse.y))
                        //{
                        //    ev.Use();
                        //}

                        if (emulateTouch && _touchEmulated)
                        {
                            _uiView.TouchUp((int)mouse.x, (int)mouse.y, 0);
                            _touchEmulated = false;
                        }
                        else
                        {
                            // Ignore events generated by Unity to simulate a mouse up when a touch event occurs
                            bool mouseEmulated = Input.simulateMouseWithTouches && Input.touchCount > 0;
                            if (!mouseEmulated)
                            {
                                if (Input.GetMouseButtonUp(0))
                                    _uiView.MouseButtonUp((int)mouse.x, (int)mouse.y, MouseButton.Left);
                                else if (Input.GetMouseButtonUp(1))
                                    _uiView.MouseButtonUp((int)mouse.x, (int)mouse.y, MouseButton.Right);
                                else if (Input.GetMouseButtonUp(2))
                                    _uiView.MouseButtonUp((int)mouse.x, (int)mouse.y, MouseButton.Middle);
                            }
                        }
                    }
                    break;
                }
            case UnityEngine.EventType.KeyDown:
                {
                    if (enableKeyboard)
                    {
                        foreach (var kvp in NoesisKeyCodes._unityToNoesis)
                        {
                            if (Input.GetKeyDown(kvp.Key))
                                _uiView.KeyDown(kvp.Value);

                        }

                    }
                    break;
                }
            case UnityEngine.EventType.KeyUp:
                {
                    //// Don't process key when IME composition is being used
                    //if (enableKeyboard)
                    //{
                    //    if (ev.keyCode != KeyCode.None && NoesisUnity.IME.compositionString == "")
                    //    {
                    //        Noesis.Key noesisKeyCode = NoesisKeyCodes.Convert(ev.keyCode);
                    //        if (noesisKeyCode != Noesis.Key.None)
                    //        {
                    //            _uiView.KeyUp(noesisKeyCode);
                    //        }
                    //    }
                    //}
                    break;
                }
        }
    }

Re: Unity 2019.2.x OnGUI Heap Allocation

Posted: 12 Dec 2019, 13:51
by stonstad
To handle KeyUp I'm thinking of an enum and bitwise OR for deltas. Would still have to enumerate through all keys ... but a non-allocating iteration to from 0 to 100 per frame is no big deal. I'd be happy to develop this further and submit as a PR for consideration. I won't be offended if it isn't used. Right now I have two changes for NoesisView that I'm merging on version upgrades. 1) The exception GC allocation behavior for unimplemented inputs, and this behavior. Just trying to minimize heap allocations to avoid GC frame stutter.

These two allocations in NoesisView are literally the last two ("every frame") allocations I have in my game.

Re: Unity 2019.2.x OnGUI Heap Allocation

Posted: 12 Dec 2019, 14:39
by stonstad
https://docs.unity3d.com/Manual/GUIScri ... 1570476412
The Immediate Mode GUI system is commonly used for:

Creating in-game debugging displays and tools.
Creating custom inspectors
for script components.
Creating new editor windows and tools to extend Unity itself.

The IMGUI system is not generally intended to be used for normal in-game user interfaces that players might use and interact with.

Re: Unity 2019.2.x OnGUI Heap Allocation

Posted: 16 Dec 2019, 19:12
by jsantos
When we started NoesisGUI, OnGUI was the only way to get events. And that was, yeah, long time ago 😃

I would like to find a better way than polling the events for each key. I am sure UnityUI is doing it in a different way and the source code is public so I will try to find time to analyze this soon.

As always, thanks for pushing this, we need to get rid of those GC allocs.

Re: Unity 2019.2.x OnGUI Heap Allocation

Posted: 27 Dec 2019, 19:32
by stonstad
For Unity 2019.1.x and later versions...

"IMGUIContainer: If you have a lot of existing Editor UI written in IMGUI, you can use the special IMGUIContainer element to embed IMGUI UI within a UIElements UI as just another element. The IMGUIContainer takes a callback that serves as your OnGUI() loop, receiving all the events from outside as it normally would" (https://blogs.unity3d.com/2019/04/23/wh ... in-2019-1/).

API reference: https://docs.unity3d.com/ScriptReferenc ... ndler.html

Example:
 _IMGUIContainer.onGUIHandler = () => {
   Event.current...
};
Perhaps a checkbox or switch that toggles implementation?

Re: Unity 2019.2.x OnGUI Heap Allocation

Posted: 05 Jan 2020, 14:57
by jsantos
Interesting, thanks! But I would like to implement the new system without IMGUI at all. I didn't have time for this, but I want to figure out how Unity UI is getting events and do the same with Noesis. I assume the new UI is not using IMGUI at all.

Re: Unity 2019.2.x OnGUI Heap Allocation

Posted: 06 Jan 2020, 17:14
by stonstad
Makes sense... I wasn't sure about support requirements for older versions of Unity.


General Eventing
https://docs.unity3d.com/Packages/com.u ... ndler.html
private Event m_ProcessingEvent = new Event();
 
public virtual void OnUpdateSelected(BaseEventData eventData)
{
    while (Event.PopEvent(m_ProcessingEvent))
        if (m_ProcessingEvent.rawType == EventType.KeyDown)
        {
            //process event
        }

   
    eventData.Use();
}
 
Granular Eventing
https://docs.unity3d.com/Manual/SupportedEvents.html
public class Example: IEventSystemHandler, IPointerEnterHandler, IPointerExitHandler, IPointerDownHandler, IPointerUpHandler, IPointerClickHandler, IBeginDragHandler, IInitializePotentialDragHandler, IDragHandler, IEndDragHandler, IDropHandler, IScrollHandler, IUpdateSelectedHandler, ISelectHandler, IDeselectHandler, IMoveHandler, ISubmitHandler, ICancelHandler
    {
        public virtual void OnPointerEnter(PointerEventData eventData)
        {
        }

        public virtual void OnPointerExit(PointerEventData eventData)
        {
        }

        public virtual void OnDrag(PointerEventData eventData)
        {
        }

        public virtual void OnDrop(PointerEventData eventData)
        {
        }

        public virtual void OnPointerDown(PointerEventData eventData)
        {
        }

        public virtual void OnPointerUp(PointerEventData eventData)
        {
        }

        public virtual void OnPointerClick(PointerEventData eventData)
        {
        }

        public virtual void OnSelect(BaseEventData eventData)
        {
        }

        public virtual void OnDeselect(BaseEventData eventData)
        {
        }

        public virtual void OnScroll(PointerEventData eventData)
        {
        }

        public virtual void OnMove(AxisEventData eventData)
        {
        }

        public virtual void OnUpdateSelected(BaseEventData eventData)
        {
        }

        public virtual void OnInitializePotentialDrag(PointerEventData eventData)
        {
        }

        public virtual void OnBeginDrag(PointerEventData eventData)
        {
        }

        public virtual void OnEndDrag(PointerEventData eventData)
        {
        }

        public virtual void OnSubmit(BaseEventData eventData)
        {
        }

        public virtual void OnCancel(BaseEventData eventData)
        {
        }
}

Re: Unity 2019.2.x OnGUI Heap Allocation

Posted: 22 Sep 2020, 19:37
by stonstad
Hey Jesús. Just checking in on the status of this issue. I know it seems low priority, but it would be really awesome to have no per-frame allocations in Noesis. That's such a big deal for console devs. Thanks!