User avatar
stonstad
Topic Author
Posts: 131
Joined: 06 Jun 2016, 18:14
Location: Lesser Magellanic Cloud
Contact:

Unity 2019.2.x OnGUI Heap Allocation

06 Dec 2019, 17:01

The Unity MonoBehavior function OnGUI has a 0.7 kb per-frame gc allocation. Unity does not intend to fix this, as OnGUI is deprecated, per this thread: https://forum.unity.com/threads/gc-coll ... ui.642229/

NoesisView.cs uses OnGUI as follows:
    void OnGUI()
    {
        if (_uiView != null)
        {
            UnityEngine.GUI.depth = -(int)_myCamera.depth;
            ProcessEvent(UnityEngine.Event.current, _enableKeyboard, _enableMouse, _emulateTouch);
        }
    }
I can confirm the following:
1. OnGUI leaks memory without the internal Noesis logic.
2. Commenting out OnGUI completely removes the 0.7 kb per frame allocation.

To eliminate this allocation, is there a different location this Noesis logic may be moved to?

Image

Tags:
 
User avatar
jsantos
Site Admin
Posts: 2900
Joined: 20 Jan 2012, 17:18
Contact:

Re: Unity 2019.2.x OnGUI Heap Allocation

12 Dec 2019, 01:02

Thanks for taking the time to report this (#1596). This is definitely something we need to investigate because I am not sure how we should get the input events if we are not using OnGUI. I assume this would be using the new UI, but the fact that there is a new UI coming soon makes me wonder if this is the right moment to do this change. Also, using OnGUI we needed to workaround many things to avoid weird bugs in Unity. I assume if we change to something new this is going to happen again taking more time than expected. But yes, thanks!
 
User avatar
stonstad
Topic Author
Posts: 131
Joined: 06 Jun 2016, 18:14
Location: Lesser Magellanic Cloud
Contact:

Re: Unity 2019.2.x OnGUI Heap Allocation

12 Dec 2019, 04:18

A few thoughts around this ... I'm honestly not keen to release my game to production with a 60 KB per second GC allocation. I believe it will cause issues on XBOX. I totally agree that this is a Unity bug... and unfortunately, the Unity team isn't likely to fix these IMGUI GC allocations, not now, not ever. IMGUI was replaced in 2015 by Unity UI and five years is ancient history to Unity developers Moreover, OnGUI() is broken by design because it allocates heap objects for eventing -- several hundred or thousand a second in some scenarios.

So what's the viable alternative to OnGUI? Since you asked... One approach is to use Input.GetMouseDown and Input.GetKeyboard. It is doable -- I'm sharing a working proof of concept here. I need to modify it further so that it handles multiple input events simultaneously, but for now, this expresses the concept and it works in my game.

* I edited my post to be more agreeable. I was tired and my initial thoughts didn't translate well.

1. Comment Out
//void OnGUI()
    //{
    //    if (_uiView != null)
    //    {
    //        UnityEngine.GUI.depth = -(int)_myCamera.depth;
    //        ProcessEvent(UnityEngine.Event.current, _enableKeyboard, _enableMouse, _emulateTouch);
    //    }
    //}
2. Add:
   
   private void Update()
    {
        if (_uiView != null)
            ProcessEvent2(_enableKeyboard, _enableMouse, _emulateTouch);
    }
3. Add
private void ProcessEvent2(bool enableKeyboard, bool enableMouse, bool emulateTouch)
    {
        // Process keyboard modifiers
        if (enableKeyboard)
        {
            EventModifiers currentModifiers = EventModifiers.None;
            if (Input.GetKey(KeyCode.LeftShift))
                currentModifiers |= EventModifiers.Shift;
            if (Input.GetKey(KeyCode.LeftControl))
                currentModifiers |= EventModifiers.Control;

            EventModifiers delta = currentModifiers ^ _modifiers;
            if (delta > 0)
            {
                _modifiers = currentModifiers;

                ProcessModifierKey(currentModifiers, delta, EventModifiers.Shift, Key.LeftShift);
                ProcessModifierKey(currentModifiers, delta, EventModifiers.Control, Key.LeftCtrl);
                ProcessModifierKey(currentModifiers, delta, EventModifiers.Command, Key.LeftCtrl);
                ProcessModifierKey(currentModifiers, delta, EventModifiers.Alt, Key.LeftAlt);
            }
        }

        EventType eventType = EventType.Ignore;

        if (Input.GetMouseButtonDown(0) || Input.GetMouseButtonDown(1) || Input.GetMouseButtonDown(2))
            eventType = EventType.MouseDown;
        else if (Input.GetMouseButtonUp(0) || Input.GetMouseButtonUp(1) || Input.GetMouseButtonUp(2)) // naieve approach given multiple inputs may fire simultaneously
            eventType = EventType.MouseUp;
        else if (Input.anyKeyDown)
            eventType = EventType.KeyDown;

        switch (eventType)
        {
            case UnityEngine.EventType.MouseDown:
                {
                    if (enableMouse)
                    {
                        //UnityEngine.Vector2 mouse = ProjectPointer(Input.mousePosition.x, UnityEngine.Screen.height - Input.mousePosition.y);
                        UnityEngine.Vector2 mouse = new Vector2(Input.mousePosition.x, UnityEngine.Screen.height - Input.mousePosition.y);

                        //if (HitTest(mouse.x, mouse.y))
                        //{
                        //    ev.Use();
                        //}

                        if (emulateTouch)
                        {
                            _uiView.TouchDown((int)mouse.x, (int)mouse.y, 0);
                            _touchEmulated = true;
                        }
                        else
                        {
                            // Ignore events generated by Unity to simulate a mouse down when a touch event occurs
                            bool mouseEmulated = Input.simulateMouseWithTouches && Input.touchCount > 0;
                            if (!mouseEmulated)
                            {
                                //if (ev.clickCount == 1)
                                //{
                                if (Input.GetMouseButtonDown(0))
                                    _uiView.MouseButtonDown((int)mouse.x, (int)mouse.y, MouseButton.Left);
                                else if (Input.GetMouseButtonDown(1))
                                    _uiView.MouseButtonDown((int)mouse.x, (int)mouse.y, MouseButton.Right);
                                else if (Input.GetMouseButtonDown(2))
                                    _uiView.MouseButtonDown((int)mouse.x, (int)mouse.y, MouseButton.Middle);
                                //}
                                
                                // TODO: store click count, if truly needed
                                //else 
                                //{
                                //_uiView.MouseDoubleClick((int)mouse.x, (int)mouse.y, (Noesis.MouseButton)ev.button);
                                //}
                            }
                        }
                    }
                    break;
                }
            case UnityEngine.EventType.MouseUp:
                {
                    if (enableMouse)
                    {
                        //UnityEngine.Vector2 mouse = ProjectPointer(Input.mousePosition.x, UnityEngine.Screen.height - Input.mousePosition.y);
                        UnityEngine.Vector2 mouse = new Vector2(Input.mousePosition.x, UnityEngine.Screen.height - Input.mousePosition.y);

                        //if (HitTest(mouse.x, mouse.y))
                        //{
                        //    ev.Use();
                        //}

                        if (emulateTouch && _touchEmulated)
                        {
                            _uiView.TouchUp((int)mouse.x, (int)mouse.y, 0);
                            _touchEmulated = false;
                        }
                        else
                        {
                            // Ignore events generated by Unity to simulate a mouse up when a touch event occurs
                            bool mouseEmulated = Input.simulateMouseWithTouches && Input.touchCount > 0;
                            if (!mouseEmulated)
                            {
                                if (Input.GetMouseButtonUp(0))
                                    _uiView.MouseButtonUp((int)mouse.x, (int)mouse.y, MouseButton.Left);
                                else if (Input.GetMouseButtonUp(1))
                                    _uiView.MouseButtonUp((int)mouse.x, (int)mouse.y, MouseButton.Right);
                                else if (Input.GetMouseButtonUp(2))
                                    _uiView.MouseButtonUp((int)mouse.x, (int)mouse.y, MouseButton.Middle);
                            }
                        }
                    }
                    break;
                }
            case UnityEngine.EventType.KeyDown:
                {
                    if (enableKeyboard)
                    {
                        foreach (var kvp in NoesisKeyCodes._unityToNoesis)
                        {
                            if (Input.GetKeyDown(kvp.Key))
                                _uiView.KeyDown(kvp.Value);

                        }

                    }
                    break;
                }
            case UnityEngine.EventType.KeyUp:
                {
                    //// Don't process key when IME composition is being used
                    //if (enableKeyboard)
                    //{
                    //    if (ev.keyCode != KeyCode.None && NoesisUnity.IME.compositionString == "")
                    //    {
                    //        Noesis.Key noesisKeyCode = NoesisKeyCodes.Convert(ev.keyCode);
                    //        if (noesisKeyCode != Noesis.Key.None)
                    //        {
                    //            _uiView.KeyUp(noesisKeyCode);
                    //        }
                    //    }
                    //}
                    break;
                }
        }
    }
 
User avatar
stonstad
Topic Author
Posts: 131
Joined: 06 Jun 2016, 18:14
Location: Lesser Magellanic Cloud
Contact:

Re: Unity 2019.2.x OnGUI Heap Allocation

12 Dec 2019, 13:51

To handle KeyUp I'm thinking of an enum and bitwise OR for deltas. Would still have to enumerate through all keys ... but a non-allocating iteration to from 0 to 100 per frame is no big deal. I'd be happy to develop this further and submit as a PR for consideration. I won't be offended if it isn't used. Right now I have two changes for NoesisView that I'm merging on version upgrades. 1) The exception GC allocation behavior for unimplemented inputs, and this behavior. Just trying to minimize heap allocations to avoid GC frame stutter.

These two allocations in NoesisView are literally the last two ("every frame") allocations I have in my game.
 
User avatar
stonstad
Topic Author
Posts: 131
Joined: 06 Jun 2016, 18:14
Location: Lesser Magellanic Cloud
Contact:

Re: Unity 2019.2.x OnGUI Heap Allocation

12 Dec 2019, 14:39

https://docs.unity3d.com/Manual/GUIScri ... 1570476412
The Immediate Mode GUI system is commonly used for:

Creating in-game debugging displays and tools.
Creating custom inspectors
for script components.
Creating new editor windows and tools to extend Unity itself.

The IMGUI system is not generally intended to be used for normal in-game user interfaces that players might use and interact with.
 
User avatar
jsantos
Site Admin
Posts: 2900
Joined: 20 Jan 2012, 17:18
Contact:

Re: Unity 2019.2.x OnGUI Heap Allocation

16 Dec 2019, 19:12

When we started NoesisGUI, OnGUI was the only way to get events. And that was, yeah, long time ago 😃

I would like to find a better way than polling the events for each key. I am sure UnityUI is doing it in a different way and the source code is public so I will try to find time to analyze this soon.

As always, thanks for pushing this, we need to get rid of those GC allocs.
 
User avatar
stonstad
Topic Author
Posts: 131
Joined: 06 Jun 2016, 18:14
Location: Lesser Magellanic Cloud
Contact:

Re: Unity 2019.2.x OnGUI Heap Allocation

27 Dec 2019, 19:32

For Unity 2019.1.x and later versions...

"IMGUIContainer: If you have a lot of existing Editor UI written in IMGUI, you can use the special IMGUIContainer element to embed IMGUI UI within a UIElements UI as just another element. The IMGUIContainer takes a callback that serves as your OnGUI() loop, receiving all the events from outside as it normally would" (https://blogs.unity3d.com/2019/04/23/wh ... in-2019-1/).

API reference: https://docs.unity3d.com/ScriptReferenc ... ndler.html

Example:
 _IMGUIContainer.onGUIHandler = () => {
   Event.current...
};
Perhaps a checkbox or switch that toggles implementation?
 
User avatar
jsantos
Site Admin
Posts: 2900
Joined: 20 Jan 2012, 17:18
Contact:

Re: Unity 2019.2.x OnGUI Heap Allocation

05 Jan 2020, 14:57

Interesting, thanks! But I would like to implement the new system without IMGUI at all. I didn't have time for this, but I want to figure out how Unity UI is getting events and do the same with Noesis. I assume the new UI is not using IMGUI at all.
 
User avatar
stonstad
Topic Author
Posts: 131
Joined: 06 Jun 2016, 18:14
Location: Lesser Magellanic Cloud
Contact:

Re: Unity 2019.2.x OnGUI Heap Allocation

06 Jan 2020, 17:14

Makes sense... I wasn't sure about support requirements for older versions of Unity.


General Eventing
https://docs.unity3d.com/Packages/com.u ... ndler.html
private Event m_ProcessingEvent = new Event();
 
public virtual void OnUpdateSelected(BaseEventData eventData)
{
    while (Event.PopEvent(m_ProcessingEvent))
        if (m_ProcessingEvent.rawType == EventType.KeyDown)
        {
            //process event
        }

   
    eventData.Use();
}
 
Granular Eventing
https://docs.unity3d.com/Manual/SupportedEvents.html
public class Example: IEventSystemHandler, IPointerEnterHandler, IPointerExitHandler, IPointerDownHandler, IPointerUpHandler, IPointerClickHandler, IBeginDragHandler, IInitializePotentialDragHandler, IDragHandler, IEndDragHandler, IDropHandler, IScrollHandler, IUpdateSelectedHandler, ISelectHandler, IDeselectHandler, IMoveHandler, ISubmitHandler, ICancelHandler
    {
        public virtual void OnPointerEnter(PointerEventData eventData)
        {
        }

        public virtual void OnPointerExit(PointerEventData eventData)
        {
        }

        public virtual void OnDrag(PointerEventData eventData)
        {
        }

        public virtual void OnDrop(PointerEventData eventData)
        {
        }

        public virtual void OnPointerDown(PointerEventData eventData)
        {
        }

        public virtual void OnPointerUp(PointerEventData eventData)
        {
        }

        public virtual void OnPointerClick(PointerEventData eventData)
        {
        }

        public virtual void OnSelect(BaseEventData eventData)
        {
        }

        public virtual void OnDeselect(BaseEventData eventData)
        {
        }

        public virtual void OnScroll(PointerEventData eventData)
        {
        }

        public virtual void OnMove(AxisEventData eventData)
        {
        }

        public virtual void OnUpdateSelected(BaseEventData eventData)
        {
        }

        public virtual void OnInitializePotentialDrag(PointerEventData eventData)
        {
        }

        public virtual void OnBeginDrag(PointerEventData eventData)
        {
        }

        public virtual void OnEndDrag(PointerEventData eventData)
        {
        }

        public virtual void OnSubmit(BaseEventData eventData)
        {
        }

        public virtual void OnCancel(BaseEventData eventData)
        {
        }
}

Who is online

Users browsing this forum: ivan.fuertes and 1 guest