Assert while restarting Noesis (trying to init after shutdown)

11 Nov 2021, 18:02

Hey :)
We're wondering what's the reason behind not being able to restart Noesis, i.e.: trying to init once it was shut down? We're obviously hitting this assert:

NS_CHECK(gKernelState != KState::ShutdownInvoked, "Invoking Init after Shutdown not allowed");

Our use case would benefit from being able to restart it. Currently we have to keep it running, which is not necessarily memory-efficient.

We're using Noesis 3.1.0 (10745 revision)

Thanks,
Szymon

12 Nov 2021, 11:30

For historical reasons we don't support restarting Noesis because the reflection architecture is using internally local static variables. We have plans to revisit this with the new reflection system we are working on.

If you destroy all views, objects and resources, memory usage should be almost insignificant. Please, give us more detail about your scenario in a ticket if this is important for your project.

12 Nov 2021, 14:12

Thanks @jsantos, that's what we suspected. Is there any ETA on the new reflection system?

We need the ability to attach/detach our SDK, which contains Noesis - e.g. right now it fails to restart Noesis in our Unreal and Unity projects (inside the editor when we hit play and stop few times). That's why we keep Noesis running, while shutting down the rest of our stuff. In the future we might also need the ability to have several instances that we can freely init and shut down.

12 Nov 2021, 17:18

Thanks @jsantos, that's what we suspected. Is there any ETA on the new reflection system?

Not yet, as this is a risky step that will break client code we need to think carefully all the implications. But basically we are working in a less intrusive and more efficient reflection architecture. In this direction we are not sure if we should also remove Init and Shutdown functions as they will no longer make sense.

We need the ability to attach/detach our SDK, which contains Noesis - e.g. right now it fails to restart Noesis in our Unreal and Unity projects (inside the editor when we hit play and stop few times). That's why we keep Noesis running, while shutting down the rest of our stuff. In the future we might also need the ability to have several instances that we can freely init and shut down.

Not calling Shutdown() doesn't mean Noesis is still running, you could even initialize Noesis at static time and perform Shutdown atexit time (a few clients are doing this AFAIK).

In the future we might also need the ability to have several instances that we can freely init and shut down

You mean several instances of your SDK right? In this case, considering a global initialization of Noesis at static time makes more sense. We are doing something similar for our Unity plugin.

12 Nov 2021, 18:35

Not calling Shutdown() doesn't mean Noesis is still running, you could even initialize Noesis at static time and perform Shutdown atexit time (a few clients are doing this AFAIK).

Could you please elaborate? Not sure if I follow.

I just did a simple test and called `Noesis::GetAllocatedMemory()` before shutting down Noesis and after it:

Before shutting down Noesis: 2646678 bytes, allocation count 159340
After shutting down Noesis:  0 bytes, allocation count 159390

Unsure how bad `2646678 bytes` is, but doesn't seem to be insignificant as you mentioned earlier. Maybe we could do something more that `Noesis::Shutdown()` does, without shutting it down entirely?

12 Nov 2021, 19:06

Could you please elaborate? Not sure if I follow.

Something like this:

bool InitNoesis()
{
    Gui::SetLogHandler(myLogHandler);
    Gui::SetMemoryCallbacks(myMemoryCallbacks);
    Gui::Noesis::Init();
    #ifdef TRACK_MEMORY
      atexit(Gui::Shutdown);
    #endif
}

Unsure how bad `2646678 bytes` is, but doesn't seem to be insignificant as you mentioned earlier. Maybe we could do something more that `Noesis::Shutdown()` does, without shutting it down entirely?

Yes, that's a lot. I was thinking on the hundreds of KB range, not MB. Could you please debug and tell me memories after each step at Shutdown?

void Noesis::Shutdown()
{
    if (gKernelState == KState::Initialized)
    {
        NS_LOG_INFO("Noesis Shutdown");
        UnregisterPackages();
        Boxing::Shutdown();
        SymbolManager::Shutdown();

15 Nov 2021, 20:09

Hey @jsantos,
sorry it took me a while to get back to you, it wasn't trivial to get the actual values. Finally I got them:

void Noesis::Shutdown()
{
    if (gKernelState == KState::Initialized)
    {
        NS_LOG_INFO("Noesis Shutdown");
        // Memory allocation: 2 721 711 bytes
        UnregisterPackages();
        // Memory allocation: 417 177 bytes
        Boxing::Shutdown();
        // Memory allocation: 188 512 bytes
        SymbolManager::Shutdown();
        // Memory allocation: 0 bytes
    }
}

Looks like unregistering the packages reduces the allocation by a lot. Is there anything we can do (call manually)?

15 Nov 2021, 20:40

I think that memory belongs to the global resources dictionary and more things we have in the UI module. Could you try setting to null everything you setup at init time from the IntegrationAPI header (SetApplicationResources, SetFontFallbacks, SetFontProvider, SetTextureProvider, etc).

Besides there is extra memory for the profiler, I assume this is a NS_PROFILE build right?

This probably needs a ticket to examine carefully.

15 Nov 2021, 22:28

We already set this to `nullptr`:

    Noesis::GUI::SetApplicationResources(nullptr);
    Noesis::GUI::SetSoftwareKeyboardCallback(nullptr, nullptr);
    Noesis::GUI::SetTextureProvider(nullptr);
    Noesis::GUI::SetXamlProvider(nullptr);
    Noesis::GUI::SetFontProvider(nullptr);

We also call `Reset()` for all of the `Noesis::Ptr` we use. We call `Noesis::UnregisterComponent` for all of the components, converters we register on init (btw. I verified that registering components does allocate memory, but unregistering doesn't clean it up).

Yes, it's a `NS_PROFILE` build.

16 Nov 2021, 19:25

It seems there are many things to improve and analyze here, please, create a ticket and let's continue the discussion there.

Assert while restarting Noesis (trying to init after shutdown)

Assert while restarting Noesis (trying to init after shutdown)

Re: Assert while restarting Noesis (trying to init after shutdown)

Re: Assert while restarting Noesis (trying to init after shutdown)

Re: Assert while restarting Noesis (trying to init after shutdown)

Re: Assert while restarting Noesis (trying to init after shutdown)

Re: Assert while restarting Noesis (trying to init after shutdown)

Re: Assert while restarting Noesis (trying to init after shutdown)

Re: Assert while restarting Noesis (trying to init after shutdown)

Re: Assert while restarting Noesis (trying to init after shutdown)

Re: Assert while restarting Noesis (trying to init after shutdown)

Who is online