Using Noesis to render a cursor with non-blocking mouse clicks

mingingmingler · 27 Feb 2023, 22:46

Hi, I'm using UE5.1 and want to use Noesis to render a cursor that is always visible on screen. I have achieved this by hooking the MouseMove event in my view, grabbing the mouse coordinates, then setting them to the cursor's view model. The root element also has a transparent background so that mouse input is detected across the whole screen. All good.

The problem is this completely blocks all mouse clicks to the game. I want mouse clicks over transparent pixels to be detected by the game, and only blocked when over a visible UI element. Think RTS style.

I have tried varying transparent/null backgrounds with IsHitTestVisible to no avail. I have, however, got this working by instead rendering the cursor with unreal's UMG system instead of Noesis, with the root element having a null background. I think this is a bit unfortunate... Is what I describe not possible with Noesis?

mingingmingler · 28 Feb 2023, 22:54

Nevermind, I got this working by making some changes to NoesisInstance.cpp.

First I made a duplicate of NoesisHitTestVisibleTester and the HitTest(FVector2D) function, called NoesisHitTestVisibleIgnoreTransparencyTester and HitTestIgnoreTransparency respectively.

Then in the new hit test struct, I changed the Filter function to the following

    Noesis::HitTestFilterBehavior Filter(Noesis::Visual* Visual)
    {
        Noesis::UIElement* Element = Noesis::DynamicCast<Noesis::UIElement*>(Visual);
        if (Element && (!Element->GetIsEnabled() || Element->GetOpacity() == 0 || !Element->GetIsHitTestVisible()))
        {
            return Noesis::HitTestFilterBehavior_ContinueSkipSelfAndChildren;
        }

	Noesis::Panel* Panel = Noesis::DynamicCast<Noesis::Panel*>(Visual);
        if (Panel && Panel->GetBackground() && Panel->GetBackground()->IsTransparent())
        {
            return Noesis::HitTestFilterBehavior_ContinueSkipSelf;
        }

        return Noesis::HitTestFilterBehavior_Continue;
    }

Then I looked for where the HitTest functions were called in this file, and changed all the mouse button ones to use the new IgnoreTransparency stuff. I left the MouseMove hit test alone.

01 Mar 2023, 11:56

Thanks for sharing your solution.

Anyway, I'm thinking you don't need to hook to the MouseMove UI event which requires to have a background element with a Transparent brush (the cause of your problem). You can use Unreal API instead to get mouse coordinates and update your viewmodel:

float mouseX;
float mouseY;
UGameplayStatics::GetPlayerController(GetWorld(), 0)->GetMousePosition(mouseX, mouseY);
UE_LOG(MyLog, Warning, TEXT("Mouse Location: %f, %f"), mouseX, mouseY);

Besides, rendering the mouse with the UI will tie its movement to the game framerate which could feel awkward, even more now with high-dpi mouse devices.

mingingmingler · 01 Mar 2023, 19:43

Right, in the end using hardware cursor is probably the right way to go, but it was a fun experiment!

Using Noesis to render a cursor with non-blocking mouse clicks Topic is solved

Using Noesis to render a cursor with non-blocking mouse clicks

Re: Using Noesis to render a cursor with non-blocking mouse clicks

Re: Using Noesis to render a cursor with non-blocking mouse clicks

Re: Using Noesis to render a cursor with non-blocking mouse clicks

Who is online