Page 1 of 1

Two Issues with Noesis GUI - Blocking OnMouseDown Events and Image Binding

Posted: 30 Sep 2024, 06:12
by hendawi
Hi,

I would like to report two issues I encountered while using Noesis GUI with Unity.

Issue 1: <Path> with IsHitTestVisible="False" blocks OnMouseDown events in Unity
Problem Description:
I have a world-space UI displayed above buildings. Some elements, like <Path>, are used for design purposes and are set to IsHitTestVisible="False".

I expect that when IsHitTestVisible="False" is set, events should be passed through to the 3D elements (buildings) underneath the UI. However, this is not the case. Despite the element being set as non-interactive, it still blocks the OnMouseDown event in Unity.

I used VisualTreeHelper.HitTest to verify if the UI elements were consuming the click events, and it shows that the <Path> element consumes the clicks even though IsHitTestVisible="False" is set.
Test Code:
<Path Stroke="#90000009" StrokeThickness="30" Fill="Transparent" IsHitTestVisible="False" Data="{Binding BackgroundGeometry}" />
VisualTreeHelper.HitTest Output:
[World-Space UI] Mouse is over a UI element: Noesis.Path

Physics.Raycast Output
[Unity Raycast] Building clicked!

OnMouseDown() not called
Expected Behavior:
I expect that when IsHitTestVisible="False" is set on a <Path> element, all click events are passed through to the underlying 3D elements (like buildings).

Issue 2: Image Binding Issue

Problem Description:
When using an <Image> element with a static image source, the image displays correctly. For example, the following code works as expected:
<Image Source="/Assets/View/GamePlay/build-all.png" 
       Width="20" Height="20" 
       HorizontalAlignment="Center" 
       VerticalAlignment="Center" 
       Opacity="0.7"/>
However, when I try to bind the image source using an Icon property (Binding), the image does not display, even though the bound value for Icon is exactly the same as the static path. Here is the code that does not work:
<Image Source="{Binding Icon}"
       Width="20" Height="20" 
       HorizontalAlignment="Center" 
       VerticalAlignment="Center" 
       Opacity="0.7"/>
The image is not found when using the binding, despite the value being correct..
[NOESIS] Image not found '/Assets/View/GamePlay/build-all.png'

Interestingly, if I place both the static image and the bound image together in the XAML, both images display correctly. This suggests that the binding itself is valid, but there might be an issue with loading the image solely through binding.
<Image Source="/Assets/View/GamePlay/build-all.png" 
       Width="20" Height="20" 
       HorizontalAlignment="Center" 
       VerticalAlignment="Center" 
       Opacity="0.7"/>

<Image Source="{Binding Icon}"
       Width="20" Height="20" 
       HorizontalAlignment="Center" 
       VerticalAlignment="Center" 
       Opacity="0.7"/>
Expected Behavior:
I expect that binding the image source using {Binding Icon} should load and display the image independently, just like when providing a static path.


thank you .

Re: Two Issues with Noesis GUI - Blocking OnMouseDown Events and Image Binding

Posted: 30 Sep 2024, 06:19
by hendawi
Hi,

I would like to add a quick clarification regarding the issue I previously reported about <Path> with IsHitTestVisible="False". I have noticed that the same behavior also occurs with the <Border> element. Even when IsHitTestVisible="False" is set on a <Border>, it still blocks click events (like OnMouseDown) from reaching the 3D elements beneath the UI.

I believe this is a general issue with how Noesis handles non-interactive elements (IsHitTestVisible="False").

Thank you.

Re: Two Issues with Noesis GUI - Blocking OnMouseDown Events and Image Binding

Posted: 02 Oct 2024, 11:09
by sfernandez
Hi,

1. IsHitTestVisible has a bit weird behavior in WPF, because normal calls to VisualTreeHelper.HitTest() will return objects with that property set to False, and Noesis is following that approach too. An alternative is to use the HitTest() method with callbacks, and filter the elements that have IsHitTestVisible set to False, like I explained here: https://www.noesisengine.com/bugs/view. ... 3476#c9787, but we need to think if we want to do that by default in Unity to consume input events.

2. This is happening because when you use only a Binding to the image, the corresponding Texture asset is not referenced by any element of the scene. You can add explicit dependencies in any xaml by using this:
<UserControl
  xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
  xmlns:noesis="clr-namespace:NoesisGUIExtensions;assembly=Noesis.GUI.Extensions">

  <noesis:Xaml.Dependencies>
    <noesis:Dependency Source="Language-en.xaml"/>
    <noesis:Dependency Source="Language-fr.xaml"/>
    <noesis:Dependency Source="Language-jp.xaml"/>
  </noesis:Xaml.Dependencies>

</UserControl>

Re: Two Issues with Noesis GUI - Blocking OnMouseDown Events and Image Binding

Posted: 02 Oct 2024, 13:00
by hendawi
Hi,

1. IsHitTestVisible has a bit weird behavior in WPF, because normal calls to VisualTreeHelper.HitTest() will return objects with that property set to False, and Noesis is following that approach too. An alternative is to use the HitTest() method with callbacks, and filter the elements that have IsHitTestVisible set to False, like I explained here: https://www.noesisengine.com/bugs/view. ... 3476#c9787, but we need to think if we want to do that by default in Unity to consume input events.

2. This is happening because when you use only a Binding to the image, the corresponding Texture asset is not referenced by any element of the scene. You can add explicit dependencies in any xaml by using this:
<UserControl
  xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
  xmlns:noesis="clr-namespace:NoesisGUIExtensions;assembly=Noesis.GUI.Extensions">

  <noesis:Xaml.Dependencies>
    <noesis:Dependency Source="Language-en.xaml"/>
    <noesis:Dependency Source="Language-fr.xaml"/>
    <noesis:Dependency Source="Language-jp.xaml"/>
  </noesis:Xaml.Dependencies>

</UserControl>
Thanks for help .

1 - The proposed code seems to solve the first problem
https://www.noesisengine.com/bugs/view. ... 3476#c9787
Will this code be included in any future updates or do I have to do this myself in every update?


2 - I still don't understand how to do it in my case

Suppose I get data from a web server when I start the game

The web server returns the paths according to what is involved in the game.

What is the best way to deal with this?

Thank you .

Re: Two Issues with Noesis GUI - Blocking OnMouseDown Events and Image Binding

Posted: 02 Oct 2024, 14:10
by jsantos
1 - The proposed code seems to solve the first problem
https://www.noesisengine.com/bugs/view. ... 3476#c9787
Will this code be included in any future updates or do I have to do this myself in every update?
Please, file a ticket about this and we will think about it.
2 - I still don't understand how to do it in my case
Suppose I get data from a web server when I start the game
For the scenario you were describing in your first post, you need to maintain a hard-reference to the texture somewhere. This is a requirement of the Unity asset system, we offer an extension to inject manual dependencies to XAML as Sergio described, but you can use any Unity native way (for example, have the reference in a ScriptableObject).

Downloading data from a web server is a different problem and depending on what you need, you probably want to use AssetBundles for that.

Re: Two Issues with Noesis GUI - Blocking OnMouseDown Events and Image Binding

Posted: 02 Oct 2024, 17:51
by hendawi
For the scenario you were describing in your first post, you need to maintain a hard-reference to the texture somewhere. This is a requirement of the Unity asset system, we offer an extension to inject manual dependencies to XAML as Sergio described, but you can use any Unity native way (for example, have the reference in a ScriptableObject).

Downloading data from a web server is a different problem and depending on what you need, you probably want to use AssetBundles for that.

Thanks for the reply

I did not mean to download the image from the server.

The images are already in the game. The server just sends the path

As I explained {Binding Icon}
The Icon value here is string of the image path in the game like "/Assets/View/GamePlay/build-all.png"

Let's assume that I have a list of images that I would like to use when needed as Binding with image path value .

like this
public class MyViewModel : INotifyPropertyChanged
{
    private string _icon;
    public string Icon
    {
        get { return _icon; }
        set
        {
            _icon = value;
            OnPropertyChanged(nameof(Icon));
        }
    }

    public MyViewModel()
    {
        Icon = "/Assets/View/GamePlay/build-all.png";
    }
    
    ...
}


<Image Source="{Binding Icon}" />

Can you explain an example of doing this?



Thank you

Re: Two Issues with Noesis GUI - Blocking OnMouseDown Events and Image Binding

Posted: 03 Oct 2024, 11:49
by jsantos
Let's assume that I have a list of images that I would like to use when needed as Binding with image path value .
If these images are not directly referenced in any XAML (as appears to be the case here), you'll need to manually inject their dependencies (as described above) for each one.

Re: Two Issues with Noesis GUI - Blocking OnMouseDown Events and Image Binding

Posted: 05 Oct 2024, 13:02
by hendawi
Let's assume that I have a list of images that I would like to use when needed as Binding with image path value .
If these images are not directly referenced in any XAML (as appears to be the case here), you'll need to manually inject their dependencies (as described above) for each one.
I didn't get it

The example you are referring to is about translation

i doesn't know how I can make "/Assets/View/GamePlay/build-all.png" usable by Binding

Can you explain via code?

Re: Two Issues with Noesis GUI - Blocking OnMouseDown Events and Image Binding

Posted: 05 Oct 2024, 17:06
by jsantos
Add this line to the root of your XAML:
  <noesis:Xaml.Dependencies>
    <noesis:Dependency Source="/Assets/View/GamePlay/build-all.png"/>
  </noesis:Xaml.Dependencies>
And your binding will start to work.

Re: Two Issues with Noesis GUI - Blocking OnMouseDown Events and Image Binding

Posted: 05 Oct 2024, 17:21
by hendawi
Add this line to the root of your XAML:
  <noesis:Xaml.Dependencies>
    <noesis:Dependency Source="/Assets/View/GamePlay/build-all.png"/>
  </noesis:Xaml.Dependencies>
And your binding will start to work.

Thank you 🙏

I thought <noesis:Dependency> only dealt with xaml . This makes it very easy

Thank you. 🙏🙏🙏