VR interface

11 Mar 2016, 22:53

I've been following along with this tutorial on getting a UI working in VR: https://developer.oculus.com/blog/unity ... tem-in-vr/

It works with Unity's Canvas, but I can't get it working with NoesisGUI. I used a RenderTexture to map a xaml gui onto a quad in world space. I can see the gui, but I can't interact with it. I tried making the Noesis GUI Panel a child of a Canvas, but I still didn't get any mouse/touch events.

Has anyone gotten NoesisGUI working in VR?

thanks,
Phong

13 Mar 2016, 16:54

Hi!

Did you follow the instructions in the Render To Texture section of the Unity tutorial?

You need a mesh collider component in the destination object.

Please verify the steps and let us know if it works.

Thanks,

15 Mar 2016, 17:56

In addition to that info I want to point out that since 1.2.5f8 we have support to render inside uGUI using a RawImage component.

I have attached a simple sample demonstrating how to do it. First import NoesisGUI package, then import the sample. There is a Unity scene called uGUI you can load and is ready to play.

27 Apr 2016, 02:53

Thank you for your suggestions. I'm not sure why the forum didn't notify me of your responses (I have to subscribe to it, even if I started the thread?).

I figured this out the day after I posted my question, but to address the suggestions:

1. I had my gui rendered to a texture and the mesh had a collider, but no mouse events were passed to it. At the time, I could detect when the user was looking at my UI, but I didn't know how to figure out where within the UI that the user was looking.
2. uGUI renders in screen space, and VR puts everything into world space, so I can't use any of Unity's normal UI systems. UI in VR needs to be done with render textures and projected onto meshes.

The solution is to do a raycast from the user's "gaze" (center of the screen) to see where it hits within the mesh, then translate that into what the screen coordinates would be if the UI were in screen space. Then I call the appropriate mouse event methods on the NoesisGUIPanel to simulate the mouse events.

Thanks for designing NoesisGUI so that this was possible without too much effort!

28 Apr 2016, 10:00

Thanks for the kind words.

But regarding 1) and the solution you finally implemented, in theory is what we are already doing, sending a ray from mouse position and checking if it intersects with the textured game object (_target) where UI is being drawn:

// NOTE: A MeshCollider must be attached to the target to obtain valid
// texture coordintates, otherwise Hit Testing won't work

UnityEngine.Vector2 mouse = UnityEngine.Input.mousePosition;
UnityEngine.Ray ray = UnityEngine.Camera.main.ScreenPointToRay(
    new UnityEngine.Vector3(mouse.x, mouse.y, 0););

UnityEngine.RaycastHit hit;
if (UnityEngine.Physics.Raycast(ray, out hit))
{
    if (hit.collider.gameObject == _target)
    {
        float localX = _texture.width * hit.textureCoord.x;
        float localY = _texture.height * (1.0f - hit.textureCoord.y);

        //...
        Noesis_MouseMove(_rendererId, localX, localY);
    }
}

Maybe there is something that doesn't apply to your scene configuration. We would like to know if we can fix/improve our support for that kind of scenarios. Anything you can find would help, thank you.

28 Apr 2016, 10:05

Thank you for your suggestions. I'm not sure why the forum didn't notify me of your responses (I have to subscribe to it, even if I started the thread?).

Notifications are not sent by default. Whenever you post a reply or start a new topic you have the option to enable notifications by checking the "Notify me when a reply is posted" option.

28 Apr 2016, 15:35

But regarding 1) and the solution you finally implemented, in theory is what we are already doing, sending a ray from mouse position and checking if it intersects with the textured game object (_target) where UI is being drawn:

The issue is that there is no mouse in VR. If I run the VR project on Windows, then yes, the mouse does interact with the UI, as expected. However, when I run the project on my GearVR, there is no mouse cursor to move around. Essentially, I have to emulate a mouse using the center of the viewport as the cursor position.

(actually, the GearVR's touchpad acts as a mouse, but that is only useful to detect taps and swipes)

29 Apr 2016, 14:49

Perfect. That explains everything.

07 Dec 2016, 15:04

Hello! We start to develope VR-project with HTC Vive (Unity 3D ver. 5.5, Noesis 1.2.6f5) and faced same problem. Is there any way to emulate mouse events for Noesis UI objects or tricky hook to make Noesis UI objects interactive behaviour same as Unity 3D GUI objects beaviour?

07 Dec 2016, 15:25

Hello! We start to develope VR-project with HTC Vive (Unity 3D ver. 5.5, Noesis 1.2.6f5) and faced same problem. Is there any way to emulate mouse events for Noesis UI objects or tricky hook to make Noesis UI objects interactive behaviour same as Unity 3D GUI objects beaviour?

The solution I posted above will work for any Unity project, and should support any VR device (I've tested with the GearVR and Oculus Rift).

VR interface Topic is solved

VR interface

Re: VR interface

Re: VR interface

Re: VR interface

Re: VR interface

Re: VR interface

Re: VR interface

Re: VR interface

Re: VR interface

Re: VR interface

Who is online