View Issue Details

Jump to NotesJump to History
IDProjectCategoryView StatusDate SubmittedLast Update
0001465NoesisGUIC++ SDKpublic2019-05-11 18:572019-11-26 19:53
Reportermingingmingler Assigned Tojsantos  
PrioritynormalSeverityfeature 
Status resolvedResolutionfixed 
Summary0001465: Allow 3D world-view-projection transforms in MVVM, independent of screen space for high performance spatial UI
Description

Hi, I've been dabbling with spatial UI in Noesis. I have a need to render many spatial elements at the same time, each with their own storyboards.

I first started doing it the render texture method. The performance of doing this isn't good, and the visual quality can suffer when the camera gets close to the element. So it's not desirable.

I decided it'd be far better to render the geometry directly into the world. So I tried something else - each spatial element now has its own view, and manages its own world view projection matrix. Before I tell Noesis to render each spatial view, I tell the grpahics device that I'm about to render a spatial element, and I pass it the element's world view projection. Now, when Noesis sets the buffers for this view, I override batch.projMtx directly with the new matrix.

The result is I get wonderful graphics quality, and the performance is better than using tons of render textures. But it's still not as good as it could be. Because each view is rendered separately, it doesn't take advantage of batching across all the views, and performance can suffer when each element has text in it.

I can't achieve what I want with the Noesis Projection attached property. Try as I might, I just can't work around the way Noesis handles a Matrix3DProjection behind the scenes.

So I believe the best solution would be, for example, to have a single Noesis scene/view, which has an ItemsControl bound to a collection of SpatialElement view models, which contain their own world-view-projection matrices. Then in the DataTemplate, we can assign this matrix to the root element, where Noesis would render the element (applying the transform directly to the vertices?) while ignoring any screen space projection that is applied behind the scenes. To be able to do this via MVVM would be incredibly powerful.

PlatformAny

Activities

jsantos

jsantos

2019-05-14 20:56

manager   ~0005675

you nailed this. We are working on this feature for the next major version. We are going to deprecate the current implementation of the Projection property in favor of having real 3D.
mingingmingler

mingingmingler

2019-05-15 01:09

reporter   ~0005677

That's exciting, thanks for getting back to me!
realesmedia

realesmedia

2019-05-15 14:32

reporter   ~0005678

Hi

in version 2.3 there will be support for 3D primitives (mesh, line ...)?

I want to implement 3D Gizmo (manipulators) on Noesis)))
jsantos

jsantos

2019-05-15 22:01

manager   ~0005679

We are working in a feature to inject 3D geometries inside a XAML. For example, to have a button that is a 3D-mesh. But the render must be done by the engine (Unity, Unreal, custom). That's something Noesis won't do, apart from calling a callback at the right time.
realesmedia

realesmedia

2019-05-16 14:26

reporter   ~0005682

ok
thank
jsantos

jsantos

2019-11-26 19:51

manager   ~0006012

Last edited: 2019-11-26 19:53

Many internal changes are coming in the next major version about this. Externally, the API has been improved to support setting a projection matrix to the view.



/// Sets the projection matrix that transforms the coordinates of each primitive to a

/// non-normalized homogeneous space where (0, 0) is at the lower-left corner and

/// (width, height) is at the upper-right cornet after projection

virtual void SetProjectionMatrix(const Matrix4& projection) = 0;

Also we added, this overload to IRenderer to allow rendering each eye in VR



/// Renders UI into the current render target and viewport. This function overrides the default

/// view projection matrix. Can be used for example to render each eye in VR

virtual void Render(const Matrix4& projection, bool flipY = false) = 0;

Each view, by default, is initialized to a perspective transform that behaves like the identity (like Noesis 2.2) at the near plane but includes perspective when elements are rotated in 3D using the new Transform3D property



<Rectangle>

<Rectangle.Transform3D>

<CompositeTransform3D RotateY="45"/>

</Rectangle.Transform3D>

</Rectangle>





<Rectangle>

<Rectangle.Transform3D>

<MatrixTransform3D Matrix="{Binding NodeMatrix}"/>

</Rectangle.Transform3D>

</Rectangle>

We also added support to rotate in XamlPlayer with ALT + right button drag.
We are preparing a document explaining all this.

Issue History

Date Modified Username Field Change
2019-05-11 18:57 mingingmingler New Issue
2019-05-11 19:03 mingingmingler Description Updated
2019-05-11 19:05 mingingmingler Description Updated
2019-05-11 19:06 mingingmingler Description Updated
2019-05-14 20:55 jsantos Assigned To => jsantos
2019-05-14 20:55 jsantos Status new => assigned
2019-05-14 20:55 jsantos Target Version => 2.3.0
2019-05-14 20:56 jsantos Note Added: 0005675
2019-05-15 01:09 mingingmingler Note Added: 0005677
2019-05-15 14:32 realesmedia Note Added: 0005678
2019-05-15 22:01 jsantos Note Added: 0005679
2019-05-16 14:26 realesmedia Note Added: 0005682
2019-11-26 19:51 jsantos Status assigned => resolved
2019-11-26 19:51 jsantos Resolution open => fixed
2019-11-26 19:51 jsantos Fixed in Version => 2.3.0
2019-11-26 19:51 jsantos Note Added: 0006012
2019-11-26 19:53 jsantos Note Edited: 0006012
2019-11-26 19:53 jsantos Note Edited: 0006012