View Issue Details

IDProjectCategoryView StatusLast Update
0003050NoesisGUIUnrealpublic2024-06-05 17:37
ReporterCPVRGlen Assigned Tohcpizzi  
PrioritynormalSeverityminor 
Status resolvedResolutionfixed 
Product Version3.2.2 
Target Version3.2.4Fixed in Version3.2.4 
Summary0003050: NoesisWorldUI doesn't render in VR projects
Description

See https://www.noesisengine.com/forums/viewtopic.php?t=3195

Using UE 5.1.1, shader compilation often crashes when loading a VR project. With 5.2.2, if it does load, the NoesisWorldUI component fails to render anything in play.

I have also added this plugin to an existing 5.1.1 project and it didn't crash, but still wouldn't render. Assuming it works because the project settings are configured differently.

Steps To Reproduce

For the shader compilation crash:

  1. Create a new 5.1.1 VR template project
  2. Close UE after it initially loads
  3. Add the plugin
  4. Try loading the project again

For the rendering issue:

  1. Create anew 5.2.1 VR template project
  2. Close UE
  3. Add the plugin
  4. Load the project, confirm the plugin is enabled
  5. Add the sample ObjectPanel.xaml to a NoesisWorldUI in the level and play.
Attached Files
Screenshot 2024-01-25 172333.png (245,440 bytes)   
Screenshot 2024-01-25 172333.png (245,440 bytes)   
PlatformWindows

Activities

hcpizzi

hcpizzi

2024-02-21 17:18

developer   ~0009227

I'm sorry this ticket has been hanging for so long. It arrived as we were testing 3.2.3 for release and didn't have time to get back to it until recently.

I've attached a patch that fixes the rendeing issue. I was trying to get motion controller support in as well, but I thought I'd sent you what I have for now so you can test it.

Sorry for the late reply.

3050.patch (17,464 bytes)   
Index: Content/EnhancedInput/MotionControllerAction.uasset
===================================================================
Cannot display: file marked as a binary type.
svn:mime-type = application/octet-stream

Property changes on: Content/EnhancedInput/MotionControllerAction.uasset
___________________________________________________________________
Added: svn:mime-type
## -0,0 +1 ##
+application/octet-stream
\ No newline at end of property
Index: Source/NoesisEditor/Private/NoesisBlueprintCompilerContext.cpp
===================================================================
--- Source/NoesisEditor/Private/NoesisBlueprintCompilerContext.cpp	(revision 13421)
+++ Source/NoesisEditor/Private/NoesisBlueprintCompilerContext.cpp	(working copy)
@@ -62,6 +62,8 @@
 		DefaultInstance->EmulateTouch = NoesisBlueprint->EmulateTouch;
 		DefaultInstance->EnableTouch = NoesisBlueprint->EnableTouch;
 		DefaultInstance->EnableActions = NoesisBlueprint->EnableActions;
+		DefaultInstance->EnableMotionController = NoesisBlueprint->EnableMotionController;
+		DefaultInstance->MotionSource = NoesisBlueprint->MotionSource;
 		DefaultInstance->PixelDepthBias = NoesisBlueprint->PixelDepthBias;
 	}
 }
Index: Source/NoesisRuntime/Classes/NoesisBlueprint.h
===================================================================
--- Source/NoesisRuntime/Classes/NoesisBlueprint.h	(revision 13421)
+++ Source/NoesisRuntime/Classes/NoesisBlueprint.h	(working copy)
@@ -62,6 +62,12 @@
 	UPROPERTY(EditAnywhere, Category = "Noesis View")
 	bool EnableActions;
 
+	UPROPERTY(BlueprintReadWrite, Category = "Noesis View")
+	bool EnableMotionController;
+
+	UPROPERTY(BlueprintReadWrite, Category = "Noesis View")
+	FName MotionSource;
+
 	/** Set to a positive value, this property is added to the values returned by the PixelDepth node in a Material. */
 	UPROPERTY(EditAnywhere, Category = "Noesis View", meta = (DisplayName = "Material PixelDepth Bias"))
 	float PixelDepthBias;
Index: Source/NoesisRuntime/Classes/NoesisInstance.h
===================================================================
--- Source/NoesisRuntime/Classes/NoesisInstance.h	(revision 13421)
+++ Source/NoesisRuntime/Classes/NoesisInstance.h	(working copy)
@@ -137,6 +137,8 @@
 	bool IsGamepadSimulatedClick = false;
 	bool Is3DWidget = false;
 	mutable bool SupportsKeyboardFocus = true;
+	bool ValidCachedMotionControllerScreenPosition = false;
+	Noesis::Point CachedMotionControllerScreenPosition;
 
 	typedef TSharedPtr<class FNoesisSlateElement, ESPMode::ThreadSafe> FNoesisSlateElementPtr;
 	FNoesisSlateElementPtr NoesisSlateElement;
@@ -172,6 +174,12 @@
 	bool EnableActions;
 
 	UPROPERTY(BlueprintReadWrite, Category = "NoesisGUI")
+	bool EnableMotionController;
+
+	UPROPERTY(BlueprintReadWrite, Category = "NoesisGUI")
+	FName MotionSource;
+
+	UPROPERTY(BlueprintReadWrite, Category = "NoesisGUI")
 	float PixelDepthBias;
 
 	UFUNCTION(BlueprintCallable, Category = "NoesisGUI")
@@ -247,6 +255,8 @@
 	void OnEnhancedInputActionCompleted(Noesis::Key);
 	void OnEnhancedInputActionHorizontalScroll(const struct FInputActionValue&);
 	void OnEnhancedInputActionVerticalScroll(const struct FInputActionValue&);
+	void OnEnhancedInputActionMotionControllerActionTriggered();
+	void OnEnhancedInputActionMotionControllerActionCompleted();
 #endif
 
 	// UUserWidget interface
Index: Source/NoesisRuntime/NoesisRuntime.Build.cs
===================================================================
--- Source/NoesisRuntime/NoesisRuntime.Build.cs	(revision 13421)
+++ Source/NoesisRuntime/NoesisRuntime.Build.cs	(working copy)
@@ -67,7 +67,8 @@
 				"UElibPNG",
 				"Renderer",
 				"NetCore",
-				"Noesis"
+				"Noesis",
+				"HeadMountedDisplay"
 			}
 			);
 
Index: Source/NoesisRuntime/Private/NoesisBlueprint.cpp
===================================================================
--- Source/NoesisRuntime/Private/NoesisBlueprint.cpp	(revision 13421)
+++ Source/NoesisRuntime/Private/NoesisBlueprint.cpp	(working copy)
@@ -20,6 +20,8 @@
 	EmulateTouch = false;
 	EnableTouch = true;
 	EnableActions = false;
+	EnableMotionController = false;
+	MotionSource = "RightAim";
 	PixelDepthBias = -1.0f;
 }
 
Index: Source/NoesisRuntime/Private/NoesisInstance.cpp
===================================================================
--- Source/NoesisRuntime/Private/NoesisInstance.cpp	(revision 13421)
+++ Source/NoesisRuntime/Private/NoesisInstance.cpp	(working copy)
@@ -7,6 +7,7 @@
 
 // Core includes
 #include "MathUtil.h"
+#include "Features/IModularFeatures.h"
 #include "Stats/Stats.h"
 #include "Stats/Stats2.h"
 
@@ -26,7 +27,12 @@
 #else
 #include "Materials/MaterialRenderProxy.h"
 #endif
 
+// HeadMountedDisplay includes
+#include "IMotionController.h"
+#include "IXRTrackingSystem.h"
+
 // RHI includes
 #if UE_VERSION_OLDER_THAN(5, 2, 0)
 #include "RHIDefinitions.h"
@@ -265,8 +271,11 @@
 	ENQUEUE_RENDER_COMMAND(FNoesisInstance_DrawOffscreen)
 	(
 		[Renderer = Renderer, RenderDevice = RenderDevice,
-		WorldTime = WorldTime, Scene = Scene](FRHICommandListImmediate& RHICmdList)
+		WorldTime = WorldTime, Scene = Scene, ViewProjectionMatrix = ViewProjectionMatrix](FRHICommandListImmediate& RHICmdList)
 		{
+			/*if (fabs(ViewProjectionMatrix.Determinant()) < FLT_EPSILON)
+				return;*/
+
 			RenderOffscreen(Renderer, RenderDevice, WorldTime, Scene, RHICmdList);
 		}
 	);
@@ -312,6 +321,9 @@
 	if ((View->bIsMobileMultiViewEnabled || View->bIsInstancedStereoEnabled) && (View == View->GetInstancedSceneView()))
 		return;
 
+	if (fabs(ViewProjectionMatrix.Determinant()) < FLT_EPSILON)
+		return;
+
 	const auto ViewFamily = View->Family;
 
 	ViewRect = View->ViewRect;
@@ -504,6 +516,8 @@
 	EmulateTouch = false;
 	EnableTouch = true;
 	EnableActions = false;
+	EnableMotionController = false;
+	MotionSource = "RightAim";
 	PixelDepthBias = -1.0f;
 }
 
@@ -608,6 +622,17 @@
 					EnhancedInputComponent->BindAction(Action, ETriggerEvent::Triggered, this, Event.Value);
 				}
 			}
+
+			{
+				UInputAction* Action = LoadObject<UInputAction>(nullptr, TEXT("/NoesisGUI/EnhancedInput/MotionControllerAction"), nullptr, LOAD_NoWarn);
+
+				if (Action != nullptr)
+				{
+					EnhancedInputComponent->BindAction(Action, ETriggerEvent::Triggered, this, &UNoesisInstance::OnEnhancedInputActionMotionControllerActionTriggered);
+					EnhancedInputComponent->BindAction(Action, ETriggerEvent::Completed, this, &UNoesisInstance::OnEnhancedInputActionMotionControllerActionCompleted);
+					EnhancedInputComponent->BindAction(Action, ETriggerEvent::Canceled, this, &UNoesisInstance::OnEnhancedInputActionMotionControllerActionCompleted);
+				}
+			}
 		}
 #endif
 	}
@@ -737,6 +762,28 @@
 		XamlView->Scroll(-Value.Get<float>());
 	}
 }
+
+void UNoesisInstance::OnEnhancedInputActionMotionControllerActionTriggered()
+{
+	if (EnableActions && EnableMotionController && XamlView != nullptr)
+	{
+		if (ValidCachedMotionControllerScreenPosition)
+		{
+			XamlView->MouseButtonDown(CachedMotionControllerScreenPosition.x, CachedMotionControllerScreenPosition.y, Noesis::MouseButton_Left);
+		}
+	}
+}
+
+void UNoesisInstance::OnEnhancedInputActionMotionControllerActionCompleted()
+{
+	if (EnableActions && EnableMotionController && XamlView != nullptr)
+	{
+		if (ValidCachedMotionControllerScreenPosition)
+		{
+			XamlView->MouseButtonUp(CachedMotionControllerScreenPosition.x, CachedMotionControllerScreenPosition.y, Noesis::MouseButton_Left);
+		}
+	}
+}
 #endif
 
 void UNoesisInstance::ExecuteConsoleCommand(FString Command, class APlayerController* SpecificPlayer)
@@ -834,6 +881,8 @@
 	AddToRoot();
 	Is3DWidget = true;
 	EnablePPAA = true;
+	EnableActions = true;
+	EnableMotionController = true;
 	TessellationQuality = ENoesisTessellationQuality::High;
 	SetPlayerContext(FLocalPlayerContext(World->GetFirstLocalPlayerFromController(), World));
 	InitInstance();
@@ -924,6 +973,8 @@
 						const FMatrix ViewProjForCulling = ViewMatrixForCulling * StereoRenderingDevice->GetStereoProjectionMatrix(eSSE_MONOSCOPIC);
 
 						Noesis::Matrix4 ViewProj = UnrealToNoesisViewProj(ViewProjForCulling, Width, Height);
+						if (fabs(Noesis::Determinant(ViewProj)) < FLT_EPSILON)
+							return;
 
 						ENQUEUE_RENDER_COMMAND(CopyViewProj)
 						(
@@ -933,7 +984,72 @@
 							}
 						);
 						XamlView->SetProjectionMatrix(ViewProj);
+
+						if (EnableMotionController)
+						{
+							ValidCachedMotionControllerScreenPosition = false;
+							TArray<IMotionController*> MotionControllers = IModularFeatures::Get().GetModularFeatureImplementations<IMotionController>(IMotionController::GetModularFeatureName());
+							{
+								FTransform TrackingToWorld;
+								IXRTrackingSystem* TrackingSys = GEngine->XRSystem.Get();
+								if (TrackingSys)
+								{
+									TrackingToWorld = TrackingSys->GetTrackingToWorldTransform();
 								}
+
+								for (auto MotionController : MotionControllers)
+								{
+									if (MotionController == nullptr)
+									{
+										continue;
+									}
+
+									auto PlayerIndex = LocalPlayer->GetLocalPlayerIndex();
+									FRotator Orientation;
+									FVector Position;
+									if (MotionController->GetControllerOrientationAndPosition(PlayerIndex, MotionSource, Orientation, Position, WorldToMetersScale))
+									{
+										auto Direction = Orientation.Vector();
+										Direction = TrackingToWorld.TransformVector(Direction);
+										Position = TrackingToWorld.TransformPosition(Position);
+
+										Noesis::FrameworkElement* Root = XamlView->GetContent();
+										Noesis::Point3D NoesisPosition(-Position.Y, -Position.Z, -Position.X);
+										Noesis::Point3D NoesisDirection(-Direction.Y, -Direction.Z, -Direction.X);
+										Noesis::Point3D WorldPos;
+
+										auto Mouse = Root->GetMouse();
+										auto Captured = Mouse->GetCaptured();
+										if (Captured != nullptr)
+										{
+											//if (VisualTreeHelper::IntersectPlane(Captured, ...)
+										}
+										else
+										{
+											Noesis::HitTest3DResult HitTestResult = Noesis::VisualTreeHelper::HitTest3D(Root, NoesisPosition, NoesisDirection);
+											if (Noesis::Visual* Visual = HitTestResult.visualHit; Visual != nullptr)
+											{
+												ValidCachedMotionControllerScreenPosition = true;
+												WorldPos = HitTestResult.worldPos;
+											}
+										}
+										
+										if (ValidCachedMotionControllerScreenPosition)
+										{
+											auto ScreenPoint = Noesis::Vector4(WorldPos.x, WorldPos.y, WorldPos.z, 1.0f) * ViewProj;
+											CachedMotionControllerScreenPosition = Noesis::Point(ScreenPoint.x / ScreenPoint.w, ScreenPoint.y / ScreenPoint.w);
+											XamlView->MouseMove(CachedMotionControllerScreenPosition.x, CachedMotionControllerScreenPosition.y);
+										}
+									}
+								}
+							}
+						}
+					}
 					else
 					{
 						Noesis::Matrix4 ViewProj = UnrealToNoesisViewProj(ViewProjectionData.ComputeViewProjectionMatrix(), Width, Height);
@@ -1187,6 +1303,9 @@
 		(
 			[NoesisSlateElement = NoesisSlateElement](FRHICommandListImmediate& RHICmdList)
 			{
+				/*if (fabs(NoesisSlateElement->ViewProjectionMatrix.Determinant()) < FLT_EPSILON)
+					return;*/
+
 				auto Renderer = NoesisSlateElement->Renderer.GetPtr();
 				if (Renderer != nullptr)
 				{
@@ -1831,7 +1950,10 @@
 
 	InitInstance();
 
-	PreTickDelegateHandle = FSlateApplication::Get().OnPreTick().AddUObject(this, &UNoesisInstance::PreTick);
+	if (!Is3DWidget)
+	{
+		PreTickDelegateHandle = FSlateApplication::Get().OnPreTick().AddUObject(this, &UNoesisInstance::PreTick);
+	}
 }
 
 void UNoesisInstance::NativeDestruct()
@@ -1920,13 +2042,54 @@
 	return !RendererInfo.IsMobileShadingPath || RendererInfo.IsMobileDeferredShadingPath;
 }
 
+static void AddWorldUIRenderPass(FRDGBuilder& GraphBuilder, FRDGTexture* ColorTexture, FRDGTexture* DepthStencilTexture, const FViewInfo& View)
+{
+	FRenderTargetParameters* PassParameters = GraphBuilder.AllocParameters<FRenderTargetParameters>();
+	{
+		PassParameters->RenderTargets[0] = FRenderTargetBinding(ColorTexture, ERenderTargetLoadAction::ELoad);
+		PassParameters->RenderTargets.DepthStencil = FDepthStencilBinding(DepthStencilTexture, ERenderTargetLoadAction::ELoad, ERenderTargetLoadAction::EClear, FExclusiveDepthStencil::DepthRead_StencilWrite);
+	}
+	GraphBuilder.AddPass(RDG_EVENT_NAME("NoesisTranslucentPass"), PassParameters, ERDGPassFlags::Raster | ERDGPassFlags::NeverCull,
+		[&View](FRHICommandListImmediate& RHICmdList)
+		{
+			for (auto NoesisSlateElement : GNoesis3DSlateElements)
+			{
+				FNoesisRenderDevice* RenderDevice = NoesisSlateElement->RenderDevice;
+				RenderDevice->IsWorldUI = true;
+				NoesisSlateElement->RenderView(RHICmdList, (FViewInfo*)&View);
+				RenderDevice->IsWorldUI = false;
+			}
+		}
+	);
+}
+
 void NoesisDrawRenderThreadOverlay(FPostOpaqueRenderParameters& Params)
 {
+	const auto& View = (const FViewInfo&)*Params.View;
+	const auto ViewFamily = View.Family;
+
+	if (ViewFamily->Scene->IsEditorScene())
+		return;
+
+	if (GNoesis3DSlateElements.Num() == 0)
+		return;
+
+	if (ViewFamily->AllowTranslucencyAfterDOF())
+		return;
+
+	auto ColorTexture = Params.ColorTexture;
+	auto DepthStencilTexture = Params.DepthTexture;
+
+	if (ColorTexture == nullptr || DepthStencilTexture == nullptr)
+		return;
+
+	auto& GraphBuilder = *Params.GraphBuilder;
+	AddWorldUIRenderPass(GraphBuilder, ColorTexture, DepthStencilTexture, View);
 }
 
 FDelegateHandle NoesisRegisterOverlayRender()
 {
-	return GetRendererModule().RegisterPostOpaqueRenderDelegate(FPostOpaqueRenderDelegate::CreateStatic(&NoesisDrawRenderThreadOverlay));
+	return GetRendererModule().RegisterOverlayRenderDelegate(FPostOpaqueRenderDelegate::CreateStatic(&NoesisDrawRenderThreadOverlay));
 }
 
 void NoesisUnregisterOverlayRender(FDelegateHandle InOverlayRenderDelegateHandle)
@@ -1987,6 +2150,7 @@
 	virtual void PrePostProcessPass_RenderThread(FRDGBuilder& GraphBuilder, const FSceneView& InView, const FPostProcessingInputs& Inputs) override
 	{
 		const auto& View = (const FViewInfo&)InView;
+		const auto ViewFamily = View.Family;
 
 #if UE_VERSION_OLDER_THAN(5, 1, 0)
 		if (!View.bIsSceneCapture)
@@ -1998,7 +2162,7 @@
 			GViewProjectionMatrix = View.ViewMatrices.GetProjectionMatrix();
 		}
 
-		if (View.Family->Scene->IsEditorScene())
+		if (ViewFamily->Scene->IsEditorScene())
 			return;
 
 		if (GNoesis3DSlateElements.Num() == 0)
@@ -2012,23 +2176,7 @@
 		if (ColorTexture == nullptr || DepthStencilTexture == nullptr)
 			return;
 
-		FRenderTargetParameters* PassParameters = GraphBuilder.AllocParameters<FRenderTargetParameters>();
-		{
-			PassParameters->RenderTargets[0] = FRenderTargetBinding(ColorTexture, ERenderTargetLoadAction::ELoad);
-			PassParameters->RenderTargets.DepthStencil = FDepthStencilBinding(DepthStencilTexture, ERenderTargetLoadAction::ELoad, ERenderTargetLoadAction::EClear, FExclusiveDepthStencil::DepthRead_StencilWrite);
-		}
-		GraphBuilder.AddPass(RDG_EVENT_NAME("NoesisTranslucentPass"), PassParameters, ERDGPassFlags::Raster | ERDGPassFlags::NeverCull,
-			[&View](FRHICommandListImmediate& RHICmdList)
-			{
-				for (auto NoesisSlateElement : GNoesis3DSlateElements)
-				{
-					FNoesisRenderDevice* RenderDevice = NoesisSlateElement->RenderDevice;
-					RenderDevice->IsWorldUI = true;
-					NoesisSlateElement->RenderView(RHICmdList, (FViewInfo*)&View);
-					RenderDevice->IsWorldUI = false;
-				}
-			}
-		);
+		AddWorldUIRenderPass(GraphBuilder, ColorTexture, DepthStencilTexture, View);
 	}
 
 	virtual void SubscribeToPostProcessingPass(EPostProcessingPass Pass, FAfterPassCallbackDelegateArray& InOutPassCallbacks, bool bIsPassEnabled) override
@@ -2080,28 +2228,18 @@
 	virtual void PostRenderBasePassMobile_RenderThread(FRHICommandList& RHICmdList, FSceneView& InView) override
 #endif
 	{
-		check(InView.bIsViewInfo);
-		const auto& View = (const FViewInfo&)InView;
-
-#if UE_VERSION_OLDER_THAN(5, 1, 0)
-		if (!View.bIsSceneCapture)
-#else
-		if (!View.bIsSceneCapture && !View.bIsSceneCaptureCube)
-#endif
-		{
-			GViewRect = View.ViewRect;
-			GViewProjectionMatrix = View.ViewMatrices.GetProjectionMatrix();
-		}
-
 		if (GNoesis3DSlateElements.Num() == 0)
 			return;
 
-		FRendererInfo RendererInfo = GetRendererInfo(&View);
+		check(InView.bIsViewInfo);
+		const FViewInfo* View = (FViewInfo*)&InView;
+
+		FRendererInfo RendererInfo = GetRendererInfo(View);
 		if (Render3DPostOpaque(RendererInfo))
 			return;
 
 		// We need to clear the stencil buffer, since we're in the SceneColor pass
-		const auto& ViewRect = View.ViewRect;
+		const auto& ViewRect = View->ViewRect;
 		auto Left = ViewRect.Min.X;
 		auto Top = ViewRect.Min.Y;
 		auto Right = ViewRect.Max.X;
@@ -2110,7 +2248,7 @@
 		DrawClearQuad(RHICmdList, false, FLinearColor(), false, 0.f, true, 0);
 		for (auto NoesisSlateElement : GNoesis3DSlateElements)
 		{
-			NoesisSlateElement->RenderView(RHICmdList, (FViewInfo*)&View);
+			NoesisSlateElement->RenderView(RHICmdList, View);
 		}
 	}
 
3050.patch (17,464 bytes)   
hcpizzi

hcpizzi

2024-02-27 11:34

developer   ~0009240

This fixes the shader compiler crashes on 5.1, while also improving some aspects of WorldUI stereo rendering and fixing some edge cases.

WorldUIStereoFixes.patch (15,086 bytes)   
Index: Unreal/Plugins/NoesisGUI/Shaders/Private/NoesisVS.usf
===================================================================
--- Unreal/Plugins/NoesisGUI/Shaders/Private/NoesisVS.usf	(revision 13463)
+++ Unreal/Plugins/NoesisGUI/Shaders/Private/NoesisVS.usf	(revision 13464)
@@ -1,5 +1,6 @@
 #include "/Engine/Private/Common.ush"
 #include "/Engine/Private/GammaCorrectionCommon.ush"
+#include "/Engine/Private/InstancedStereo.ush"
 
 struct In
 {
@@ -85,7 +86,11 @@
 
 #if WITH_STEREO
 #if INSTANCED_STEREO
+#if MULTI_VIEW
     nointerpolation uint viewportIndex : SV_ViewPortArrayIndex;
+#else
+    float clipDistance : SV_ClipDistance;
+#endif
 #elif MOBILE_MULTI_VIEW
     nointerpolation uint viewId : VIEW_ID;
 #endif
@@ -97,11 +102,17 @@
 {
 #if WITH_STEREO
 #if INSTANCED_STEREO
+    o.position = mul(NoesisVSConstantsStereo.projectionMtx[i.instanceId], float4(i.position, 0, 1));
+#if MULTI_VIEW
     o.viewportIndex = i.instanceId;
-    o.position = mul(i.instanceId == 0 ? NoesisVSConstants.projectionMtx : NoesisVSConstantsRightEye.projectionMtx, float4(i.position, 0, 1));
+#else
+    o.clipDistance = dot(o.position, EyeClipEdge[i.instanceId]);
+    o.position.x *= 0.5/* * ResolvedView.HMDEyePaddingOffset*/;
+    o.position.x += (EyeOffsetScale[i.instanceId] * o.position.w) * (1.0f - 0.5/* * ResolvedView.HMDEyePaddingOffset*/);
+#endif
 #elif MOBILE_MULTI_VIEW
     o.viewId = i.viewId;
-    o.position = mul(i.viewId == 0 ? NoesisVSConstants.projectionMtx : NoesisVSConstantsRightEye.projectionMtx, float4(i.position, 0, 1));
+    o.position = mul(NoesisVSConstantsStereo.projectionMtx[i.viewId], float4(i.position, 0, 1));
 #else
     o.position = mul(NoesisVSConstants.projectionMtx, float4(i.position, 0, 1));
 #endif
Index: Unreal/Plugins/NoesisGUI/Source/NoesisRuntime/Private/NoesisInstance.cpp
===================================================================
--- Unreal/Plugins/NoesisGUI/Source/NoesisRuntime/Private/NoesisInstance.cpp	(revision 13463)
+++ Unreal/Plugins/NoesisGUI/Source/NoesisRuntime/Private/NoesisInstance.cpp	(revision 13464)
@@ -108,7 +108,7 @@
 // Only to be modified from the render thread. Enqueue any operation needed to the render thread.
 TArray<FNoesisSlateElement*> GNoesis3DSlateElements;
 
-Noesis::Matrix4 UnrealToNoesisViewProj(const FMatrix& UnrealViewProj, float Width, float Height)
+static Noesis::Matrix4 UnrealToNoesisViewProj(const FMatrix& UnrealViewProj, float Width, float Height)
 {
 	/// ViewProj = BaseChange * UnrealViewProj * Scale * Trans * Invert, where
 	/// BaseChange = Matrix4(
@@ -312,6 +312,9 @@
 	if ((View->bIsMobileMultiViewEnabled || View->bIsInstancedStereoEnabled) && (View == View->GetInstancedSceneView()))
 		return;
 
+	if (fabs(ViewProjectionMatrix.Determinant()) < FLT_EPSILON)
+		return;
+
 	const auto ViewFamily = View->Family;
 
 	ViewRect = View->ViewRect;
@@ -323,30 +326,55 @@
 	WorldTime = ViewFamily->Time;
 	IsMobileMultiView = View->bIsMobileMultiViewEnabled && View->StereoPass == EStereoscopicPass::eSSP_PRIMARY;
 	IsInstancedStereo = View->bIsInstancedStereoEnabled && View->StereoPass == EStereoscopicPass::eSSP_PRIMARY;
-	RHICmdList.SetViewport(Left, Top, 0.0f, Right, Bottom, 1.0f);
 	if (IsMobileMultiView || IsInstancedStereo)
 	{
-		ViewProj = UnrealToNoesisViewProj(ViewProjectionMatrix, Right - Left, Bottom - Top);
+#if UE_VERSION_OLDER_THAN(5, 1, 0)
+		if (!View->bIsMultiViewEnabled)
+		{
+			ViewProj = UnrealToNoesisViewProj(ViewProjectionMatrix, Right - Left, Bottom - Top);
 
-		auto LeftEyeViewProjectionMatrix = View->ViewMatrices.GetViewProjectionMatrix();
-		LeftEyeViewProj = UnrealToNoesisViewProj(LeftEyeViewProjectionMatrix, Right - Left, Bottom - Top);
+			auto LeftEyeViewProjectionMatrix = View->ViewMatrices.GetViewProjectionMatrix();
+			LeftEyeViewProj = UnrealToNoesisViewProj(LeftEyeViewProjectionMatrix, Right - Left, Bottom - Top);
 
-		auto InstancedView = (FViewInfo*)View->GetInstancedSceneView();
-		const auto& InstancedViewRect = InstancedView->ViewRect;
-		float InstancedLeft = InstancedViewRect.Min.X;
-		float InstancedTop = InstancedViewRect.Min.Y;
-		float InstancedRight = InstancedViewRect.Max.X;
-		float InstancedBottom = InstancedViewRect.Max.Y;
+			auto InstancedView = (FViewInfo*)View->GetInstancedSceneView();
+			const auto& InstancedViewRect = InstancedView->ViewRect;
+			float InstancedLeft = InstancedViewRect.Min.X;
+			float InstancedTop = InstancedViewRect.Min.Y;
+			float InstancedRight = InstancedViewRect.Max.X;
+			float InstancedBottom = InstancedViewRect.Max.Y;
 
-		auto RightEyeViewProjectionMatrix = InstancedView->ViewMatrices.GetViewProjectionMatrix();
-		RightEyeViewProj = UnrealToNoesisViewProj(RightEyeViewProjectionMatrix, InstancedRight - InstancedLeft, InstancedBottom - InstancedTop);
+			auto RightEyeViewProjectionMatrix = InstancedView->ViewMatrices.GetViewProjectionMatrix();
+			RightEyeViewProj = UnrealToNoesisViewProj(RightEyeViewProjectionMatrix, InstancedRight - InstancedLeft, InstancedBottom - InstancedTop);
 
-		RHICmdList.SetStereoViewport(Left, InstancedLeft, Top, InstancedTop, 0.0f, Right, InstancedRight, Bottom, InstancedBottom, 1.0f);
+			RHICmdList.SetViewport(Left, Top, 0.0f, View->InstancedStereoWidth, Bottom, 1.0f);
+		}
+		else
+#endif
+		{
+			ViewProj = UnrealToNoesisViewProj(ViewProjectionMatrix, Right - Left, Bottom - Top);
+
+			auto LeftEyeViewProjectionMatrix = View->ViewMatrices.GetViewProjectionMatrix();
+			LeftEyeViewProj = UnrealToNoesisViewProj(LeftEyeViewProjectionMatrix, Right - Left, Bottom - Top);
+
+			auto InstancedView = (FViewInfo*)View->GetInstancedSceneView();
+			const auto& InstancedViewRect = InstancedView->ViewRect;
+			float InstancedLeft = InstancedViewRect.Min.X;
+			float InstancedTop = InstancedViewRect.Min.Y;
+			float InstancedRight = InstancedViewRect.Max.X;
+			float InstancedBottom = InstancedViewRect.Max.Y;
+
+			auto RightEyeViewProjectionMatrix = InstancedView->ViewMatrices.GetViewProjectionMatrix();
+			RightEyeViewProj = UnrealToNoesisViewProj(RightEyeViewProjectionMatrix, InstancedRight - InstancedLeft, InstancedBottom - InstancedTop);
+
+			RHICmdList.SetStereoViewport(Left, InstancedLeft, Top, InstancedTop, 0.0f, Right, InstancedRight, Bottom, InstancedBottom, 1.0f);
+		}
 	}
 	else
 	{
 		auto MonoViewProjectionMatrix = View->ViewMatrices.GetViewProjectionMatrix();
 		ViewProj = UnrealToNoesisViewProj(MonoViewProjectionMatrix, Right - Left, Bottom - Top);
+
+		RHICmdList.SetViewport(Left, Top, 0.0f, Right, Bottom, 1.0f);
 	}
 	RenderOnscreen(RHICmdList, true);
 }
@@ -1831,7 +1859,10 @@
 
 	InitInstance();
 
-	PreTickDelegateHandle = FSlateApplication::Get().OnPreTick().AddUObject(this, &UNoesisInstance::PreTick);
+	if (!Is3DWidget)
+	{
+		PreTickDelegateHandle = FSlateApplication::Get().OnPreTick().AddUObject(this, &UNoesisInstance::PreTick);
+	}
 }
 
 void UNoesisInstance::NativeDestruct()
Index: Unreal/Plugins/NoesisGUI/Source/NoesisRuntime/Private/Render/NoesisRenderDevice.cpp
===================================================================
--- Unreal/Plugins/NoesisGUI/Source/NoesisRuntime/Private/Render/NoesisRenderDevice.cpp	(revision 13463)
+++ Unreal/Plugins/NoesisGUI/Source/NoesisRuntime/Private/Render/NoesisRenderDevice.cpp	(revision 13464)
@@ -828,7 +828,7 @@
 	NOESIS_BIND_DEBUG_BUFFER_LABEL(DynamicIndexBuffer, IBName);
 
 	VSConstantBuffer = TUniformBufferRef<FNoesisVSConstants>::CreateUniformBufferImmediate(FNoesisVSConstants(), UniformBuffer_MultiFrame);
-	VSConstantBufferRightEye = TUniformBufferRef<FNoesisVSConstantsRightEye>::CreateUniformBufferImmediate(FNoesisVSConstantsRightEye(), UniformBuffer_MultiFrame);
+	VSConstantBufferStereo = TUniformBufferRef<FNoesisVSConstantsStereo>::CreateUniformBufferImmediate(FNoesisVSConstantsStereo(), UniformBuffer_MultiFrame);
 	TextureSizeBuffer = TUniformBufferRef<FNoesisTextureSize>::CreateUniformBufferImmediate(FNoesisTextureSize(), UniformBuffer_MultiFrame);
 	PSRgbaConstantBuffer = TUniformBufferRef<FNoesisPSRgbaConstants>::CreateUniformBufferImmediate(FNoesisPSRgbaConstants(), UniformBuffer_MultiFrame);
 	PSOpacityConstantBuffer = TUniformBufferRef<FNoesisPSOpacityConstants>::CreateUniformBufferImmediate(FNoesisPSOpacityConstants(), UniformBuffer_MultiFrame);
@@ -1488,7 +1488,7 @@
 	DynamicVertexBuffer.SafeRelease();
 	DynamicIndexBuffer.SafeRelease();
 	VSConstantBuffer.SafeRelease();
-	VSConstantBufferRightEye.SafeRelease();
+	VSConstantBufferStereo.SafeRelease();
 	TextureSizeBuffer.SafeRelease();
 	PSRgbaConstantBuffer.SafeRelease();
 	PSOpacityConstantBuffer.SafeRelease();
@@ -2510,28 +2510,15 @@
 			NumInstances = 2;
 		}
 
-		const Noesis::UniformData& UniformData = Batch.vertexUniforms[0];
-		if (BufferNeedsUpdate(VSConstantsHash, UniformData))
-		{
-			uint32 UniformDataSize = UniformData.numDwords * 4 / 2;
-			const uint8* UniformDataValues = (const uint8 *)UniformData.values;
-			UpdateUniformBuffer(RHICmdList, VSConstantBuffer, UniformDataSize, UniformDataValues);
-
-			UniformDataValues += UniformDataSize;
-			UpdateUniformBuffer(RHICmdList, VSConstantBufferRightEye, UniformDataSize, UniformDataValues);
-
-			VSConstantsHash = UniformData.hash;
-		}
-
-		VertexShader->SetVSConstantsRightEye(*RHICmdList, VSConstantBufferRightEye);
+		ConditionalUpdateUniformBuffer(RHICmdList, VSConstantBufferStereo, VSConstantsHash, Batch.vertexUniforms[0]);
+		VertexShader->SetVSConstantsStereo(*RHICmdList, VSConstantBufferStereo);
 	}
 	else
 	{
 		ConditionalUpdateUniformBuffer(RHICmdList, VSConstantBuffer, VSConstantsHash, Batch.vertexUniforms[0]);
+		VertexShader->SetVSConstants(*RHICmdList, VSConstantBuffer);
 	}
 
-	VertexShader->SetVSConstants(*RHICmdList, VSConstantBuffer);
-
 	ConditionalUpdateUniformBuffer(RHICmdList, TextureSizeBuffer, TextureSizeHash, Batch.vertexUniforms[1]);
 
 	ConditionalUpdateUniformBuffer(RHICmdList, PSUniformBuffer0, PSUniformBuffer0Hash, Batch.pixelUniforms[0]);
Index: Unreal/Plugins/NoesisGUI/Source/NoesisRuntime/Private/Render/NoesisRenderDevice.h
===================================================================
--- Unreal/Plugins/NoesisGUI/Source/NoesisRuntime/Private/Render/NoesisRenderDevice.h	(revision 13463)
+++ Unreal/Plugins/NoesisGUI/Source/NoesisRuntime/Private/Render/NoesisRenderDevice.h	(revision 13464)
@@ -50,7 +50,7 @@
 	FBufferRHIRef DynamicIndexBuffer;
 #endif
 	FUniformBufferRHIRef VSConstantBuffer;
-	FUniformBufferRHIRef VSConstantBufferRightEye;
+	FUniformBufferRHIRef VSConstantBufferStereo;
 	FUniformBufferRHIRef TextureSizeBuffer;
 	FUniformBufferRHIRef PSRgbaConstantBuffer;
 	FUniformBufferRHIRef PSOpacityConstantBuffer;
Index: Unreal/Plugins/NoesisGUI/Source/NoesisRuntime/Private/Render/NoesisShaders.cpp
===================================================================
--- Unreal/Plugins/NoesisGUI/Source/NoesisRuntime/Private/Render/NoesisShaders.cpp	(revision 13463)
+++ Unreal/Plugins/NoesisGUI/Source/NoesisRuntime/Private/Render/NoesisShaders.cpp	(revision 13464)
@@ -6,7 +6,7 @@
 #include "NoesisShaders.h"
 
 IMPLEMENT_GLOBAL_SHADER_PARAMETER_STRUCT(FNoesisVSConstants, "NoesisVSConstants");
-IMPLEMENT_GLOBAL_SHADER_PARAMETER_STRUCT(FNoesisVSConstantsRightEye, "NoesisVSConstantsRightEye");
+IMPLEMENT_GLOBAL_SHADER_PARAMETER_STRUCT(FNoesisVSConstantsStereo, "NoesisVSConstantsStereo");
 IMPLEMENT_GLOBAL_SHADER_PARAMETER_STRUCT(FNoesisTextureSize, "NoesisTextureSize");
 IMPLEMENT_GLOBAL_SHADER_PARAMETER_STRUCT(FNoesisPSRgbaConstants, "NoesisPSRgbaConstants");
 IMPLEMENT_GLOBAL_SHADER_PARAMETER_STRUCT(FNoesisPSOpacityConstants, "NoesisPSOpacityConstants");
Index: Unreal/Plugins/NoesisGUI/Source/NoesisRuntime/Private/Render/NoesisShaders.h
===================================================================
--- Unreal/Plugins/NoesisGUI/Source/NoesisRuntime/Private/Render/NoesisShaders.h	(revision 13463)
+++ Unreal/Plugins/NoesisGUI/Source/NoesisRuntime/Private/Render/NoesisShaders.h	(revision 13464)
@@ -135,8 +135,8 @@
 SHADER_PARAMETER(FMatrix44f, projectionMtx)
 END_GLOBAL_SHADER_PARAMETER_STRUCT()
 
-BEGIN_GLOBAL_SHADER_PARAMETER_STRUCT(FNoesisVSConstantsRightEye, )
-SHADER_PARAMETER(FMatrix44f, projectionMtx)
+BEGIN_GLOBAL_SHADER_PARAMETER_STRUCT(FNoesisVSConstantsStereo, )
+SHADER_PARAMETER_ARRAY(FMatrix44f, projectionMtx, [2])
 END_GLOBAL_SHADER_PARAMETER_STRUCT()
 
 BEGIN_GLOBAL_SHADER_PARAMETER_STRUCT(FNoesisTextureSize, )
@@ -174,11 +174,11 @@
 	{
 #if UE_VERSION_OLDER_THAN(5, 3, 0)
 		VSConstantsBuffer.Bind(Initializer.ParameterMap, FNoesisVSConstants::StaticStructMetadata.GetShaderVariableName());
-		VSConstantsBufferRightEye.Bind(Initializer.ParameterMap, FNoesisVSConstantsRightEye::StaticStructMetadata.GetShaderVariableName());
+		VSConstantsBufferStereo.Bind(Initializer.ParameterMap, FNoesisVSConstantsStereo::StaticStructMetadata.GetShaderVariableName());
 		TextureSizeBuffer.Bind(Initializer.ParameterMap, FNoesisTextureSize::StaticStructMetadata.GetShaderVariableName());
 #else
 		VSConstantsBuffer.Bind(Initializer.ParameterMap, FNoesisVSConstants::GetStructMetadata()->GetShaderVariableName());
-		VSConstantsBufferRightEye.Bind(Initializer.ParameterMap, FNoesisVSConstantsRightEye::GetStructMetadata()->GetShaderVariableName());
+		VSConstantsBufferStereo.Bind(Initializer.ParameterMap, FNoesisVSConstantsStereo::GetStructMetadata()->GetShaderVariableName());
 		TextureSizeBuffer.Bind(Initializer.ParameterMap, FNoesisTextureSize::GetStructMetadata()->GetShaderVariableName());
 #endif
 	}
@@ -203,16 +203,16 @@
 #endif
 	}
 
-	void SetVSConstantsRightEye(FRHICommandList& RHICmdList, const FUniformBufferRHIRef& VSConstantsRightEye)
+	void SetVSConstantsStereo(FRHICommandList& RHICmdList, const FUniformBufferRHIRef& VSConstantsStereo)
 	{
 		FRHIVertexShader* ShaderRHI = RHICmdList.GetBoundVertexShader();
 
-		check(VSConstantsBufferRightEye.IsBound());
+		check(VSConstantsBufferStereo.IsBound());
 #if UE_VERSION_OLDER_THAN(5, 3, 0)
-		SetUniformBufferParameter(RHICmdList, ShaderRHI, VSConstantsBufferRightEye, VSConstantsRightEye);
+		SetUniformBufferParameter(RHICmdList, ShaderRHI, VSConstantsBufferStereo, VSConstantsStereo);
 #else
 		FRHIBatchedShaderParameters& BatchedParameters = RHICmdList.GetScratchShaderParameters();
-		BatchedParameters.SetShaderUniformBuffer(VSConstantsBufferRightEye.GetBaseIndex(), VSConstantsRightEye);
+		BatchedParameters.SetShaderUniformBuffer(VSConstantsBufferStereo.GetBaseIndex(), VSConstantsStereo);
 		RHICmdList.SetBatchedShaderParameters(ShaderRHI, BatchedParameters);
 #endif
 	}
@@ -232,7 +232,7 @@
 	}
 
 	LAYOUT_FIELD(FShaderUniformBufferParameter, VSConstantsBuffer);
-	LAYOUT_FIELD(FShaderUniformBufferParameter, VSConstantsBufferRightEye);
+	LAYOUT_FIELD(FShaderUniformBufferParameter, VSConstantsBufferStereo);
 	LAYOUT_FIELD(FShaderUniformBufferParameter, TextureSizeBuffer)
 };
 
WorldUIStereoFixes.patch (15,086 bytes)   

Issue History

Date Modified Username Field Change
2024-01-29 10:49 CPVRGlen New Issue
2024-01-29 10:49 CPVRGlen Tag Attached: Rendering
2024-01-29 10:49 CPVRGlen Tag Attached: Unreal
2024-01-29 10:49 CPVRGlen Tag Attached: world ui
2024-01-29 10:49 CPVRGlen File Added: Screenshot 2024-01-25 172333.png
2024-01-29 11:08 jsantos Assigned To => hcpizzi
2024-01-29 11:08 jsantos Status new => assigned
2024-01-29 11:08 jsantos Target Version => 3.2.4
2024-02-21 17:18 hcpizzi Note Added: 0009227
2024-02-21 17:18 hcpizzi File Added: 3050.patch
2024-02-27 11:34 hcpizzi Note Added: 0009240
2024-02-27 11:34 hcpizzi File Added: WorldUIStereoFixes.patch
2024-02-27 11:34 hcpizzi Status assigned => feedback
2024-06-05 17:37 jsantos Status feedback => resolved
2024-06-05 17:37 jsantos Resolution open => fixed
2024-06-05 17:37 jsantos Fixed in Version => 3.2.4