User avatar
stonstad
Topic Author
Posts: 241
Joined: 06 Jun 2016, 18:14
Location: Lesser Magellanic Cloud
Contact:

Background Effect and Texture Formats w/ HDR

16 May 2023, 15:38

I'm working with BackgroundEffect and as a first step, I am compositing my camera output to a single render texture. Initially, I used ARGB32 Linear, and this works with a Rectangle and TextureSource. Great, almost there, I'm thinking!

Unfortunately, the ARGB32 Linear format disables Unity HDR rendering, which removes many post-processing effects, such as Bloom. I can enable output of bloom to the render texture by using render texture format DefaultHDR Linear. However, when I use this HDR format, Noesis TextureSource fails to show the texture properly.

In Unity, what are my options for making Noesis BackgroundEffect work with a scene or RenderTexture containing HDR output?

Thanks,
Shaun
 
User avatar
jsantos
Site Admin
Posts: 3918
Joined: 20 Jan 2012, 17:18
Contact:

Re: Background Effect and Texture Formats w/ HDR

16 May 2023, 17:08

We don't support HDR textures (at least, they haven't been tested). Could you please report this including your issue with Background Effect? We have plans to support HDR in different parts of our pipeline.

Meanwhile, I think the better workaround is converting the HDR texture to SDR with a shader. Far from ideal but it will work (make sure to output the result in premultiplied alpha).
 
User avatar
stonstad
Topic Author
Posts: 241
Joined: 06 Jun 2016, 18:14
Location: Lesser Magellanic Cloud
Contact:

Re: Background Effect and Texture Formats w/ HDR

16 May 2023, 22:25

Thank you.

Feature request is here: https://www.noesisengine.com/bugs/view.php?id=2601

I'll look into HDR to SDR conversion via a shader.
 
User avatar
stonstad
Topic Author
Posts: 241
Joined: 06 Jun 2016, 18:14
Location: Lesser Magellanic Cloud
Contact:

Re: Background Effect and Texture Formats w/ HDR

16 May 2023, 22:42

I'm uncertain that HDR to SDR conversion is what I need. HDR to SDR exhibits the same behavior as routing camera render target output to a non-HDR texture. I think Unity strips HDR when the render target is SDR.

Or maybe this does work, but due to the gamma correction or Reinhard tone mapping, it just doesn't look great. OK, I have to ask about your crystal ball -- how far into the future might Noesis support a grab pass w/ HDR or TextureSource with native HDR? Three months or next major release in a year?
Shader "Custom/HDRtoSDR" {
	Properties{
		_MainTex("Texture", 2D) = "white" {}
	}
		SubShader{
			Tags { "RenderType" = "Opaque" }
			LOD 100

			Pass {
				CGPROGRAM
				#pragma vertex vert
				#pragma fragment frag

				#include "UnityCG.cginc"

				struct appdata {
					float4 vertex : POSITION;
					float2 uv : TEXCOORD0;
				};

				struct v2f {
					float2 uv : TEXCOORD0;
					float4 vertex : SV_POSITION;
				};

				sampler2D _MainTex;
				float4 _MainTex_ST;

				v2f vert(appdata v) {
					v2f o;
					o.vertex = UnityObjectToClipPos(v.vertex);
					o.uv = TRANSFORM_TEX(v.uv, _MainTex);
					return o;
				}

				fixed4 frag(v2f i) : SV_Target {
				
					// Sample the texture
				fixed4 col = tex2D(_MainTex, i.uv);

				// Apply Reinhard Tone Mapping
				col.rgb = col.rgb / (col.rgb + 1.0);

				// Gamma correction (assuming gamma = 2.2)
				col.rgb = pow(col.rgb, 1.0 / 2.2);

				// Premultiply alpha
				col.rgb *= col.a;

				return col;
			}
			ENDCG
		}
	}
}
using System;
using UnityEngine;
using UnityEngine.Rendering.PostProcessing;

[Serializable]
[PostProcess(typeof(HDRToSDRRenderer), PostProcessEvent.AfterStack, "Custom/HDRtoSDR")]
public sealed class HDRToSDR : PostProcessEffectSettings
{
}

public sealed class HDRToSDRRenderer : PostProcessEffectRenderer<HDRToSDR>
{
    public override void Render(PostProcessRenderContext context)
    {
        var sheet = context.propertySheets.Get(Shader.Find("Custom/HDRtoSDR"));
        context.command.BlitFullscreenTriangle(context.source, context.destination, sheet, 0);
    }
}
 
User avatar
jsantos
Site Admin
Posts: 3918
Joined: 20 Jan 2012, 17:18
Contact:

Re: Background Effect and Texture Formats w/ HDR

17 May 2023, 15:24

I'm uncertain that HDR to SDR conversion is what I need. HDR to SDR exhibits the same behavior as routing camera render target output to a non-HDR texture. I think Unity strips HDR when the render target is SDR.

Or maybe this does work, but due to the gamma correction or Reinhard tone mapping, it just doesn't look great. OK, I have to ask about your crystal ball -- how far into the future might Noesis support a grab pass w/ HDR or TextureSource with native HDR? Three months or next major release in a year?
Well, first we need to understand what's required here because we will face the same problems you are facing with your shader. If that's not working like you want, Noesis will produce a similar result (we can do the blur in HDR, but after that we need to convert to SDR).

So I think, it will be very helpful if you could recreate the result you need with a shader before we start analyzing how to implement this in Noesis.
 
User avatar
stonstad
Topic Author
Posts: 241
Joined: 06 Jun 2016, 18:14
Location: Lesser Magellanic Cloud
Contact:

Re: Background Effect and Texture Formats w/ HDR

17 May 2023, 17:16

That's fair. I'm trying to get Noesis BackgroundEffect working with Unity RenderTexture output when Unity Post-Processing is enabled.

Example 1 : Camera Backbuffer. Unity Post-Processing w/ HDR is enabled, but BackgroundEffect is disabled (as expected). Bloom exists around star and starship engines.
https://stellarconquest.blob.core.windo ... buffer.mp4

Example 2 : ARGB32 RenderTexture . Unity Post-Processing w/ HDR is disabled, but BackgroundEffect is enabled. Bloom is missing around star and starship engines.
https://stellarconquest.blob.core.windo ... exture.mp4

Example 3 : DefaultHDR RenderTexture. Unity Post-Processing w/ HDR is enabled, and BackgroundEffect is enabled. Output is incorrect.
https://stellarconquest.blob.core.windo ... exture.mp4
 
User avatar
stonstad
Topic Author
Posts: 241
Joined: 06 Jun 2016, 18:14
Location: Lesser Magellanic Cloud
Contact:

Re: Background Effect and Texture Formats w/ HDR

17 May 2023, 17:17

...Maybe I need to modify Unity's post-processing stack to premultiply alpha. I tried this with a post-processing effect (similar to above) but maybe that output is overwritten. I'll try premultiplying alpha via a camera OnRenderImage pass.

Update. This didn't work for me, but perhaps I am missing something.
 
User avatar
jsantos
Site Admin
Posts: 3918
Joined: 20 Jan 2012, 17:18
Contact:

Re: Background Effect and Texture Formats w/ HDR

17 May 2023, 18:46

Nice art by the way :)

I think you need a combination of 1) and 2)

So after postprocessing 1) you need to generate a ARGB32 RenderTexture using some kind of tone mapping (does Unity provide a way to do this out of the box?).

I assume this render target doesn't have pixels with alpha less than one, so fixing the premuliplied alpha won't do anything (as you observed).

In these cases, tools like RenderDoc helps a lot to understand what's going on with the GPU.
 
User avatar
stonstad
Topic Author
Posts: 241
Joined: 06 Jun 2016, 18:14
Location: Lesser Magellanic Cloud
Contact:

Re: Background Effect and Texture Formats w/ HDR

17 May 2023, 19:28

Unity exposes tonemapping features via the post-processing pipeline. If I change tonemapping from ACES to Neutral, or disable tonemapping, or set output to standard def, the bloom and dynamic range is lost. I don't know if I can remove HDR without disabling HDR effects in the post-processing pipeline. In the sample shader above, I attempt to remove HDR in a follow-on pass using Reinhard tone mapping.
col.rgb = col.rgb / (col.rgb + 1.0);

It doesn't seem to work, but most of this is over my head.

I'm guessing that the post-processing pipeline uses DefaultHDR as the underlying texture format. At some point in the overall Unity rendering pipeline, the HDR format is either maintained (if the monitor supports HDR) or it is tonemapped to non-HDR.

I can share the DefaultHDR output texture here -- it outputs to a RenderTexture with full fidelity. But when I send this to Noesis via TextureSource I get the blown-out effect.

I'm not sure what my next diagnostic step is.
 
User avatar
jsantos
Site Admin
Posts: 3918
Joined: 20 Jan 2012, 17:18
Contact:

Re: Background Effect and Texture Formats w/ HDR

18 May 2023, 10:41

I can share the DefaultHDR output texture here -- it outputs to a RenderTexture with full fidelity. But when I send this to Noesis via TextureSource I get the blown-out effect.
I think you are close to the expected result. You need to convert that texture to A8R8G8B8 and send it to Noesis.

Who is online

Users browsing this forum: No registered users and 25 guests