nokola
Topic Author
Posts: 188
Joined: 10 Mar 2015, 05:29

NoesisGUI (Unity) on iOS Simulator?

19 Mar 2017, 08:43

I've been trying for the last couple of days to run our app on iOS Simulator (iPhone 7) to take Apple-required high-res screenshots to submit to the Apple App Store, and failed.
Using Noesis 2 in Unity.

On iOS I can choose Metal, OpenGL ES 2, or ES 3 for my app.
But:
1. the Simulator does not support Metal
2. Unity does not support ES 2 properly anymore: https://issuetracker.unity3d.com/issues ... 2#comments
3. when I run OpenGL ES 3 on Simulator I get this exception in xCode that references Noesis:
dyld: lazy symbol binding failed: Symbol not found: _SWIGRegisterExceptionCallbacks_NoesisGUI_
Referenced from: /Users/guest/Library/Developer/CoreSimulator/Devices/E543C2C9-E02A-468F-AD26-7E058821FC3B/data/Containers/Bundle/Application/443A5B8E-3FB8-480B-AFC1-E3B747AA6837/sculptorios.app/sculptorios
Expected in: flat namespace

My app runs with Noesis 2 + Metal on iPhone 5s no problem.
How can I use Noesis in the Simulator or take high-res screenshots?
 
nokola
Topic Author
Posts: 188
Joined: 10 Mar 2015, 05:29

Re: NoesisGUI (Unity) on iOS Simulator?

19 Mar 2017, 19:44

Figured out how to take high-res screenshots on PC https://www.assetstore.unity3d.com/en/#!/content/24122
Does Noesis support Simulator?
 
User avatar
jsantos
Site Admin
Posts: 2900
Joined: 20 Jan 2012, 17:18
Contact:

Re: NoesisGUI (Unity) on iOS Simulator?

20 Mar 2017, 04:51

iOS simulator is not supported. It is not a real technical problem but it enlarges the final size of the package and also complicates our QA process. And I would say you are the second person asking for it. Can't you take the screenshots from the PC?
 
nokola
Topic Author
Posts: 188
Joined: 10 Mar 2015, 05:29

Re: NoesisGUI (Unity) on iOS Simulator?

20 Mar 2017, 05:33

Just tried from PC and Noesis crashes Unity Editor:
Noesis 2
Unity Editor [version: Unity 5.5.2f1_3829d7f588f3]

Noesis.dll caused an Access Violation (0xc0000005)
in module Noesis.dll at 0033:ca271ad6.

Error occurred at 2017-03-19_213040.
C:\Program Files\Unity\Editor\Unity.exe

To reproduce:
1. Create new project with some XAML displaying with Noesis
2. Use this free asset to take screenshot: https://www.assetstore.unity3d.com/en/#!/content/24122 (very simple source code available)
I tried 1080x1920

Expected: All OK
Actual: Noesis crashes "Noesis.dll caused an Access Violation (0xc0000005) in module Noesis.dll at 0033:ca271ad6."

Edit: Also: the screenshot gets taken, but without any XAML painted in it (just the other objects/textures)
Edit2: I understand the package size decision - would like to have it as small as possible. Another option would be to have a dev package vs release package but that would be more cost for you. Ideally if the PC version worked for high-res screenshots that would be great :)
Edit3: The App Store screenshot requirements are 2048 x 2732 - bigger than my monitor https://developer.apple.com/library/con ... 5-CH26-SW2
 
nokola
Topic Author
Posts: 188
Joined: 10 Mar 2015, 05:29

Re: NoesisGUI (Unity) on iOS Simulator?

23 Mar 2017, 16:20

ping :) - it it a known issue or should I report a bug? Thanks!
 
User avatar
sfernandez
Site Admin
Posts: 1911
Joined: 22 Dec 2011, 19:20

Re: NoesisGUI (Unity) on iOS Simulator?

23 Mar 2017, 20:00

Hi,

I tried the screenshot asset you mentioned but didn't crash after a screenshot was taken.
But as you said, no UI is rendered in the screenshot. That asset may be reading the camera surface before we render the UI.

Have you tried to just create a RenderTexture and attach it to the camera to create the screenshot yourself?
 
nokola
Topic Author
Posts: 188
Joined: 10 Mar 2015, 05:29

Re: NoesisGUI (Unity) on iOS Simulator?

26 Mar 2017, 04:51

Got it to work! (partially)
The screenshot is taken without crash when I set EnablePostProcess = true; in code (we create the view in code) or "Enable Post Process" to true in the Inspector.

One last issue - the Noesis view comes upside-down, even though the rest of the drawing is right side up. Trying to fix that by meddling with the pixel matrix settings.
Here's the code that I use, you can call it like: TakeScreenshot(myCamera, 2560, 1440, @"c:\file.png");
    private static void TakeScreenshot(Camera camera, int width, int height, string outputFilePath)
    {
        RenderTexture renderTexture = new RenderTexture(width, height, 0, RenderTextureFormat.ARGB32, RenderTextureReadWrite.Linear);
        renderTexture.Create();
        if (!renderTexture.IsCreated())
        {
            throw new Exception("Failed to create render texture");
        }

        // render
        RenderTexture savedTexture = camera.targetTexture;
        camera.targetTexture = renderTexture;
        camera.Render();
        camera.targetTexture = savedTexture;

        // copy to Texture2D
        RenderTexture savedRenderTexture = RenderTexture.active;
        RenderTexture.active = renderTexture;
        Texture2D screenshot = new Texture2D(width, height, TextureFormat.ARGB32, mipmap: false);
        screenshot.ReadPixels(new Rect(0, 0, width, height), 0, 0);
        screenshot.Apply();
        RenderTexture.active = savedRenderTexture;

        // write to file
        byte[] bytes = screenshot.EncodeToPNG();
        System.IO.File.WriteAllBytes(outputFilePath, bytes);
        Debug.Log("Saved screenshot to " + outputFilePath);
    }
 
nokola
Topic Author
Posts: 188
Joined: 10 Mar 2015, 05:29

Re: NoesisGUI (Unity) on iOS Simulator?

26 Mar 2017, 05:35

Here is the Unity/screenshot result: https://1drv.ms/i/s!AspGCVZdWgTBuvJ6dKM4HPxKexfF0A
Repro project: https://1drv.ms/u/s!AspGCVZdWgTBuvJ7jBieG_-fjunQUw

Setup:
1. XAML:
<UserControl ...>
    <Grid x:Name="LayoutRoot">
        <Viewbox>
		    <Grid Width="900" Height="600" Margin="50">
				<TextBlock Text="XAML Text" FontSize="72" />
		    </Grid>
	    </Viewbox>
    </Grid>
  </UserControl>
2. Scene: Add 3D text "Unity 3D Text: UP↑"
To repro:
1. Take screenshot
Expected: "XAML Text" and "Unity 3D Text: UP↑" are correctly positioned
Actual: "XAML Text" is upside down (see screenshot above)

Is there a quick workaround?
 
nokola
Topic Author
Posts: 188
Joined: 10 Mar 2015, 05:29

Re: NoesisGUI (Unity) on iOS Simulator?

26 Mar 2017, 09:45

Figured it out. Noesis flips the texture Y in NoesisView.
Unfortunately, it doesn't respect my app's choice when I set noesisView.RenderFlags = Noesis.View.RenderFlagsFlipY;

So I had to add this hacky code :(
In NoesisView.cs before UpdateSettings():
    View.RenderFlags? _renderFlagsOverride = null;
    public void OverrideRenderFlags(View.RenderFlags? newRenderFlags)
    {
        _renderFlagsOverride = newRenderFlags;
        LateUpdate();
    }

    private void UpdateSettings()
...
Also change the last line of UpdateSettings():
        if (_renderFlagsOverride.HasValue)
        {
            _uiView.SetFlags(_renderFlagsOverride.Value);
        }
        else
        {
            _uiView.SetFlags(renderFlags);
        }
then call Noesis to draw a frame (because it doesn't update immediately), then render to texture:
/// <summary>
/// Add ability to take screenshots with Noesis by pressing ` key
/// </summary>
public class ScreenGrabber : MonoBehaviour
{
    Camera _camera;
    NoesisView _noesisView;
    public string OutputPng = @"D:\@Niko\Projects\Fantasia\Assets\TestScreenshots\test.png";
    public int Width = 4096;
    public int Height = 3072;

    // Use this for initialization
    void Start()
    {
        _camera = GetComponent<Camera>();
        _noesisView = GetComponent<NoesisView>();
    }

    int _framesToScreenshot = -1;
    bool _savedEnablePostProcess;

    // Update is called once per frame
    void Update()
    {
        if (Input.GetKeyDown(KeyCode.BackQuote)) // if "`"
        {
            _framesToScreenshot = 1;
            _savedEnablePostProcess = _noesisView.EnablePostProcess;
            _noesisView.EnablePostProcess = true;
            _noesisView.OverrideRenderFlags(Noesis.View.RenderFlags.FlipY);
        }

        if (_framesToScreenshot >= 0)
        {
            _framesToScreenshot--;
        }

        if (_framesToScreenshot == 0)
        {
            TakeScreenshot(_camera, _noesisView, Width, Height, OutputPng);
            Application.OpenURL(OutputPng);

            _noesisView.EnablePostProcess = _savedEnablePostProcess;
            _noesisView.OverrideRenderFlags(null);
        }
    }

    private static RenderTexture CreateRenderTexture(int width, int height, RenderTextureFormat format = RenderTextureFormat.ARGB32)
    {
        RenderTexture result = new RenderTexture(width, height, 0, format, RenderTextureReadWrite.Linear);
        result.useMipMap = false;
        result.antiAliasing = 1;
        result.Create();
        if (!result.IsCreated())
        {
            throw new Exception("NOOO!!");
        }
        return result;
    }

    private static void TakeScreenshot(Camera camera, NoesisView noesis, int width, int height, string outputFilePath)
    {
        // create a clone of noesis to render to a temp texture
        //GameObject noesisObject = new GameObject("NoesisOnly");

        //Camera noesisCamera = noesisObject.AddComponent<Camera>();
        //noesisCamera.clearFlags = CameraClearFlags.SolidColor;
        //noesisCamera.backgroundColor = new UnityEngine.Color(0.0f, 0.0f, 0.0f, 0.0f);
        //noesisCamera.renderingPath = RenderingPath.Forward;
        //noesisCamera.depthTextureMode = DepthTextureMode.None;
        //noesisCamera.opaqueSortMode = UnityEngine.Rendering.OpaqueSortMode.NoDistanceSort;
        //noesisCamera.transparencySortMode = TransparencySortMode.Orthographic;
        //noesisCamera.clearStencilAfterLightingPass = false;
        //noesisCamera.hdr = false;
        //noesisCamera.useOcclusionCulling = false;
        //noesisCamera.cullingMask = 0;
        //noesisCamera.targetTexture = CreateRenderTexture(width, height);


        //NoesisView noesisClone = noesisObject.AddComponent<NoesisView>();
        ////noesisClone.Xaml = noesis.Xaml;

        //noesisClone.Xaml = (NoesisXaml)UnityEngine.Resources.Load("ScoreWindow1");
        //Debug.Log(noesisClone.Xaml);
        //noesisClone.LoadXaml(true); // CRASH!!!
        //noesisClone.EnablePostProcess = true;
        //noesisClone.AntiAliasingMode = Noesis.View.AntialiasingMode.MSAA;
        //noesisClone.TessellationQuality = Noesis.View.TessellationQuality.High;
        //noesisClone.RenderFlags = Noesis.View.RenderFlags.FlipY
        //noesisCamera.Render(); // CRASH!!!

        // render to texture
        RenderTexture sceneNoNoesisTexture = CreateRenderTexture(width, height);
        RenderTexture savedTexture = camera.targetTexture;
        camera.targetTexture = sceneNoNoesisTexture;
        camera.Render();
        camera.targetTexture = savedTexture;

        // copy to Texture2D
        RenderTexture savedRenderTexture = RenderTexture.active;
        RenderTexture.active = sceneNoNoesisTexture;
        Texture2D screenshot = new Texture2D(width, height, TextureFormat.ARGB32, mipmap: false);
        screenshot.ReadPixels(new Rect(0, 0, width, height), 0, 0);
        screenshot.Apply();
        RenderTexture.active = savedRenderTexture;

        // write to file
        byte[] bytes = screenshot.EncodeToPNG();
        System.IO.File.WriteAllBytes(outputFilePath, bytes);
        Debug.Log("Saved screenshot to " + outputFilePath);
    }

}

Hope this helps someone else. Ideally the RenderFlags should be respected or even better no FlipY when rendering to texture.
 
User avatar
jsantos
Site Admin
Posts: 2900
Joined: 20 Jan 2012, 17:18
Contact:

Re: NoesisGUI (Unity) on iOS Simulator?

27 Mar 2017, 22:43

Hope this helps someone else. Ideally the RenderFlags should be respected or even better no FlipY when rendering to texture.
Definitely this seems to be a bug. We should respect that render flag. We need internally to use it though, because Unity renders differently in D3D vs OpenGL.

Could you please report it?

Thanks!

Who is online

Users browsing this forum: No registered users and 3 guests