saji8k
Topic Author
Posts: 16
Joined: 07 Jul 2023, 08:35

Apple Vision Pro Support

04 Oct 2023, 08:05

I was wondering if you guys working on getting NoesisGUI working on the Apple Vision Pro using Apple's Swift-based RealityKit 3D Engine?
https://developer.apple.com/documentation/realitykit

There are basically 5 types of applications on the Apple Vision Pro.
iPad/iPhone Windowed apps
Vision Pro Windowed apps
VR Immersive apps
AR Shared apps
AR Immersive apps

iPad/iPhone apps with NoesisGUI should already work. Vision Pro Windowed apps and VR Immersive apps both allow access to the Metal API so it should be straight forward to get NoesisGUI working there. That just leaves AR apps. The only way to create AR apps is using RealityKit, which does not give you access to the Metal API. Is there any work being done to get NoesisGUI working using RealityKit? Is that even possible?

Unity are building a Unity to RealityKit translator called PolySpatial to allow Unity users to create Vision Pro AR apps. Did you guys sign up for their beta program?
https://create.unity.com/spatial
Maybe it is possible to get NoesisGUI working via Unity.
 
User avatar
jsantos
Site Admin
Posts: 3869
Joined: 20 Jan 2012, 17:18
Contact:

Re: Apple Vision Pro Support

04 Oct 2023, 14:11

Hello! We haven't had enough time to explore this. We have a few important VR clients and as soon as they demand more things I assume we will move forward. For now, I think the most viable path is going through Unity but definitely we are going to explore other alternatives.

Thanks for the detailed post!
 
saji8k
Topic Author
Posts: 16
Joined: 07 Jul 2023, 08:35

Re: Apple Vision Pro Support

22 Nov 2023, 06:27

Last week, Unity opened up the PolySpatial beta to all Unity Pro, Unity Enterprise and Unity Industry subscribers.
https://discussions.unity.com/t/welcome ... ram/270282

A lot of rendering system components are not supported compared to regular Unity.
https://docs.unity3d.com/Packages/com.u ... tures.html
Render Textures do seem to be supported though so than might be another path.
 
User avatar
jsantos
Site Admin
Posts: 3869
Joined: 20 Jan 2012, 17:18
Contact:

Re: Apple Vision Pro Support

24 Nov 2023, 15:09

It seems native plugins are already supported. Wondering if we will be able to render directly in 3D with our Metal renderer. Definitely interesting. Thanks for sharing!
 
saji8k
Topic Author
Posts: 16
Joined: 07 Jul 2023, 08:35

Re: Apple Vision Pro Support

28 Nov 2023, 12:32

You'll be able to use your metal renderer to render directly in 3D for VR apps, since these apps have access to metal. This should work in Unity without making any new changes as it doesn't use the PolySpatial system.
You should also be able to make VR apps with the Native SDK version using this as a guide: https://github.com/metal-by-example/met ... -rendering

I don't think you'll be able to render directly in 3D for AR apps though. To make AR apps you have to use Apple's RealityKit high-level rendering engine, which does not give you a way to render your own 3D metal code. Unity's PolySpatial system simply translates Unity projects into RealityKit, so if it it is not possible in RealityKit, it won't be possible in Unity PolySpatial either.

While I am not sure how to do this using Unity PolySpatial, I think it might be possible to use your metal renderer in the Native SDK to render a UI and display it in 2D using RealityKit. The metal renderer in the Native SDK seems to use CAMetalLayer API which is also available in visionOS. There are a couple posts in the apple forums that seems to indicate you can render using metal into a CAMetalLayer and then display that in a flat 2D window in RealityKit:
https://developer.apple.com/forums/thread/732355
https://developer.apple.com/forums/thread/733398
 
saji8k
Topic Author
Posts: 16
Joined: 07 Jul 2023, 08:35

Re: Apple Vision Pro Support

12 Jan 2024, 01:19

The latest version of Xcode (15.2) now has official visionOS support. So you no longer need the beta version, if you happen to want to build the core library for visionOS. ;)
The Apple Vision Pro will be released in 3 weeks (Feb 2, 2024).

If you want to build Vision Pro Windowed support into the application framework, I think these two examples will help you get started:

CAMetalLayer

https://github.com/satoshi0212/visionOS ... main/Day35

Metal Shader and Transparency

https://github.com/satoshi0212/visionOS ... main/Day36

Who is online

Users browsing this forum: Bing [Bot], laudaicat, Semrush [Bot] and 0 guests