Apple Vision Pro UI Interactions with Unity PolySpatial | Keyboard and Voice Inputs

Hi XR Developers! In this video we will look at how to use the visionOS system keyboard for our Unity UI and how to achieve speech to text input using AI (Whisper from OpenAI)!

Resources:
🔗 Unity PolySpatial 1.1 Documentation: https://docs.unity3d.com/Packages/com.unity.polyspatial.visionos@1.1/manual/index.html
🔗 Play to Device: https://docs.unity3d.com/Packages/com.unity.polyspatial.visionos@1.1/manual/PlayToDevice.html
🔗 Input Documentation: https://docs.unity3d.com/Packages/com.unity.polyspatial.visionos@1.1/manual/Input.html
🔗 Whisper for Unity: https://github.com/Macoron/whisper.unity?tab=readme-ov-file
🔗 Whisper Models: https://huggingface.co/ggerganov/whisper.cpp/tree/main

Other Videos:
📹 Create and Manage SwiftUI windows with Unity PolySpatial for Apple visionOS: https://www.youtube.com/watch?v=iJjfavO6E1A&t
📹 Apple Vision Pro Input & Object Manipulation with Unity PolySpatial: https://www.youtube.com/watch?v=k9cJfn4JcTA&t
📹 Develop for Apple Vision Pro with Unity’s PolySpatial: https://www.youtube.com/watch?v=LYRJIewPiLU&t
📹 How to become an Apple XR Developer with Unity for visionOS: https://www.youtube.com/watch?v=OX72Fb-hGpw&t

Join the Discord!
👾 https://discord.gg/v8fVjweUPB

Support the Channel:
🫂 Patreon: https://www.patreon.com/xrdevrob
🐦 Twitter: https://twitter.com/xrdevrob

🤝 Join this channel to get access to perks:
https://www.youtube.com/channel/UCat2tlm29PRViBe8_yz4j5Q/join

Need professional XR & Multiplayer development for your project?
👨‍💻 Get in touch with us: https://bento.me/blackwhale

Chapters:
0:00 Intro
0:39 Setup Scene with TMP Input Field
2:03 Testing the Keyboard Input
2:25 Set up Whisper AI in Unity
4:06 Whisper Scene Setup
5:44 Whisper Voice Input Script
7:09 Testing Whisper AI
8:08 Outro

Видео Apple Vision Pro UI Interactions with Unity PolySpatial | Keyboard and Voice Inputs канала XR Dev Rob - XR AI Tutorials