Hello!
My name is Mason Prather. I'm a graduate student at Kennesaw State University and a Research Engineer working in XR environments through my Graduate Research Assistant role. I’m currently building a research prototype that connects a mobile companion application to a VR headset.
The mobile application is built in Unity and deployed on iOS, and it streams video frames to a remote Unity client using WebRTC.
Environment
Device: iPhone 15
OS: iOS 26.3 (tested on physical device, not Simulator)
Engine: Unity 2022.3.57f1
Graphics API: Metal
Streaming Technology: WebRTC (Unity WebRTC package)
Architecture: Mobile Unity app streaming video frames to a remote Unity client
Receiver Device: Meta Quest Pro headset (Unity application)
Networking: LAN (UDP discovery + TCP signaling)
Video Source: Unity RenderTexture
Goal
The goal of the system is to allow a VR user to view media stored on their phone inside a VR environment.
The iOS app:
- renders or captures media content
- converts frames into a WebRTC video track
- streams the video to the headset
Current Status
Connection setup works correctly.
Observed behavior:
- Signaling connection successful
- ICE candidate exchange successful
- PeerConnection state becomes Connected
- Video track created successfully
However, the receiving application displays black frames.
iOS App Details
The video source originates from a Unity RenderTexture.
Inside the phone application:
- RenderTexture displays correctly
- Frames appear correct locally
But the receiving peer does not display the frames.
Relevant Components
- Unity WebRTC package
- iOS Metal rendering pipeline
- Custom TCP signaling
- LAN discovery via UDP
Expected Behavior
Rendered frames should transmit via WebRTC and appear on the remote device.
Actual Behavior
The remote video track is active, but the rendered frames appear black on the receiving client.
Questions
- Are there known issues involving Unity WebRTC + iOS Metal texture capture?
- Are there specific pixel format requirements when streaming textures from Unity on iOS?
- Could the issue relate to texture readback limitations or GPU synchronization?
I am more than happy to provide screenshots and console logs upon request.
If anyone has experience streaming Unity video frames via WebRTC on iOS, I would greatly appreciate any guidance.