S.L.P. image questions, stereoscopic video livestream and recording, stereoscopic photo capture etc.
-
wildag
- Posts: 19
- Joined: Sat May 04, 2019 9:45 pm
Post
by wildag » Mon May 06, 2019 12:21 am
This is just documenting some success for others.
I've managed to get my local gstreamer install to decode and modify the stereopi udp stream into a side-by-side stereo equirectangular projection.
I used the following article to build gst-plugins-vr with a fork of fixes for OSX.
https://medium.com/@yofujimo/how-to-mak ... fcab0bc020
Here is the command with a test viewer.
Code: Select all
gst-launch-1.0 -mv udpsrc port=3000 buffer-size=300000 ! h264parse ! avdec_h264 ! glupload ! glcolorconvert ! hmdwarp ! gldownload ! videoconvert ! video/x-raw, width=1920, height=1080 ! autovideosink
And here is the command to forward the stream to another udp stream viewer
Code: Select all
gst-launch-1.0 -mv udpsrc port=3000 buffer-size=300000 ! h264parse ! avdec_h264 ! glupload ! glcolorconvert ! hmdwarp ! gldownload ! videoconvert ! video/x-raw, width=1920, height=1080 ! x264enc ! rndbuffersize max=6500 ! udpsink host=DESTINATION_IP port=3000
-
thaytan
- Posts: 2
- Joined: Wed May 15, 2019 4:55 pm
Post
by thaytan » Wed May 15, 2019 5:31 pm
wildag wrote: ↑Mon May 06, 2019 12:21 am
This is just documenting some success for others.
I...
Here is the command with a test viewer.
Code: Select all
gst-launch-1.0 -mv udpsrc port=3000 buffer-size=300000 ! h264parse ! avdec_h264 ! glupload ! glcolorconvert ! hmdwarp ! gldownload ! videoconvert ! video/x-raw, width=1920, height=1080 ! autovideosink
I'm surprised if you need the gldownload and friends to make this work on MacOS. This should work, without the GPU -> CPU -> GPU overhead:
gst-launch-1.0 -mv udpsrc port=3000 buffer-size=300000 ! h264parse ! avdec_h264 ! glupload ! glcolorconvert ! hmdwarp ! glimagesink
or, even better see if this works using the native H264 decoder:
gst-launch-1.0 -mv udpsrc port=3000 buffer-size=300000 ! h264parse ! vtdec ! glupload ! glcolorconvert ! hmdwarp ! glimagesink
-
wildag
- Posts: 19
- Joined: Sat May 04, 2019 9:45 pm
Post
by wildag » Thu May 16, 2019 1:02 pm
The purpose of the stream above is to push the image to an oculus or other VR headset. I believed that requires the gldownload step. Any other way to do it? I’m not an expert
-
Realizator
- Site Admin
- Posts: 295
- Joined: Tue Apr 16, 2019 9:23 am
-
Contact:
Post
by Realizator » Sat May 18, 2019 10:29 am
thaytan wrote: ↑Wed May 15, 2019 5:31 pm
I'm surprised if you need the gldownload and friends to make this work on MacOS. This should work, without the GPU -> CPU -> GPU overhead:
Thaytan, Wildag's task is to obtain equirectangular projection on the Oculus Go. Ideally we have only StereoPi (CM3 onboard) and Oculus Go (or Quest). So we have two options here:
1. Do this real-time image transformation onboard (CM3) and live stream already processed video.
2. Send video "as is" to Oculus, and do this transformation onboard.
His mentioned Mac implementation is a proof-of concept. I agree with you, that Mac has enough performance to do this without GPU. But both Raspberry Pi CM3 and Oculus are embedded systems with limited performance, so GPU acceleration is the only way to obtain FPS and latency needed.
Eugene a.k.a. Realizator