Page 1 of 1

3d pyramid hologram stream

Posted: Sat Jun 06, 2020 11:48 am
by MagnusHerrlin
Hello

This is my first post here
I bought two stereopis and I like to combine the videostreams into one
arranged for display in a 3d hologram pyramid
like this one: https://www.youtube.com/watch?v=Y60mfBvXCj8

Does anybody here know how to do it?
perhaps using ffmpg or something?

Thanks
/Magnus

Re: 3d pyramid hologram stream

Posted: Sat Jun 06, 2020 2:47 pm
by Realizator
Hi Mangus,
Great project idea! :-)
Synchronizing 4 cameras is not so simple. One StereoPi can record synchronized video streams from the two cameras, so you need to sync two StereoPis.
Also combining them to the single video stream can be done in several ways.
Mangus, what is your global aim: do a 4-cams recording in a real-time, or you can combine your records by the post-processing?

Re: 3d pyramid hologram stream

Posted: Sun Jun 07, 2020 10:47 am
by MagnusHerrlin
Hi

Thanks Realizator.

Yes my goal is to combine four livestreams into one
so it can be displayed in a 3d pyramid, all this happening in realtime.
For example you can have a chat app and see the person you are talking to in a 3d pyramid

You could make this very complicated by for example
combining the livestreams from four syncronized drones with cameras.

It is not really stereopi specific I just thougt it would be easier to use two stereopis than four regular Pis

I did try this with four RPi3s and combine their signals.
I made a program/automated script that broadcast a udp command to all four pis to start/stop the recording
then using ffmpeg to rotate the videos then uploading them to a folder on my PC
where I combined them into one film using a free videoprogram.
But the syncronisation wasnt perfect or even half good it was a bit of a hassle to get them all right.

Of course I want to avoid all that postproccessing hassle and have some sort of box doing the combination in realtime.
Or could you get better sync using a gpio on each pi connected to the same trigger?

I guess the best way to do it is using some sort of fpga to read all four cameras but that is beyond my skills.

Cheers
/Magnus

Re: 3d pyramid hologram stream

Posted: Tue Jun 09, 2020 1:11 pm
by stereomaton
I never tried this kind of pyramidal display because I always thought that it was just a bad joke.
In particular, there is a single image for the two eyes (per side), so it is likely just like a head-up display (2D image in the air), repeated 4 times. The shadows and movement can probably trick some people, though.
By the way, by sampling the video you linked, except for a few exceptions (one with 1/4 out of sync + one that has two opposite sides upside-down + one with real 4 viewpoints), the 4 images are identical; which makes sense since there is no real depth encoded. The one with 4 real viewpoints can add an interesting effect, but since the model turns it may be less obvious.

However, I guess that if you want to do such a project, you found it nice. Actually, I think that the four viewpoints idea is a good addition, even if in my opinion it will never have the same awesome feeling as stereoscopic images where true depth can be perceived (assuming that you are not in the small part of the population that cannot use the strong binocular depth cues).

For your project, I would stream the videos from the StereoPi boards to a computer and do all the processing on this powerful machine, including the rotations (or possibly rotate the cameras physically depending of the expected aspect ratio). The videos will be in sync two by two. To synchronize the two pairs, it will be harder. However, a sub-frame synchronization is probably not necessary in your case. You might try to start the videos in response to a change on a GPIO, which might help to have a better starting point (far less variable than network datagrams); but there are several places where a drift can be added nonetheless. For the combination, you seem to have found one way; I personally would have started by exploring with https://obsproject.com/ although I do not know if it can receive live feeds (probably).

Let us know if you manage to get something interesting.