Oculus Go livestream?

S.L.P. image questions, stereoscopic video livestream and recording, stereoscopic photo capture etc.
wildag
Posts: 19
Joined: Sat May 04, 2019 9:45 pm

Oculus Go livestream?

Post by wildag » Sun May 05, 2019 12:32 am

One of the stereopi demos showed some people using an Oculus Go with a stereopi mounted above and behind them in a “3rd person view of yourself” mode.

Does anyone know how they streamed live video to the Oculus Go? Was it a custom Oculus app? Or is there some stream viewing capability in the Go?

If it was a custom app, can we get the source code for it?

wildag
Posts: 19
Joined: Sat May 04, 2019 9:45 pm

Re: Oculus Go livestream?

Post by wildag » Sun May 05, 2019 1:13 am

I found the page on the wiki that says the following android application can be used on the Oculus Go. But I can't figure out how to download the APK file for the app.

https://play.google.com/store/apps/deta ... l.stereopi

wildag
Posts: 19
Joined: Sat May 04, 2019 9:45 pm

Re: Oculus Go livestream?

Post by wildag » Sun May 05, 2019 1:37 am

I found that the Oculus Go browser directly to the StereoPi web UI works ok. But there are lots of packet drops (might be my network) causing video distortion.

I found that 1280x800 seems to be the most ideal native resolution. But the stream needs to be modified to be rid of the spherical look (projecting planar video on a spherical surface to retain planar view - forgot the technical name)

User avatar
Realizator
Site Admin
Posts: 227
Joined: Tue Apr 16, 2019 9:23 am
Contact:

Re: Oculus Go livestream?

Post by Realizator » Sun May 05, 2019 9:14 am

Oops, it's our fail. After adding our app to Google Play, I removed link to APK download from our Wiki.
Now it is fixed in Wiki, and here is a link for APK download (and installation on Oculus Go):
APK file
This is the version used in our Oculus experiment.
Eugene a.k.a. Realizator

wildag
Posts: 19
Joined: Sat May 04, 2019 9:45 pm

Re: Oculus Go livestream?

Post by wildag » Sun May 05, 2019 12:56 pm

Thanks for the APK!

I side loaded the APK file via "adb install -r StereoPi....apk" command, but it's not showing up in the Go's Library -> Unknown Sources list (or anywhere else in the library). This follow Redit page says that only apps compiled for Occulus go will show up in the library. It provides a work around using "AppStarter" - but I can't get those side loaded APK files to show up in the Oculus Go library either.

https://www.reddit.com/r/OculusGo/wiki/ ... wiki_notes

If you shared the source code of the APK, maybe we could recompile this for Oculus Go?

Or maybe there's some other step I'm missing.

wildag
Posts: 19
Joined: Sat May 04, 2019 9:45 pm

Re: Oculus Go livestream?

Post by wildag » Sun May 05, 2019 1:49 pm

I found that you can use AppStarter and/or OculusTvLauncher to launch Android native apps into the Oculus Go. But they just show up as a virtual living room with a big screen projector of the app. So it's not a "full screen" experience.

Still looking for a way to view stereoscopic live streaming from a StereoPi to an Oculus Go....

User avatar
Realizator
Site Admin
Posts: 227
Joined: Tue Apr 16, 2019 9:23 am
Contact:

Re: Oculus Go livestream?

Post by Realizator » Mon May 06, 2019 1:01 pm

Now we're digging in our logs to make a kind of step-by-step guide for installing our app to Oculus. Hope we'll finish it today.
Eugene a.k.a. Realizator

User avatar
Gol
Posts: 6
Joined: Tue Apr 16, 2019 10:07 am

Re: Oculus Go livestream?

Post by Gol » Mon May 06, 2019 2:09 pm

Here is an StereoPi.apk for Oculus Go. Archive also contains adb.
StereoPi-OculusGo.zip
(4.26 MiB) Downloaded 126 times

Install application:

Code: Select all

adb install StereoPi.apk
Uninstall:

Code: Select all

adb uninstall com.virt2real.stereopi
After installation application will be available in Library -> Unknown Sources -> com.virt2real.stereopi

stereopi-in-unknown-sources.png
stereopi-in-unknown-sources.png (795.95 KiB) Viewed 1512 times

User avatar
Realizator
Site Admin
Posts: 227
Joined: Tue Apr 16, 2019 9:23 am
Contact:

Re: Oculus Go livestream?

Post by Realizator » Mon May 06, 2019 4:07 pm

@wildag, Gol uploaded updated APK with fixed manifest file. This ZIP also contains adb.exe we used to upload. Please check if it works for you.

Also I've updated our Wiki by adding Oculus Go application section in S.L.P. image description.

upd :idea: > Wildag, do you have an experience of doing apps for Oculus Go? We need some example to start with. If you have any basic application (ideally non-Unity), we can provide you with our part of the code (autodiscovery, livestream etc.). By combining our efforts we can do simple open source application, which can be used by other users to make their project-specific applications.
Eugene a.k.a. Realizator

wildag
Posts: 19
Joined: Sat May 04, 2019 9:45 pm

Re: Oculus Go livestream?

Post by wildag » Mon May 06, 2019 8:23 pm

Awesome! Thanks for uploading it. I've loaded that StereoPi-OculusGo app, and it worked! Very cool to see yourself in 3d :)

Wishlist for features
  • Lower lag
    • I've found that UDP streams have much lower lag time than RTMP/RTSP streams
      The native android app has super low lag
    Head Tracking
    • The users of my application need to be able to look left and right in the fixed point of view frame.
@Realizator I don't have any Oculus Go application development experience outside of Unity. But I do have a lot of Java and mobile app development experience (separately - not in Android). I'm happy to pitch in on an open source StereoPi app for Oculus Mobile.

User avatar
Gol
Posts: 6
Joined: Tue Apr 16, 2019 10:07 am

Re: Oculus Go livestream?

Post by Gol » Mon May 06, 2019 8:55 pm

wildag wrote:
Mon May 06, 2019 8:23 pm
  • Lower lag
    • I've found that UDP streams have much lower lag time than RTMP/RTSP streams
      The native android app has super low lag
Oculus application is the same as in Google Play, but with different manifest. So, video latency is the same, video transferred with UDP too.

User avatar
Realizator
Site Admin
Posts: 227
Joined: Tue Apr 16, 2019 9:23 am
Contact:

Re: Oculus Go livestream?

Post by Realizator » Mon May 06, 2019 9:34 pm

wildag wrote:
Mon May 06, 2019 8:23 pm
Awesome! Thanks for uploading it. I've loaded that StereoPi-OculusGo app, and it worked! Very cool to see yourself in 3d :)
Cool! You are the first one who repeated our Oculus experiment. Congratulations! :D
wildag wrote:
Mon May 06, 2019 8:23 pm
Wishlist for features
  • Lower lag
    • I've found that UDP streams have much lower lag time than RTMP/RTSP streams
      The native android app has super low lag
    Head Tracking
    • The users of my application need to be able to look left and right in the fixed point of view frame.
As Gol mentioned, we use UDP and raw H264, not RTMP/RTSP. So I can not understand, if you wish to obtain even lower lag, or current latency is good for you? You mentioned this in a wishlist, that's why I'm trying to clarify this for myself.

As for other features - I think we can try to do this together. ;)
wildag wrote:
Mon May 06, 2019 8:23 pm
@Realizator I don't have any Oculus Go application development experience outside of Unity. But I do have a lot of Java and mobile app development experience (separately - not in Android). I'm happy to pitch in on an open source StereoPi app for Oculus Mobile.
A-ha, sounds interesting. Can you help us to create a very simple application (like our current one) using Unity? We can provide you with all our video-related code and connectivity logic.
Eugene a.k.a. Realizator

wildag
Posts: 19
Joined: Sat May 04, 2019 9:45 pm

Re: Oculus Go livestream?

Post by wildag » Tue May 07, 2019 2:45 pm

Realizator wrote:
Mon May 06, 2019 9:34 pm

Cool! You are the first one who repeated our Oculus experiment. Congratulations! :D
That demo is the reason I bought the kit - so I feel like you delivered well!

Realizator wrote:
Mon May 06, 2019 9:34 pm

As Gol mentioned, we use UDP and raw H264, not RTMP/RTSP. So I can not understand, if you wish to obtain even lower lag, or current latency is good for you? You mentioned this in a wishlist, that's why I'm trying to clarify this for myself.
Good information. I'll mess around with bitrate and resolution to see if I can optimize.
Realizator wrote:
Mon May 06, 2019 9:34 pm

As for other features - I think we can try to do this together. ;)

A-ha, sounds interesting. Can you help us to create a very simple application (like our current one) using Unity? We can provide you with all our video-related code and connectivity logic.
Yes, let's collaborate on an app. Feel free to share a git repository of what you have, and I'll try to find a workable framework. Then we can collaborate on making some basic features for it.

User avatar
Gol
Posts: 6
Joined: Tue Apr 16, 2019 10:07 am

Re: Oculus Go livestream?

Post by Gol » Tue May 07, 2019 5:32 pm

I also tried to feed StereoPi live video into Skybox video player on Oculus Go. It is my favourive video player for Oculus Go, but it can't play streams by default. Here is the first result:



I'll make a some tests and publish code. It's on Node.js

wildag
Posts: 19
Joined: Sat May 04, 2019 9:45 pm

Re: Oculus Go livestream?

Post by wildag » Wed May 08, 2019 11:00 am

I looked at options for Oculus Mobile development SDKs. We have a couple of options.

1) Oculus Mobile Native - which is straight java with JNI bindings to C code. This would likely the be the highest performance option

https://developer.oculus.com/documentat ... ok-native/

2) Unity 3d with the Universal Media Plugin - this would allow for more rapid development of UI features, like menus etc. But it might not be the lowest lag option - as the Oculus has to perform some transformations on the video.

I'm considering engaging a freelancer to get the project started - as this is a bit out of my wheel house. I am sure that once the framework of the application is started, I can contribute to expanding the features.

User avatar
Realizator
Site Admin
Posts: 227
Joined: Tue Apr 16, 2019 9:23 am
Contact:

Re: Oculus Go livestream?

Post by Realizator » Wed May 08, 2019 3:42 pm

wildag, ok, now we need to find some experienced programmer. I'll try to find one too.
Eugene a.k.a. Realizator

wildag
Posts: 19
Joined: Sat May 04, 2019 9:45 pm

Re: Oculus Go livestream?

Post by wildag » Sat May 11, 2019 1:38 pm

I put in a request for bids on upwork.com. Received two believable bids for $1,500. One of them is a group from Ukraine. I've invited that group from the Ukraine to view this forum and the StereoPi project.

They're proposing that the project be built in Unity 3d - which I think is a good choice. That would provide a starting place me and other groups to integrate what they create into more complex applications (think menu systems, overlay information, logos, etc).

I'm willing to cost split on this - but I imagine this would make StereoPi very unique amongst the other options.

User avatar
Realizator
Site Admin
Posts: 227
Joined: Tue Apr 16, 2019 9:23 am
Contact:

Re: Oculus Go livestream?

Post by Realizator » Sat May 11, 2019 3:39 pm

@wildag, I've putted help request in our last update. Let's wait for some time.
I'm also talking with some my colleagues now, and will try to find specialists too. :-)
Eugene a.k.a. Realizator

orotot
Posts: 6
Joined: Sun May 12, 2019 4:45 pm

Re: Oculus Go livestream?

Post by orotot » Tue May 14, 2019 6:09 am

Hi all,

I think i was able to side load the StereoPi.apk to my Oculus Go - it is now showing up in the Unknown Sources menu. When i launch the app on the Go tho it is just a black screen. I presume it is a connection problem to the StereoPi.

1. is it a usb connection to the StereoPi?
2. did i miss something - i will go back and re-check the wiki
3. is is a wifi connection - that would maybe also make sense.

thank you !! :)

User avatar
Realizator
Site Admin
Posts: 227
Joined: Tue Apr 16, 2019 9:23 am
Contact:

Re: Oculus Go livestream?

Post by Realizator » Tue May 14, 2019 6:40 am

orotot wrote:
Tue May 14, 2019 6:09 am
Hi all,

I think i was able to side load the StereoPi.apk to my Oculus Go - it is now showing up in the Unknown Sources menu. When i launch the app on the Go tho it is just a black screen. I presume it is a connection problem to the StereoPi.

1. is it a usb connection to the StereoPi?
2. did i miss something - i will go back and re-check the wiki
3. is is a wifi connection - that would maybe also make sense.

thank you !! :)
ororot, black screen means that app is started, but StereoPi is not found.
1. No, it is not USB connection to the StereoPi. You need to have your Oculus Go ang StereoPi to be in the same network. StereoPi can be connected to the router by Ethernet (RJ45 connector) or WiFi, it does not matter.
2. I think you need just check network connection for both Oculus and StereoPi
3. For WiFi connection setup you need just to fill up 3 fields in stereopi.conf mentioned here. Set "wlan0", but not "wifi0", as "wifi0" settings is for built-in WiFi on the classic Raspberry (not for StereoPi).
Eugene a.k.a. Realizator

Post Reply