Oculus Go livestream?

[wildag]

One of the stereopi demos showed some people using an Oculus Go with a stereopi mounted above and behind them in a “3rd person view of yourself” mode.

Does anyone know how they streamed live video to the Oculus Go? Was it a custom Oculus app? Or is there some stream viewing capability in the Go?

If it was a custom app, can we get the source code for it?

[wildag]

I found the page on the wiki that says the following android application can be used on the Oculus Go. But I can't figure out how to download the APK file for the app.

https://play.google.com/store/apps/deta ... l.stereopi

[wildag]

I found that the Oculus Go browser directly to the StereoPi web UI works ok. But there are lots of packet drops (might be my network) causing video distortion.

I found that 1280x800 seems to be the most ideal native resolution. But the stream needs to be modified to be rid of the spherical look (projecting planar video on a spherical surface to retain planar view - forgot the technical name)

[Realizator]

Oops, it's our fail. After adding our app to Google Play, I removed link to APK download from our Wiki.
Now it is fixed in Wiki, and here is a link for APK download (and installation on Oculus Go):
APK file
This is the version used in our Oculus experiment.

[wildag]

Thanks for the APK!

I side loaded the APK file via "adb install -r StereoPi....apk" command, but it's not showing up in the Go's Library -> Unknown Sources list (or anywhere else in the library). This follow Redit page says that only apps compiled for Occulus go will show up in the library. It provides a work around using "AppStarter" - but I can't get those side loaded APK files to show up in the Oculus Go library either.

https://www.reddit.com/r/OculusGo/wiki/ ... wiki_notes

If you shared the source code of the APK, maybe we could recompile this for Oculus Go?

Or maybe there's some other step I'm missing.

[wildag]

I found that you can use AppStarter and/or OculusTvLauncher to launch Android native apps into the Oculus Go. But they just show up as a virtual living room with a big screen projector of the app. So it's not a "full screen" experience.

Still looking for a way to view stereoscopic live streaming from a StereoPi to an Oculus Go....

[Realizator]

Now we're digging in our logs to make a kind of step-by-step guide for installing our app to Oculus. Hope we'll finish it today.

[Gol]

Here is an StereoPi.apk for Oculus Go. Archive also contains adb.

StereoPi-OculusGo.zip

(4.26 MiB) Downloaded 2192 times

Install application:

Code: Select all

adb install StereoPi.apk

Uninstall:

Code: Select all

adb uninstall com.virt2real.stereopi

After installation application will be available in Library -> Unknown Sources -> com.virt2real.stereopi

stereopi-in-unknown-sources.png (795.95 KiB) Viewed 57308 times

[Realizator]

@wildag, Gol uploaded updated APK with fixed manifest file. This ZIP also contains adb.exe we used to upload. Please check if it works for you.

Also I've updated our Wiki by adding Oculus Go application section in S.L.P. image description.

upd > Wildag, do you have an experience of doing apps for Oculus Go? We need some example to start with. If you have any basic application (ideally non-Unity), we can provide you with our part of the code (autodiscovery, livestream etc.). By combining our efforts we can do simple open source application, which can be used by other users to make their project-specific applications.

[wildag]

Awesome! Thanks for uploading it. I've loaded that StereoPi-OculusGo app, and it worked! Very cool to see yourself in 3d

Wishlist for features

• Lower lag
  • I've found that UDP streams have much lower lag time than RTMP/RTSP streams
    The native android app has super low lag
  Head Tracking
  • The users of my application need to be able to look left and right in the fixed point of view frame.

@Realizator I don't have any Oculus Go application development experience outside of Unity. But I do have a lot of Java and mobile app development experience (separately - not in Android). I'm happy to pitch in on an open source StereoPi app for Oculus Mobile.

[Gol]

• Lower lag
  • I've found that UDP streams have much lower lag time than RTMP/RTSP streams
    The native android app has super low lag

Oculus application is the same as in Google Play, but with different manifest. So, video latency is the same, video transferred with UDP too.

[Realizator]

Awesome! Thanks for uploading it. I've loaded that StereoPi-OculusGo app, and it worked! Very cool to see yourself in 3d

Cool! You are the first one who repeated our Oculus experiment. Congratulations!

Wishlist for features

• Lower lag
  • I've found that UDP streams have much lower lag time than RTMP/RTSP streams
    The native android app has super low lag
  Head Tracking
  • The users of my application need to be able to look left and right in the fixed point of view frame.

As Gol mentioned, we use UDP and raw H264, not RTMP/RTSP. So I can not understand, if you wish to obtain even lower lag, or current latency is good for you? You mentioned this in a wishlist, that's why I'm trying to clarify this for myself.

As for other features - I think we can try to do this together.

@Realizator I don't have any Oculus Go application development experience outside of Unity. But I do have a lot of Java and mobile app development experience (separately - not in Android). I'm happy to pitch in on an open source StereoPi app for Oculus Mobile.

A-ha, sounds interesting. Can you help us to create a very simple application (like our current one) using Unity? We can provide you with all our video-related code and connectivity logic.

[wildag]

Cool! You are the first one who repeated our Oculus experiment. Congratulations!

That demo is the reason I bought the kit - so I feel like you delivered well!

As Gol mentioned, we use UDP and raw H264, not RTMP/RTSP. So I can not understand, if you wish to obtain even lower lag, or current latency is good for you? You mentioned this in a wishlist, that's why I'm trying to clarify this for myself.

Good information. I'll mess around with bitrate and resolution to see if I can optimize.

As for other features - I think we can try to do this together.

A-ha, sounds interesting. Can you help us to create a very simple application (like our current one) using Unity? We can provide you with all our video-related code and connectivity logic.

Yes, let's collaborate on an app. Feel free to share a git repository of what you have, and I'll try to find a workable framework. Then we can collaborate on making some basic features for it.

[Gol]

I also tried to feed StereoPi live video into Skybox video player on Oculus Go. It is my favourive video player for Oculus Go, but it can't play streams by default. Here is the first result:



I'll make a some tests and publish code. It's on Node.js

[wildag]

I looked at options for Oculus Mobile development SDKs. We have a couple of options.

1) Oculus Mobile Native - which is straight java with JNI bindings to C code. This would likely the be the highest performance option

https://developer.oculus.com/documentat ... ok-native/

2) Unity 3d with the Universal Media Plugin - this would allow for more rapid development of UI features, like menus etc. But it might not be the lowest lag option - as the Oculus has to perform some transformations on the video.

I'm considering engaging a freelancer to get the project started - as this is a bit out of my wheel house. I am sure that once the framework of the application is started, I can contribute to expanding the features.

[Realizator]

wildag, ok, now we need to find some experienced programmer. I'll try to find one too.

[wildag]

I put in a request for bids on upwork.com. Received two believable bids for $1,500. One of them is a group from Ukraine. I've invited that group from the Ukraine to view this forum and the StereoPi project.

They're proposing that the project be built in Unity 3d - which I think is a good choice. That would provide a starting place me and other groups to integrate what they create into more complex applications (think menu systems, overlay information, logos, etc).

I'm willing to cost split on this - but I imagine this would make StereoPi very unique amongst the other options.

[Realizator]

@wildag, I've putted help request in our last update. Let's wait for some time.
I'm also talking with some my colleagues now, and will try to find specialists too.

[orotot]

Hi all,

I think i was able to side load the StereoPi.apk to my Oculus Go - it is now showing up in the Unknown Sources menu. When i launch the app on the Go tho it is just a black screen. I presume it is a connection problem to the StereoPi.

1. is it a usb connection to the StereoPi?
2. did i miss something - i will go back and re-check the wiki
3. is is a wifi connection - that would maybe also make sense.

thank you !!

[Realizator]

Hi all,

I think i was able to side load the StereoPi.apk to my Oculus Go - it is now showing up in the Unknown Sources menu. When i launch the app on the Go tho it is just a black screen. I presume it is a connection problem to the StereoPi.

1. is it a usb connection to the StereoPi?
2. did i miss something - i will go back and re-check the wiki
3. is is a wifi connection - that would maybe also make sense.

thank you !!

ororot, black screen means that app is started, but StereoPi is not found.
1. No, it is not USB connection to the StereoPi. You need to have your Oculus Go ang StereoPi to be in the same network. StereoPi can be connected to the router by Ethernet (RJ45 connector) or WiFi, it does not matter.
2. I think you need just check network connection for both Oculus and StereoPi
3. For WiFi connection setup you need just to fill up 3 fields in stereopi.conf mentioned here. Set "wlan0", but not "wifi0", as "wifi0" settings is for built-in WiFi on the classic Raspberry (not for StereoPi).