In April, Facebook announced it had built a “Surround360” 3D-360 video capture system, but that it did not plan to sell it. Instead, the social networking giant promised it would open source both the hardware and the Ubuntu Linux-based software used to stitch together images from the camera into stereoscopic 360 panoramas. This week, Facebook did just that, posting full specs and code for the device on GitHub.
The Surround360 incorporates 14 wide-angle cameras with 4-megapixel resolution and global shutters, arranged in a ring-like design. The cameras can output 4K, 6K, or 8K video resolution. There are also two fisheye lenses on the bottom and one on top to complete the 360-degree immersive experience. The rugged aluminum chassis sits on a pole, which is masked in software so it is invisible when you look down with your VR headset.
The device is built entirely with off-the-shelf hardware that can be ordered online. Facebook offers a cartoon-like manual that promises to help you build the device in as little as four hours.
Facebook is motivated by the need to create content not only for its Oculus Rift VR headset and Samsung’s lower-end, Oculus-infused Gear VR, but also its own Facebook.com ads and user-generated content. In its announcement, Facebook said it wanted to “accelerate the growth of the 3D-360 ecosystem,” and that “anyone will be able to contribute to, build on top of, improve, or distribute the camera based on these specs.”
Well, maybe not just anyone. The parts cost about $30,000 — much more than many 3D-360 cameras, such as the dual-fisheye, $350 Ricoh Theta S or the soon-to-ship, 16-camera GoPro Odyssey, which costs $15,000. Yet, it’s cheaper than most professional-level models, such as the spherical, eight-camera Nokia OZO, which sells for $60,000.
Linux Software Vastly Reduces Processing Time
The major benefit of the higher end cameras — and the Surround360 in particular — is not only quality and durability, but much shorter processing time stitching videos into a seamless whole. The open source Linux software “vastly reduces the typical 3D-360 processing time while maintaining the 8K-per-eye quality we think is optimal for the best VR viewing experience,” says Facebook.
The task is daunting considering the amount of RAW video data involved – the Surround360 can capture 120GB per minute at 30fps, or 240GB at 60fps. To stitch the disparate views together, the system uses an optical flow algorithm to compute left-right eye stereo disparity rather than using tedious, manual “hand stitching.”
Optical flow lets you synthesize views separately for the left and right eyes, as well as flow the top and bottom camera views into the side views, and flow pairs of cameras views to match objects appearing at different distances. As a result, the camera can maintain “image quality, accurate perception of depth and scale, comfortable stereo viewing, and increased immersion,” says Facebook.
The Surround360 can be controlled remotely from any device with a web browser. The Ubuntu Linux 14.04 desktop used for processing, however, must be top-of-the-line, and capable of a 17Gbps sustained transfer rate. An 8-way level-5 RAID SSD is required to keep up with the isochronous camera capture rates.
Playback is currently optimized for the Oculus Rift and smartphone-enabled GearVR. On the latter, 8K playback requires a “dynamic streaming” feature that adds a “noticeable lag,” but is still acceptable, according to a hands-on report from TechCrunch. Presumably, you could also use other headsets, and you can view panoramas on computers with 2D displays without the immersive vertical views.
Facebook is a major consumer and producer of open source, mostly Linux-based software, and has previously experimented with open source hardware in its Open Compute initiative for servers. It chose not to open source the Oculus Rift, however.
Razer’s OSVR HDK 2.0 headset is open source, as well as Google’s much lower end Cardboard. The other major open source VR effort is OpenVR, an open source version of Valve’s SteamVR software, which forms the basis for HTC’s commercial Vive headset.
The Surround360 is not the first open source panoramic video camera. Elphel’s $60,000 and up Eyesis4Pi is a panoramic, stereophotogrammetric rig that incorporates 24 Linux-driven Elphel cameras with 5-megapixel resolution. After stitching, this results in a panoramic image resolution of 64 megapixels, says Elphel. A similar Elphel panoramic camera was mounted on the first Google Streetview cars before being replaced with an in-house design.
