Moving on from fixed position spherical video
Freedom 360
With reports suggesting that VR could generate $70 billion by 2020 [1], it’s not surprising to see major media companies publishing VR stories as fast as they can. However, telling stories using 360-degree spherical video poses some interesting problems, and there are a growing number of articles questioning the validity of 360-degree spherical video techniques for storytelling in journalism and documentary filmmaking.
As spherical capture systems see front, back, left, right, up and down, all at once (allowing viewers to look at the spherical panoramic scene in any direction they want at any time), it is not possible to focus attention using traditional film grammar. Audio content now plays an increasingly important role as a narrative guide – content and direction of the sound and audio narration, alerting the viewer where to look and what to focus on within the fixed position spherical scene. Some of the problems were highlighted in ‘The Displaced,’ a VR story from The New York Times, which sparked some controversy, as it was obvious that, because of the issues surrounding setting up and operating a spherical camera rig, many of the ‘scenes’ were necessarily staged.[2]
Other techniques of engaging the viewer, and telling a more enriched story, are coming into play. Nonny de la Peña, a pioneer in immersive journalism, has developed a form of reenactment-based VR journalism, with computer game-type virtual actors performing a scene driven by audio from 911 calls. FRONTLINE’s ‘Ebola Outbreak’ tells a richer story, not only by using a well-crafted audio narration, but also by inserting regular ‘flat’ interviews as overlays on a spherical VR background - a kind of hybrid VR approach. ‘6x9’ from The Guardian gives an immersive experience of solitary confinement in an American prison, and the recent PBS reenactment of the American Civil War, ‘My Brothers Keeper,’ is specifically staged for VR viewing. All demonstrate fantastic uses of spherical VR.
For me as an ex BBC documentary producer and veteran panoramic photographer, all this is very exciting. But the question is how can VR become usable in the more uncontrolled and unpredictable storytelling world of mainstream journalism and documentary production?
To explore this question, I have been working with a single camera One Shot VR technique that allows some of the more standard documentary production approaches to be used, while providing enough of the VR immersive properties to enhance the storytelling and keep the audience enthusiastic about the VR experience.
Section of the Rotunda, Leicester Square, in which is Exhibited the Panorama, 1801 (aquatint) by Mitchell, Robert)
With all the hype about immersive VR It’s easy to get the idea that it is totally new, but, in fact, it started quite some time ago. In the late 18th and early 19th century, Barker’s 360-degree art form became a mass spectacle throughout Europe and America. Of course, the fantastic thing about today is that these architectural experiences are now presented in a wearable headset, on a tablet and on a phone in our pocket!
Apparatus creating an experimental visual wide-field display. C is the center of curvature of a hemispherical screen. Numbers indicate dimensions in millimeters.
More relevant to the question about documentary VR production is an early VR photographic technique that shot still pictures with a single fisheye lens. This produced a circular image capturing a 180-degree hemispherical angle of view. The flat image on film exhibited severe geometric distortion, a result of the hemispherical field being projected on a flat plane. But when projected back through another fisheye lens onto a hemispherical screen (rather like the dome of a planetarium), all the geometric distortion was canceled out. When standing in the dome, a viewer had the experience of being immersed in the scene. Today’s version of this technique replaces the physical projection of the image with real-time computer mapping in VR applications, running in VR headsets and on smart tablets and phones.
Until recently, it has been more common to see images shot with the fisheye lens facing up. The hemispherical image gives a horizontal 360-degree panoramic view – around and 180-degrees up, from edge to edge, but not down – as can be seen in the pictures here of Wells Cathedral.
With the lens in the facing forward position, the hemispherical image gives a vertical 360-degree angle of view and 180 degrees left and right, but nothing behind. This orientation more accurately represents our normal viewing experience. We can see about 190 degrees left to right, without moving our head, and about 250-270 degrees, when we turn our head from side to side. Not a lot of time is spent looking behind us; we focus our attention on what is happening across the 190-270 degree forward facing angle of view in front of us, constantly scanning from left to right and up and down. (It is interesting to note that in ‘Ebola Outbreak’ the interview overlays all appear within this ‘normal’ field of view, even though the rest of the spherical image completely surrounds the viewer.)
I believe that the front facing hemispherical fisheye technique could become a primary VR storytelling format for journalism and documentary production and, therefore, a rich area to explore and develop. It is particularly suited to near field application, such as in demonstrations or where the visual context is important and close up, like is shown by these still frames below from an interview with a coffee roaster.
This sequence was shot with a 180-degree fisheye and shows the process. The fisheye recording is mapped onto a longitudinal hemisphere and then prepared as a distorted projection for a VR player to correct, as the viewer moves their gaze around the image.
The front facing hemispherical fisheye technique only requires one camera with one lens, ‘One Shot VR,’ as opposed to a multitude of cameras needed to produce a full spherical image. A huge advantage of this is that as the single lens is only looking forward, the camera can be operated very much in the way that it would be in traditional documentary production. It can be handheld, track action and be moved around freely, although these moves need to be mindful of potential nausea inducing affects for viewers wearing headsets!
We have yet to see how viewers take to wearing VR headsets on a regular basis. (3-D TV was plagued by a reluctance to wear much lighter 3-D glasses!). However, all of the VR apps also play well on tablets and smart phones without the need for headsets. The viewers navigate by moving the tablet/smart phone around or swiping with a finger; this is a much more readily available viewing experience than having to resort to wearable headsets, and one that is more friendly to the One Shot VR experience.
Another advantage is that One Shot VR requires much less post-production. For example, there is no need for multiple camera shots to be stitched together with all the associated complications of synchronization and color grading. One disadvantage is that the single image of One Shot VR has to be recorded at as high a resolution as possible in order to deliver acceptable image quality. The pixels are being spread over a larger area than they would for regular 16 x 9 presentation, therefore 4K is the minimum acceptable resolution for shooting. Also, with such a wide angle of view, scene exposure is difficult to control, so a much higher dynamic range needs to be recorded, in order to deal with extremes of brightness and darkness in the single shot. The recent arrival of small, high quality 4 and 6K video cameras has made these production issues relatively straight-forward to deal with.
While shooting exploratory documentary material with a 180-degree fisheye, I found the 180-degree left to right angle of view to be just on the edge of providing an acceptable immersive experience. However, the top to bottom angle of view actually benefits from less than 180 degrees, which also helps keep the feet of the camera operator out of shot!
(See https://youtu.be/qfKRzj0ale8 and https://youtu.be/7BJTA1mj8Hw)
Entaniya MFT 250
The arrival of a new 250-degree fisheye from Entaniya, shown here, has really enhanced the immersive qualities of One Shot VR. The horizontal angle of view now closely matches our own (at around 250 to 270 degrees with side to side head movements), although this extreme angle of view brings a few new issues in handling the camera! Despite this, experimental production of 250-degree One Shot VR documentary production is proceeding rapidly.
See the finished clip here
notes:
[1]http://www.recode.net/2015/12/3/11621154/virtual-reality-market-seen-actually-generating-70-billion-by-2020
[2]http://www.nytimes.com/2015/11/15/public-editor/new-york-times-virtual-reality-margaret-sullivan-public-editor.html