Wednesday, February 17, 2016

The Narrative VR Rulebook: An Academic Approach to the Language of VR Storytelling - PART ONE

What follows is the first part of a a two-part article in which I explore the aesthetic potential of virtual reality (VR) storytelling, as extrapolated from what we already know about cinema.

VR is here! Hooray! Now what? We’ve been telling stories on-screen for a century - how do we extend that cinematic tradition into the virtual experience? There are intrepid filmmakers out there – pioneers – who are seeking first-hand answers to these questions. They’re experimenting, and the results of their work will shape the course of VR storytelling. But we don’t need to experiment blindly. I’d like to suggest an approach to VR storytelling that begins with what we already know, and that extrapolates, almost academically, a new set of parameters to guide us through the VR cinema frontier.

Let’s face it: Typically, Film Studies has very little to say about the actual craft of making films. The academic study of films is the study of the result of filmmaking, not of the process. But I contend that film studies has a lot to offer filmmaking. We’ve seen this in action when the first generation of film school filmmakers arrived in Hollywood in the late ‘60s and early ‘70s. The results of their work still inform and inspire us today.

So, let’s see what we can learn about VR cinema from a typical film studies course.

Part 1: History Repeats Itself

The very first movies (especially those of the Lumiere Brothers in France) were observational. For these “actualities”, a camera was plunked down someplace interesting, and audiences were shown that interesting place. Sound familiar? The vast majority of VR video content today does exactly the same thing. We’re invited to look around at a place, and that’s it. No story, no message, no meaning beyond the novelty of the immersive observational experience.

Next in the course of cinema’s early history, a few innovators came along and saw the potential for storytelling. In the US, Thomas Edison’s company filmed scenes from famous stage plays (see: “The Kiss” d. William Heise for Edison, 1896). 

These early narrative films are “stagey”, with much of the action presented on a sort of proscenium, as if the camera (the viewer) is in the audience of a play. Within a few years, filmmakers such as Edwin S. Porter figured out how to string staged scenes together to tell a more complete narrative, and to place the camera in places that no longer mirrored the audience’s theater experience (see: “The Great Train Robbery” d. Porter for Edison, 1903 - the camera is placed on a moving train for some scenes!) 

Across the Atlantic, Georges Melies saw in the technology opportunities to achieve effects that could not be achieved on stage. Using jump-cuts and other tricks, he pioneered the use of special and visual effects in narrative cinema for “Voyage to the Moon”.

This is the innovative moment where most VR storytellers find themselves today. VR production is still technically limited except in the most sophisticated (and expensive) scenarios. VR cameras usually don’t move, so the action that they record is typically staged around a fixed point (much like a play is staged assuming a fixed audience perspective.) When VR cameras do move, it’s because they’re affixed to a moving object (such as a car or motorcycle… or train!) In addition, the language of editing in VR is still in its infancy, so most scenes are played out without a cut. Multiple scenes might be strung together, but each scene is typically a one-shot experience - just like so many of those early movies eleven decades ago.

Of course, the use of visual effects plays heavily in the VR world. Unlike the early days of cinema, VR’s infancy includes a vast array of gaming and practical applications for the technology. There seems to be more digital/interactive VR (not to mention everything that’s happening with augmented reality) than “cinematic” VR. So it’s not surprising that there’s overlap and interplay between these worlds. Some of the highest-profile VR movies today (See: “Help” d. Justin Linn for Bullitt) feature mostly computer-generated worlds and plenty of computer-generated magic. Just like early filmmakers who applied what they knew from the stage to the new cinematic experience, VR pioneers are applying the techniques of cinema to the new virtual medium. The incorporation of cutting-edge visual effects is part of that adaptive process.

But there’s something else happening here – something a little subtler. The great early filmmakers posed a specific question with their work: What could movies do that theater couldn’t? Melies explored innovative effects in “Voyage to the Moon” - his moon creatures disappeared in puffs of smoke, a purely cinematic effect, impossible to achieve on stage. Porter did it with new perspective choices in “The Great Train Robbery” - he put the camera (and thus, the audience) on a moving train, and later used a close-up to surprising and dramatic effect. VR filmmakers are doing the same thing. They aren’t just adapting old techniques - they’re searching for the special thing that makes VR different – the thing that “flat” or even 3D movies can’t achieve.

So, what is the purely virtual effect? What sort of magic can be accomplished in VR that simply can’t be done in traditional cinema? That is the ultimate question for the new medium!

Once again, history’s repetitive nature offers clues. After the early pioneers established the use of cinema as an observational and then as a narrative tool, and once they determined what “magic” cinema could accomplish, a second wave of innovators emerged. They began to synthesize all the bits and pieces to create a “cinematic language”. The German Expressionists (Murnau, Lang) learned how to employ non-literal depictions to evoke mood (with stylized sets, evocative lighting, etc.)

Scene from Fritz Lang's "Metropolis" (1927)

The Russians (Vertov, Eisenstein) learned how to use editorial juxtaposition to convey meaning (it’s the sequence, not just the shot, that brings richness to narrative).

Images from the Odessa Steps sequence in
Sergei Eisenstein's "Battleship Potemkin" (1925)
The Americans (Griffith, De Mille) learned to orchestrate and guide the viewer’s attention to such an extent that the viewer forgets that the flickering images aren’t real.

A close-up from D.W. Griffith's "The Birth of a Nation" (1915)

The American “invisible aesthetic” soon emerged as the dominant aesthetic in cinema around the world. That’s when we learned to edit so nobody notices the cuts, to shoot so nobody notices the camera angle, and all the while, to evoke emotion using the very tools we’re camouflaging. This is the direction that cinematic VR will take in the coming years: a synthesis of tools, and a development of language, syntax and grammar that is altogether unique to the virtual experience, and that allows the viewer to “get lost” in the virtual narrative.

How do we create narrative VR experiences that transcend the cinematic? How do we allow the audience to settle into the virtual story (without them feeling us yanking the marionette strings)? Some of the answers may only emerge from the discoveries of today’s VR pioneers, but we can also extrapolate guidelines for cinematic VR by looking closely at how and why “old-fashioned” cinema works. Using film theory, we can develop the new language, syntax and grammar of virtual cinema.

What’s a good movie article without a sequel? We’ll tackle the rest in Part 2, here:

No comments:

Post a Comment