My short VR film The Pull premiered around a month ago. Originally released on four main VR platforms (MilkVR, Littlstar, VRideo and YouTube), it has since been picked up as a first wave of guest creator content on Jaunt, and now can be accessed using pretty much any 360°/VR viewer – from Google Cardboard to HTC Vive. It has also been featured at a number of screenings around the world, most recently at Berlin’s re:publica festival and conference.
I’ve been also demoing The Pull in person at a number of events, and found out that a number of topics keep coming up in conversations with the audience. This article is a summary of some of the thought and process that went into the making of this VR film.
What would I like to learn today?
With virtual reality being such a young discipline, everyone is still learning. Every new project, every experiment, are a chance to figure out another piece to the puzzle. In Prismatic I challenged myself to create depth and perception of infinity while confining the set to very small physical size. With FreeFall I wanted to test the safe limits of simulation sickness – how to disorient the audience without making them physically sick. The Pull was an opportunity to try some new stuff.
The initial principle was simple: I wanted to create an experience that truly works only when watched in VR – something that can’t fully come to fruition in traditional flat-media. This meant incorporating all the elements unique to the medium: directionality, depth perception, and the feeling of presence and crafting an experience that actively engages all of them.
Fair Exchange
Another thing I talk about a lot is the idea of fair exchange and mutual respect with your audience. While for many enthusiasts wearing and HMD is quickly becoming a second nature – for most people it’s still a rather cumbersome process. In order to properly view my work, I am asking the viewer to:
- Own appropriate hardware and have it correctly set up and configured
- Download my work, either through an app or for use with a generic VR content player
- Strap a bulky head-mounted display to their face
- Wear headphones
- Possibly deal with a mess of tangled cables
Compared to traditional media, that’s a lot of involvement I expect from the audience. If after all that trouble I end up showing them simple monoscopic 360° content, or a poorly stitched video with seams all over, or pixelated, ungraded footage – that constitutes a serious breach of trust.
The way I see it – the audience met me half way and I need to make sure to respect that and give them the experience worth all the additional trouble. My work has to look stunning, be interesting and fun to watch and leave a lasting impression. When planning and then creating The Pull, I always kept this idea in my mind.
Giving up control
I’ve been thinking a lot about the subject of control of the VR experience – of how the director must accept a new relationship with the viewer. Without the use of a frame, without the ability to crop a section of the world for the audience to focus on, we (authors) can now only suggest and propose the point of view – but never enforce it.
It’s not an easy shift (especially for the perfectionists and control freaks among us) but just as any good director learns over time to trust the judgement of their crew and actors, I think eventually it will become a second nature to our creative process.
In The Pull I decided to go one step further and give up the control over the scene to physics simulation. I always had a general idea of what I wanted to see in each of the scenes, but limited my own influence to modeling the objects (and their low-poly collision meshes) and setting up the simulation program. You can actually read the program itself on the title cards that drop before each room.
The code above is the program of Experiment C, looping in the video above. It takes place in the room with the footprint of 350x500cm and 300cm in height. There is a total of 21 items generated for the simulation.
The first gravity pull will happen in the top direction, accelerating at 300cm/s² for 10 frames. Next one will pull towards bottom, at the same value, but only 9 frames, and so on. K stands for back, F for front.
The physics are simulated at 60fps and played back at 30fps.
Virtual Insanity
Sometimes the sim would not produce the results I was hoping for. In most cases I just rolled with the result or tweaked the program. In one case, however, I wanted to achieve something very particular – a vortex of furniture, dishes and utensils for Experiment D. What I did not account for was that tugging the gravity in different directions will create movement, but not the rotation.
After several failed attempts I was about to resort to introducing aerodynamics into the simulation, when I remembered one of the oldest tricks in the book. Commonly known as that cool trick from that Jamiraquai video – the idea is very simple: if you cannot get the objects in the room to behave a certain way, attach the camera to the wall and move the entire room around the objects.
Spin Doctors
(Am I going all too 90’s with chapter names?)
On the subject of spinning – originally I transitioned between the rooms using a simple fade through black. It was certainly a safe way of changing scenes, but after 2 or 3 transitions – got pretty tiresome. It established too much of a monotonous rhythm – light go on and off, rooms fade in and out, and so on.
I decided to try the cheapest trick in the book and used the offset effect on the renders, effectively rotating the rooms on the Y axis (yaw) while cross-fading between them. It made the transitions much more dynamic, but induced simulation sickness for many. As with many problems in VR, I looked for an answer in game development.
I recalled how nearly a decade ago, DICE had a similar problem to overcome. Their game Mirror’s Edge was a first-person parkour simulator and constant shifts in the movement caused beta testers to suffer from major motion sickness, even on the traditional 2D screens. The solution DICE came up was very simple – they placed a single dot in the center of the screen. The dot remains constant in relationship to the player’s head, even if the world rendered on the rest of the screen tumbles around. Developers of Mirror’s Edge claim they, in turn, got the idea from interviewing professional ballerinas to see how they perform spins without feeling sick.
The title cards for each of the experiments are my version of DICE’s dot – an anchor the viewer can focus on while the rest of the scene spins in transition.
A good portion of people watching the test footage felt the titles solved their simulation sickness problem. Some still would miss it though, looking elsewhere in the scene. I needed to gently guide them towards film’s North and make sure the title card was within their field of view before the room begins rotating.
My solution was to render a simple spherical wipe animation that starts in the South of the scene (behind the viewer) and slowly progresses towards North (front). Using this transition as a soft matte I then darken the affected regions by approximately 10% and desaturate them by 25% or so. As a result – the areas I hope the viewer not to look at become duller and less attractive than the unaffected northern regions – in most cases motivating the gaze to travel towards the front, thus catching a glimpse of the titles, thus getting anchored onto them, thus avoiding simulation sickness.
Two Colorful Discs
I get asked a lot about whether The Pull was inspired by works of James Turrell. Sadly, I have been unaware of his body of work and never seen any of his exhibitions – something I feel I need to rectify as soon as possible, as his work is mindblowingly amazing.
Two things that inspired the film are both round, flat and emit lights associated with sound expecting you to recognize the patterns. They are: Simon Says toy and the spaceship from Close Encounters of the Third Kind.
As with both of these references – the experiments begin with a simple sequence, meant to teach the viewer about the rules of the world I created. The sequences become increasingly more intricate with each room, and just when you feel you understand how everything works… Well, the VW Camper Van happens and breaks the continuity.
Show, don’t tell
At this point in time I feel nobody really solved how to naturally narrate in VR. Addressing the viewer directly tends to be very cheesy, regardless if done done in first person by actors or in the third by the narrator. Having the story spelled out indirectly feels like a throwback to the early days of movies, when every theater had a speaker explaining on-screen action to the audience.
In The Pull I decided to abandon traditional narration altogether, in favor of environmental storytelling – leaving enough breadcrumbs for the viewer to weave their own story. As an author, I focused on creating a believable space and mood, leaving the interpretation to all of you. It’s more fun this way.
If you made it to the end of this article and feel like I did not answer questions you may have – feel free to ask them in the comments section below. I’d be happy to continue the conversation there.
Hard Data
If you enjoyed reading this article, you may also want to have a look at another one, titled View Heatmaps: New Standard in VR Video Analytics. In it, I further explore certain design aspects of The Pull – this time based on statistical data collected from thousands of viewers on Jaunt platform.
Having recently created a game without rules I have to agree with you that given the opportunity, many of us really enjoy the freedom of discovering our own narrative. Even those who initially look irritated at the idea that the responsibility is now theirs, quickly discover that they are more imaginative than they thought. I love the work that you’re doing to push the boundaries of our perception – good times.
Thanks Louise, I appreciate your kind words 🙂