3DVF : You said earlier that you really wanted to push the visuals of the project, and you also use a lot of effects. All of this can create performance issues, was it hard to find the sweet spot ?
Areito Echevarria : We had a technical designer with a background in videogames, his role was to tell us… What we couldn’t do, while our role was to tell him what we wanted to do !
So this was this natural tension : since we have a VFX background we like everything to looke like high quality, so we would put everything in, and he would put everything out ! [laughs]
Raqi Syed : This was a long process, for example we wanted very realistic skin shaders with subsurface scattering to make the skin come alive, and our level designer told us : “you can’t do that”. He would show us how the framerate dropped, and we knew we had to hit 90fps. We would always try to evaluate the ressources of various elements : using another shader, simulating the hair or the skirt, having more than one shadow-casting light… We had to make some difficult decisions regarding lighting and art direction.
3DVF : These performance concerns probably also had an impact on the animation ?
Areito Echevarria : Yes. We would block out the staging, iteratively, over a fairly long amount of time. We did a series of motion capture sessions with just ourselves playing the characters, or with our students playing the characters, this helped us get a sense of where the layout needed to be. Towards the end of the project we brought in Frankie Adams and Allan Henry to do the final mocap performances. For those just we did full body and facial capture.
Then, there were some decisions about how much facial refinement to do. We did a couple of levels, a basic level and higher detail level for the more impacting moments. That was a really hard choice to make, we did not have the time or budget to do some high quality facial animation on everything. Hopefully our choices worked out !
3DVF : There are several French companies involved, such as Floréal Films, Small Studio, Albyon. How did you get in touch, and how did you work with them ?
Raqi Syed : We had met Avi Amar and Katayoun Dibamehr (both producers of Minimum Mass) pretty early on when we did a Lab in L.A. (called DevLab by Kaleidoscope). They wanted to come on and support the project. It’s really the two of them, Avi and Katayoun, who pushed for the CNC funding and formed these relationships with Albyon, Small. It was really cool to work with Small : we speak the same language since they come from a VFX & animation background, and they are so good at what they do. It was nice to be able to work with them and add that final layer of polish.
Areito Echevarria : They came at the end of the project, they did a fantastic job. It was really fun ; working remotely is challenging, but everything was really smooth.
Raqi Syed : Albyon has already worked with companies like Atlas V on Battlescar, so they’re used to working with international teams. It should also be noted that in VR, the international funding has become the nature of how we get things done, even before Covid.
3DVF : During the Annecy Festival, you explained that Mimimum Mass was heavily influenced by the work of the photographer Todd Hido. Why do you like his work, and how did you use this inspiration ?
Raqi Syed : We knew that we wanted to have what we call “gritty surrealism”. The surreal element is partly practical, since as we said we had to limit the footprint of it, but we are also inspired by the films of David Lynch, and we love the idea of creating a surreal space that’s not dystopian.
When we found the photography of Todd Hido, it just really spoke to us, with a perfect balance between surreal, suburban landscape, and his style, very noirish, underlit spaces with bright windows, that really speaks to the strengths of the real-time engine : emissive objects are really cheap to use in terms of performance. And noir is great, since you have a single shadow casting light, it’s all very dark and moody.
This was a natural fit, both thematically and from a practical perspective.
Of course, our own work is just an interpretation of his beautiful and inspirational work.
3DVF : Could you also tell us about the creation of the effects, especially the fluids and the creation of the Void ?
Raqi Syed : Our effects were mainly designed and done by Sonya Teich who has a vfx/feature films background as well. She, like us, was learning how to translate that style into real-time.
She’s a Houdini artist so she used this tool to create the sims, then she worked with ou level designer to translate it in real-time, mainly using vertex animation with shaders I think : this was cheaper than using techniques such as Alembic caches.
She was really experimenting and trying to push the render engine as far as possible to get something cinematic-looking.
3DVF : Looking back at the whole project, what did you learn, what was the most challenging part ?
Areito Echevarria : The bigger takaway, and it’s a bit of a cliché, but VR is very hard, it’s a stubborn medium !
I think we learned to understand that VR really is its own medium, with its own strengths and weaknesses. The challenge was coming to terms with that, to learn that VR has its own language. But once you embrace that, it’s fantastic.
The physical proximity effect, which I talked about earlier, is really powerful.
Raqi Syed : I also think that following our instincts was a good idea. There were lots of things people told us we couldn’t do : no more than two character in any given scene, 3 realistic characters is insane, you shouldn’t do FX like this… And also the kind of stories that you can tell in VR : we knew we wanted to work in this tradition of indie cinema, tell the kind of story that we would love to watch at Sundance, and we wanted to see if a VR experience could be similar to this kind of indie film. And I guess the answer is yes !
3DVF : You both worked in the VFX industry in the past, why did you choose another career and join the Victoria University of Wellington ?
Areito Echevarria : As much as I love the visual effects, for me it was kind of like an internship to learn the craft. I always wanted to tell my own stories, I had spent enough time learning the skills, it was time to make my own stuff.
I also think that being a researcher at the university is a great enabler of that, Victoria has been fantastic and very supportive, helping us to go where we wanted to go. It’s a great place to be, and to give back to the community of course, through teaching.
Raqi Syed : I also feel that it’s a natural progression. I saw the feature film/visual effects industry as an “extended graduate school”, where I was learning something that might have another purpose to it. Having learned all those things, it’s kind of a superpower you have, and you can use it to tell new stories, other kinds fo stories.
The university has been very supportive and helped us re-evaluate what we can do with VR and even VFX. What we can do with those tools that has not be done yet.
3DVF : Areito, you won a Technical Academy Award for your work on deep compositing. What was your contribution to this technique ?
Areito Echevarria : This project came about during the production of The Day the Earth Stood Still, with Keanu Reeves. Basically, we had sort of reached the limits of what was possible with traditionnal compositing, at least in terms of managing the complexity of layer-based solutions. It was just becoming impractical. We started prototyping this idea, “what if we could do z compositing in a better way, more efficient and high quality way”. I was involved with conceptualizing this idea, and figuring out how it would work from a technical point of view.
And at that point, when we saw that it could work, that you could put the layers in any order and let the system work out the pixels, it was a revelation. In hindsight it can seem obvious, but at the time it wasn’t.
Weta very rapidly switched to this deep compositing workflow, and at that point it went to a developper, and the developper turned it up to a fully-fledged toolkit.
My contribution was mainly focused on the iteration and prototyping stages, where we were trying to figure out if this was a good direction to explore.
The tools evolved a lot since then, now it’s pretty incredible.
3DVF : Raqi, we saw on you LinkedIn profile that you specialize in “virtual reality research, media design through visual effects, the mediation of technology through race, class, and gender”. Did that last part come into play in the production of minimum mass, since the subject is related to gender ?
Raqi Syed : I’m definitely interested in telling stories that are told from women’s perspective. In VFX, you don’t see a lot of women in creative roles such as writers, directors, producers, VFX supervisors. I think it’s a question of opportunity, this is one of my areas of interest and I’m pushing my students to do that as well.
Other issues are also interesting : in my lighting class I do a lot of work with what it means to actually decolonize disciplines in VFX. From lighting, we look at traditions of realism in New Zealand, colonial painting, we’re even looking at digital humans : many people ask what we do with digital humans, what they are for, but the question we don’t ask is who gets to be a digital human. Even with Minimum Mass, we wanted to explore creating digital humans procedurally. We used some off the shelf software to design those characters and to push their design. They’re kind of a version of us, they look like us, so the question becomes what does it mean for an unkown, non-famous person of colour to become a digital humain and then tell their story. It’s like an ownership of technology and storytelling that come together, and I feel that’s an important part of pushing the independant part of VFX forward, so that VFX aren’t only used for big-budget movies with important actors. We want to open that up, tell all kind of stories, create digital humans for all kinds of people.
We’re actually taking our students through the same process we used on Minimum Mass, with the same pipeline of character creation, lighting, rendering. This way our students can tell their stories.
3DVF : One last question : at the beginning of the interview we talked about a sequel for Minimum Mass, do you also have other projects in mind ?
Raqi Syed : Yes, I’m working on another project linked to this idea of digital humans, it’s actually about my father, my dead father : I want to explore his life story, his relationships to our family, and to push even further the photographic quality of what we can do with digital humans in VR.
Areito Echevarria : I’m working on another VR project, which is sort of an experimental documentary about empathy. We do some computational modeling of empathy, we try to understand how people relate to one another.
For more information