It bears repeating that, no matter how fine the technology, how complete the tool you are given, you will never be able to extract its quintessence if you do not rely on a distinctive vision, one that will make your deliverable a humble and original achievement, a pledge of performance or a world-wide success (in the case of the toughest or luckiest ones). So, driven by this spirit, we – the Babylonjs team and yours truly - have decided to showcase BabylonJS, our open-source engine, through a slightly odd and off-the-wall tech demo … 'MANSION'.
A mysterious event…
I still remember the first time we talked about it. One evening, David Catuhe called me and said he had a project: to have a 3D demo of the BabylonJS engine ready for a keynote at a mysterious event in the US. It was in October 2014 and we knew a tech show was in the works for January. We also knew that if we committed to this project, we were promising to deliver by November - plus a month for fixing and optimizing – a stable and functional demo, and above all one that highlighted BabylonJS as part of the overhaul of Microsoft’s internet browser … An engaging project, so to speak. Right away, we saw the parallels between our 3D engine’s roadmap and that of what is still called Project Spartan – spatialized sound and Web Audio support. Our demo now had to be about sound, even if we had to bleed for it.
So we started putting together working hypotheses: a roller coaster simulation? A wildlife park filled with dinosaurs? Yeah, sure, those were inherently good themes. But were they original? After all, these themes had already been dealt with by other creators. And we only had a month. I did have a budget for the “assets & design” part of the demo, but having to find in a hurry someone that worked in 3D design and to have him work on a top secret project, which necessarily implied secrecy about Windows’ next Web browser: all of this made for a difficult challenge. We had to find our own path. After a lot of brainstorming, we decided on a haunted manor with an unsettling and mysterious atmosphere, just like the event we were building the demo for…
A bit of AD
AD as in Art Direction, of course. It is where everything is decided. It is the Rosetta Stone that allows us to unify, to decrypt all the efforts to come. It is a long-term task spanning across the whole project, from the first ideas to the organization and the improvements at the end of the production process. It is the artistic binder of the demo. Once the themes have been defined, I start with a bit of AD: what’s the atmosphere like? A gloomy and rainy day, or a pitch-black and oppressive night? What’s the location? A Scottish castle somewhere near a Loch, or an old and abandoned shack?
The method I use to deal with such a situation consists in… browsing the internet. The aim is to feed my imagination. So, I gather a lot of documentation, I look around, I pick up on a word, I check what it means in German, etc. I quickly end up with a hundred-or-so pictures, a number that I trim down to 10 by getting rid of those that have a different context or a different tone than what I’m looking for (a Dutch colonial house, Amityville-style, for example). Then, I get rid of anything that would take too much time to build.
Once the frame of reference for inspiration has been set, we can start on the creative phase, and I take some time to explore certain ideas by drawing them. This is the story-board stage. This allows us to present early concepts to the Americans that are funding our project, but it’s also a way for me to get a good notion of the volumetry of the final 3D scene, and of how long it is going to take us to build. Quick reminder here: when it comes to BabylonJS, the team works in their free time. So, with that sketching stage, I get an idea of how many evenings I’ll have to work on the project.
I’ve got moon issues…
As you can see on the embedded pictures, the sketches focused very quickly on the relevance of sound. Place that one sound near the pond, that other one near the trees, etc. We already understood that 3D – that is, the visual aspect – would also be carrying another sense: hearing. My original intention was to get a good balance of sound source, in such a way that a blind person could progress in the scene and reconstruct the demo thanks to the noises and auditory feelings you encounter in the different areas of the scene. So I got in touch with Cédric Pons Polo, an audio content creator based in Toulouse, and asked him to create these specific assets and especially the background music.
On a different note, you can also see that initially the first 3D draft included both the interior and the exterior of the manor. But that proved to be too time-consuming, and we had to readjust the project’s ambition: we would focus on the night-time exterior of the virtual tour.
My first attempts weren’t all that great. To be honest, I had a rough start. And to be perfectly frank, I had another sketch in which the manor was surrounded by a French formal garden. This, on top of being hard to build (time-consuming, lots of details, lots of sounds needed to make the scene compelling), was also a bit dull: a flat garden with little to no lines of sight and a recurrence of the same objects and textures. In short, I didn’t like it. Nor did I like the art direction. The night, for example: should it be pitch-black and perhaps too dark to see anything? Or should it be more along the lines of an “American night”, a rather clear one, perhaps at the expense of immersion? All these hesitations became a real problem when I started designing our moon. I needed a strong light source for my manor, but what should that light be like? A huge green and grim moon like in Rocksteady Studios’ Batman game series? Or a small one, alone in a nightmarish sky? In the end, a middle-ground solution made the most sense: a huge pale moon that would also help us design a specific effect.
Production
3D artists also suffer from writers’ block. But in this case, committing to the project was rather easy. I started with the walls of my manor, building them up from a simple outline on the ground, and I worked my way up to the roof, progressively adding details such as crenels and missing bricks, … Having a substantial folder of reference pictures and sketches does help a lot during that stage. However, the size of the project and the limitations of the development model (monothread javascript) forced me to work on the optimization. For example, any wall that has bricks in it is a single object. The same goes for windows. After a few evenings, I had modelled the manor and started decorating it with noble materials (stones, bricks, slates …).
Yet the real challenge wasn’t building the manor itself, but creating a garden that would live up to our ambitions. We wanted a scene on par with modern standards, regardless of the display technology used. We needed trees, grass, gorses, shrubs, etc. And to be honest once again, our initial vision was to have trees that would be constantly twisted and distorted by the wind. All of this was already costly to display in terms of performance. Add animation to that, and … I did a few tests, but the framerate dipped so much due to our methods (mostly vertex blending) that we quickly abandoned the idea. We could, however, have bypassed the problem (with shader animation, for example), but time was of the essence, and we had to make decisions.
There’s no real tip when it comes to production, except stick with the art direction you chose and test your scene often to keep a good visual to performance ratio. For example, we had to make sure the edges between the gravel walkway and the neglected lawn weren’t too clear-cut. We could have used texture blending for that (blending two textures through a third one in grayscale), but it proved too costly. I chose to use a geometric seam between the grass and the gravel, a seam which had its own gradient texture. I also had to fiddle with opacity to get a cloudy, dark and stormy sky.
Because the ressources were online, we also had to make sure that the graphical assets were small-sized, although this specific notion was less of a problem since the demo was built for an event. That said, while working with David Rousset, we took the opportunity to integrate, in the BabylonJS plugin for Audiodesk’s 3d Studio Max, the inputs needed to link sound files with the objects that trigger them. Testing potential technical solutions during production is also part of our work. And while we were at it, we also started integration for our upcoming interactive nodal engine,so that I could add behaviors to my objects (such as playing sounds on click, for example) without adding any code.
Audio? Righto!
This is the stage where I integrated Cédric’s sound files, after modeling the objects, texturing them and animating them. Distant storms, insect noises, the sound of wind coursing through branches, screams of terror, rusted chains clinking together … All of this contributes to the distinctive atmosphere of the 3D model, including the death knell that provides a humorous response to a specific user action. Here again we had to make sure the sound sources were smartly spread in order to have an enjoyable demo.
Back to AD
We had gone a bit too far: for example, we placed a graveyard in the scene (according to our original AD) but we also wanted a bit of humor. To that end, I included gravestones for Professor X, Batman and Mini-Me for lack of a better idea. Obviously, I couldn’t keep those names in for a Microsoft demo. And that’s why only Clippy (our quaint paperclip assistant), Schrödinger’s cat and Hamlett are six feet under our current virtual scene. Oh, and IE6 too. We do love irony and self-flagellation in our team.
Based on an idea from Pierre Lagarde, I also added at the last minute a room inside the manor, accessible through a window, where you can catch a glimpse of Patrick, our zombie-skeleton, playing on the piano a personal creation. A way for us to go back to our original AD, if you will.
January 21th, and the few days before that …
We learned in late December that we were applying for a keynote at an extremely important event: no less than the international reveal of Windows 10 and HoloLens. We were entering the last stage of the project – stabilizing the model and chasing after the last few frames per second - when Microsoft’s editorial board broke the news for us: our demo would not be featured at the Keynote. For various reasons: a busy schedule, a lot of premium 3D content (with HoloLens) and a theme perhaps a bit too dark for the event.
Once we got over the initial disappointment, we decided to workshop our bad luck into a success, by making out of this virtual model the focal point for the first day keynote at the French TechDays. On top of that, David Catuhe would be on stage with us. We killed two birds with one stone by doing a world reveal of Project Spartan, the beta version of Microsoft Edge, Internet Explorer’s replacement. At the end of the day, we weren’t short-changed. Additionally, that event attracted a lot of attention, which allowed us to showcase the quality of Microsoft Edge and of Microsoft’s open source intiatives during the many events and tech meet-ups we attended.
When you’re given great tools, there’s a chance something great will come out of it …