Lessons From The WonderRoom

In December 2024, my team and I worked on an immersive show for the UN’s COP16 Conference in Riyadh. It was a six-and-a-half minute long cinematic immersive experience called ‘Our Land. Our Future. The WonderRoom Experience’. 

The name “WonderRoom” is more than a title; rather, it’s an immersive concept we’ve been developing in our lab for several years. At its most basic, a WonderRoom is an enclosed room clad entirely in LED tiles with hidden speakers and no chairs for the audience. The walls, ceiling, floor - all of it covered in LEDs and the audience has to stay standing. A few LED distributors have experimented with setups like this but haven’t pushed them very far. We think there’s something hugely promising with this idea and if we can figure out how to do shows in rooms like these then the results could be game-changing. Along sci-fi lines, we wonder if the WonderRoom is a step towards creating a holodeck…

The WonderRoom concept is still quite new, which brought a lot of risk to this project. We’re experienced with rooms full of projectors but only a handful of times have we dabbled in LED spaces and we knew there’d be differences to sort out. Skipping to the end, the show proved to be the hit of the conference and yielded hundreds of wonderful articles online. However, our efforts at tricking the eyes and ears of the audience in such a way that suspension of disbelief was created, produced mixed results. 

We created the entire show as an animated CGI short film and told the simple story of a family of Saudi goat herders whose livestock overgraze their pasture leading to terrain degradation. (Land degradation was one of the COP16 conference’s themes.) The net result is a 6-7 minute long short that was filmed as a continuous drone shot over craggy Saudi cliffs, through sand storms, beside emaciated goats, and periodically over dried-out riverbeds and date trees. We built the Saudi environment in Unreal Engine 5 and created a virtual 360 camera rig to zip around the landscapes while filming it in an equirectangular format. The room itself was the shape of a large loaf of bread: about 5m x 5m x 9m with a raised pedestrian bridge running down the centre from entrance to exit. The ceiling and all four walls were entirely covered in pixel-pitch 2.7mm LED tiles but the floor was not, so we had five separate display surfaces to contend with. 

Using the equirectangular footage we captured within the video game environment, we scripted some code to chop the footage into five different shots that were stitched together and then finally mapped on the five LED surfaces in the room. And again, the audience stayed upright for the duration.

This is what laboriously stitching the chopped images looks like – in our partner's VFX studio in Cairo:

Get it right, and the result is hugely compelling. Get it wrong, and audiences run for the exits. Here's a clip from before we opened where you can see what correctly mapped content looks like against what's incorrectly mapped:

The sense of motion was very real and led to moments of vertigo. The vertigo was triggered when the camera either flew too fast or took flight such that the horizon line moved up or down too quickly. Because our camera rig was flying ‘forward’, audiences naturally turned away from the “wind shield” wall and preferred watching the landscapes zoom past them on the “rear window” wall, which had the double benefit of provoking less vertigo and while allowing more time to look at the landscape or characters.

Audio-wise, we mounted an array of anaconda speakers along the handrail of the pedestrian bridge and placed about twenty-five subwoofers under the bridge. The engineers were successful in tuning the room and the exceptional soundtrack by Mark Camilleri of Imagine Sound Studios was the talk of the conference. The show benefited from being in a small WonderRoom, though.  A bigger room would have required a more sophisticated mix with audio variation in different areas.

The most interesting lesson – and likely the key to a truly successful WonderRoom show – is how the video content worked in relation to the corners of the room. If an image crossed from one wall to another without bending or warping, then audience trust in the show grew – and that mattered. In particular, it mattered from a story-telling point-of-view because quickly fostering audience trust and putting them at ease is crucial to deepening engagement and generating the ultimate goal: suspension of disbelief.

The lesson is that this trust wasn’t built on the creation of an illusion. We didn’t need to make the corners disappear. Quite the opposite. In much the same way that conventional 2D projection mapping is much more exhilarating when the artists play with the architectural surfaces, finding clever and whimsical techniques to deal with the corners was equally as effective.

What do I mean by “play with the architectural surface”? When a video is projected against a wall that isn’t uniformly flat then images will warp around the wall’s contours and reliefs. However, when the artist incorporates the wall’s specific contours and reliefs into the content itself, a new effect is created and audience engagement deepens.  Here's one of my favourite examples from a few years ago by a company working in Italy"

In Riyadh, the same thing happened when our digital content incorporated or 'played with' with the architecture of the physical room. In our case, there wasn't much architecture to incorporate but we did have an open doorway at each end of the room and corners where the walls joined. Here's a simple example of how we incorporated the doorframes at the beginning of the show:

When moments like that happened, audiences were immediately delighted and paid closer attention. During moments and scenes where we didn't successfully play with the architecture, audiences shifted on their feet and bewilderment set in. Where to look? What to do? But when a character climbed over the doorframe and slid down to the ground, it drew smiles and wonder. At another point, we placed a palm tree in a corner between two walls and a character quickly swung from one wall to another. Again, smiles and engagement.

We blundered into playing a game with the audience: if they could see the corners, then could we? And if we could, how clever were we going to be? This became the way in. Suddenly, audiences knew how to engage with the material; it was a form of active mental participation, rather than passive. This is important if the immersive production isn't a seated experience. If it's passive - give the audience chairs. If it's active (even trivially so, as was in our case) then the audience will put up with standing and, in fact, prefers it. 

This points to Immersive’s key element, something I've mentioned in other posts as well as being the title of this blog, which is positioning the audience in the same frame as the story.  If the audience can see it, then so must the characters. And if the audience can’t see it, then neither can the characters.  And that includes digital characters, too. A wonderful and wondrous WonderRoom revelation.

All-in-all, did we succeed in creating a full visual and auditory illusion? No, despite the show’s popularity.

But we’ll get there.

And now we're that much closer to knowing how.