Artificial intelligence (AI) can help add ‘4D’ effects to films, potentially making them a more exciting and immersive experience by allowing viewers to experience what the characters are feeling.
While there are already a small number of 4D movies that add a physical element, such as wind, splashes of water, light, and shaking, researchers from the University of Toronto are developing a way to apply such features to everyday 2D and 3D films.
“Right now all these effects are created from the first phase of production,” said Yuhao Zhou, a fourth-year undergraduate in the university’s department of electrical and computer engineering. “We’d like to automate this kind of process for movies that were not originally created for 4D cinemas.”
A paper on the project, Now You Shake Me: Towards Automatic 4D Cinema, says that the team aims to detect which effect is being applied to each of the characters in the scene (or on camera), and also predict accompanying details such as the intensity of each effect, its duration and possibly direction.
For classification of physical effects, Zhou said their neural network, a function of machine learning that allows deep analysis and learning of data, extracted features from a short clip, including movement and audio. For detection, the neural net can predict what the effects are, and where they occur, in a long video clip.
“You don’t only want to know what happens to a character in a particular shot. You want to be able to say, “[the effect] is wind now, not only because I see the wind right now, but [because] it was probably windy before,'” Makarand Tapaswi, a postdoctoral fellow of computer science also working on the project, explained.
The work could pave the way towards 4D cinema in everyone’s homes, the researchers believe.
Tags: AI, Artificial Intelligence, 4D cinema
