Can a machine edit a documentary?
That's the question the installation 'Reflector' by artists Piotr Winiewicz and Dawid Gorny is trying to answer from a both technical and philosophical point of view.
The project hails from the Copenhagen-based production company Makropol, founded by producer Mads Damsbo and director Johan Knattrup Jensen, specialising in cinematic XR productions, from VR films to immersive installations, live performances and video projections.
Driven by a desire to explore the future of film and new technology, Mads Damsbo and Johan Knattrup Jensen leaped at the chance to experiment with artificial intelligence when Piotr Winiewicz came to them with the idea of creating an AI engine with the ability to edit a film in real time.
He named the AI engine Kaspar – named after Kaspar Hauser – and has used it to edit 60 hours of raw footage from an unfinished documentary about a Pakistani orphanage by the American director James Longley.
The result can be experienced in the installation 'Reflector'. Here, audiences can witness Kaspar at work: While participants are adjusting four parameters – brightness, number of characters, framing and dynamic – the engine will assemble the cuts accordingly.
For Mads Damsbo, Kaspar is interesting on three levels. Firstly, the installation plays into the company's experiments with physical film experiences that engage the audiences.
"By letting the audience into the machinery of this film brain, they may understand it better – because it is a bit cryptic," he says.
"Secondly, we hope to create a debate about the ways new technologies influence our film language, and which questions we need to ask ourselves about cooperating with AI in the future. Can the AI create art? Or something else valuable? And thirdly, it's contributing to creating a foundation for an applicable tool."
The aim is not to replace human editors, stresses Piotr Winiewicz. But in time, Kaspar might assume the role of an efficient assistant that can go through one hundred hours of footage in 10 minutes, categorise clips and suggest sequences.
"Every editor says that 60 percent of his job is just getting rid of material. I think this could make the work more pleasant and expand your creativity by reducing the labor," he says.
First and foremost, Piotr Winiewicz sees Kaspar as an artistic tool that can help filmmakers look at their project from a different perspective and challenge their familiar patterns.
"For example, we set it to construct sequences according to color palettes, and it was quite beautiful. I don't think an editor would have put it together like that, being preoccupied with characters and narrative. In that way, it can be a visual tool," he says.
Johan Knattrup Jensen is less intrigued by the practical use than by the philosophical discussion it inspires. "By representing the opposite of art, we're hoping Kaspar will make it more clear what art is. What makes a film personal? How does it communicate emotion through rhythm, timing and pacing? I think that reflecting ourselves in an AI will make us more aware of what it means to be human."