On the screen, an animated Chinese opera mask decorated with red pom-poms and pearls moved side to side. Its mouth opened and shut. Spindly fingers danced next to it.

But the animation was not just a simulation: it was a virtual mask, synched precisely with the face and hand movements of Northwestern student Yangdongling Liu.

Liu, a PhD candidate studying physical chemistry, coded the costume in Generative Methods, a computer science class that explores the intersection of programming and art. In the class, taught by Kate Compton, students from other academic programs collaborated with theater design students in the School of Communication to learn how theater designers think about costuming and to find parallels in their respective disciplines.

Applying the artistic lessons they’d learned, students created virtual costumes that overlaid graphic masks and animated hands on video while tracking their body and face movements.

Blending computer science and art comes naturally to Compton, a generative artist and assistant professor of instruction in computer science at Northwestern Engineering. Last fall, Compton met with Ana Kuzmanic, associate professor of costume design, to discuss a possible collaboration. “The collaboration was a way for Ana and me to communicate through our students and start to develop a common language that we both could understand,” Compton says.

Programming to Reflect Emotions

During a class demonstration on Zoom, student Rohith Jayaraman pulled up a fiery mask, and spiky red flames radiated from his face. The virtual mask was inspired by a scene in J.K. Rowling’s Harry Potter and the Goblet of Fire when Sirius Black’s face appears to Harry Potter in a fireplace. The code, he explained, reflects the emotions of the character.

As he spoke, he controlled how quickly his mask’s flames flickered—fast when angry and slow when happy. He learned the artistic process from theater design students, including undergraduate Hayley Wallenfeldt, designer and animator, and master of fine arts candidate Benjamin Kress, an experienced costume designer, who fashioned elaborate visual presentations to illustrate core principles of theater design.

“Art doesn’t come to me,” says Jayaraman, a master of science in computer science student. “I’ve done a lot of programming, but I’ve never been able to bring something cool up on screen like this. There’s something different about creating a program versus creating something fun people can play with.”

A Foundation for Meaningful Collaboration

With code, costumes can even take on their own, asynchronous personalities. In some ways, virtual costumes become characters themselves.

The real fun of these masks is that they are interactable live.Kate ComptonAssistant Professor of Instruction, Computer Science

“Imagine if your costume is moving, and then it winks at you,” Compton says. “There’s a character in your costume that’s very alive and sharing your face with you. It tears apart the connections we’ve assumed about the body.”

The possibility of virtual mask technology becoming more involved in theater productions is exciting, Compton says. “It’d be really amazing for a ‘Jekyll and Hyde’ production to include a virtual mask projected onto an actor, and then someone offstage with a joystick taking control of the actor’s face.

“I think the real fun of these masks is that they are interactable live,” Compton adds. “There are plenty of prerecorded videos used on stage, but this would let live actors and mask ’puppeteers’ improvise playfully, like the Muppets do.”

This type of interdisciplinary collaboration is a hallmark of Northwestern’s mission. To Kuzmanic, much can be learned from the ways people from different fields think creatively and approach problem-solving.

“On the surface, our fields might be perceived as opposites,” Kuzmanic says. “Fundamentally, while we speak different languages in terms of our respective disciplines, there are great similarities in our individual ways of generating and delivering information, and those similarities provide the foundation for meaningful collaboration.”