Seeing the Power of Machine Learning

Hang Yin (MSR ‘23) was in high school when he founded a robotics team to participate in the FIRST Robotics Competition. He enjoyed working on the mechanical side of robotics, but the competition opened his eyes to a new opportunity within the field: machine learning (ML).

“I was blown away by teams using machine learning to make their robots smarter, like detecting objects or automating simple routines,” Yin said. “It seemed like a cool way to cut down on human driver mistakes and make robots more efficient.”

Yin quickly realized ML was more than just cool — it was a core component in robotics and would become even more important moving forward. To best prepare himself for that future, he turned to Northwestern Engineering's Master of Science in Robotics (MSR) program.

"I understand the incredible power of ML in robotics, alongside the challenges in verifying model outputs before actually deploying them," Yin said. "Given how critical safety is in robotics, blindly trusting an ML model, regardless of how much data you train it with or how good it performs on the test set, is risky."

MSR students are trained to effectively minimize that risk, Yin said.

During his first quarter in the program, Yin used a Franka arm to play the block-moving game Jenga. For his independent project, he leveraged ML in three different ways to develop a software stack so users could give verbal commands to control a Franka arm and prepare specific meals. 

Yin's final project included developing a ML data infrastructure and training a neural network so mobile collaborative robots can help human users handle large and awkward objects. 

Today Yin is a software developer at Stanford's Vision and Learning Lab, where his focus is on a project called BEHAVIOR (an acronym for Benchmark for Everyday Household Activities in Virtual, Interactive, and Ecological Environments). The project simulates 1,000 different household activities with embodied artificial intelligence to identify software development opportunities.

BEHAVIOR is all about gathering ML data, Yin said.

“Machine learning is definitely making a big splash in robotics, especially in fields like computer vision,” he said. “We're starting to see robots that can learn and adapt more, making use of sensors of different modalities. I've been really excited about the latest advances.”

That excitement is channeled into Yin’s daily tasks, including fixing bugs and setting up simulations to test software capabilities.

“What's really starting to become clear is how important data is,” Yin said. “From some recent work, we've seen that even very minimal neural networks can do amazing things if they're fed enough quality data.”

The principle behind Yin's point is straightforward. For machines to learn, they first need to have an accurate and workable frame of reference upon which to build.

Yin may have been introduced to ML in high school, but it was his time in MSR that helped develop that frame of reference and prepared him for where he is today.

“A huge aspect of MSR is the program's hands-on approach, allowing us to work directly with real robots,” he said. “MSR students become very familiar with them, which prepares us well for the complexities of real-world robotics."