Machine Learning From Day One

Standing amid hundreds of people in the rotunda of Chicago's iconic Museum of Science and Industry, Ritika Ghosh (MSR '23) felt her confidence building.

Ghosh was teaching people of all ages about a topic she was not well-versed in just a few years earlier: machine learning (ML). She was there with her classmates from Northwestern Engineering's Master of Science in Robotics (MSR) program to celebrate the museum's Robot Block Party and demonstrate projects they worked on.  

For Ghosh, that meant showing off CopyCat, a robotic hand that emulated the movements of a human hand and used finger recognition to grasp objects of varying sizes. 

Ghosh relied on ML to detect and interpret hand gestures, which meant accounting for variations in skin tones, hand sizes, lighting, background conditions, and the gestures themselves.

"This project was a testament to the intricacies of machine learning in addressing real-world diversity," Ghosh said. 

It also was a testament to how MSR is teaching its students and providing them opportunities to work on ML. Traditionally, one of the biggest challenges in robotics has been perception and how robots see and interpret the environment around them. As ML capabilities have evolved in the past few years, new functionalities and opportunities are emerging seemingly on a daily basis.

The MSR program is committed to preparing students to work with — and in some cases lead — those emerging technologies from the moment they arrive on campus.

Every laptop provided to MSR students has advanced graphics processing units (GPUs) capable of locally running advanced models. The program also has two dedicated machine learning servers featuring state of the art graphics cards that are dedicated for use by MSR students.

"The amount of computational power that's needed is much more than what was traditionally needed or available for robotics," MSR co-director Matthew Elwin said. "Robots need to act in real-time, so the equipment that students have has to be able to run those models."

MSR features a variety of courses that involve machine learning, including multiple classes on computer vision, deep learning, and reinforcement learning. But like many of the lessons taught in MSR, the best way to learn machine learning is to get hands-on machine learning experience.

That's how Sushma Chandra (MSR '23) became comfortable with ML.

Chandra worked on a three-dimensional object detection project for autonomous driving for her winter project. For her final project, Chandra developed a multi-modal ML model for a whisker robot. The robot is inspired by how rats perceive information about their surroundings using their whiskers, and the project featured multiple neural networks responsible for different parts of a task.

She said her experience was enhanced by the MSR faculty and staff.

"Guidance and mentorship were available at every step of the way," Chandra said. "Our mentors in the MSR program had expertise in machine learning and were well acquainted with the state-of-the-art techniques applied in machine learning in modern day research and industries."

Ghosh's final project focused on human-robot collaboration and used reinforcement learning to develop an assistive agent capable of subtly adjusting human actions to optimize outcomes.

There have been a wide range of machine-learning focused student projects recently, including training a robot dog to walk and helping a robot translate text from any language to any other and then write or speak the text.

One of the program's unique elements is its partnership with the Shirley Ryan AbilityLab, a national leader in physical medicine and rehabilitation (U.S. News & World Report has named it the best hospital in its field for 33 consecutive years). Students routinely work with researchers to help patients rehabilitate themselves or develop devices to assist with daily living, from assisting with robotic wheelchairs to developing exoskeletons that can anticipate a person's next move and help them walk.  

"There's just a trove of problems that are really important to help solve that can have an impact on people's everyday lives," Elwin said. "The field of robotics is ripe for innovating and not just taking advantage of existing things from machine learning or artificial intelligence, but also helping to advance them because we have the ability to use robots to collect data from interactions with the real world."