Bringing Computer Vision to Zoox

Adam Pollack (MSR '17) is a software engineer at Zoox, a subsidiary of Amazon creating a driverless vehicle to provide mobility as a service in densely populated areas. Think of any of the popular ride-sharing services, but with vehicles designed to autonomously and safely maneuver through busy city streetscapes.

Pollack’s mission is to implement machine learning models to enable that vehicle to see and react similarly to how the human eye-brain-body combination works. To do that, he relies on lessons learned in Northwestern Engineering's Master of Science in Robotics (MSR) program.

“I didn't decide that I was really into computer vision until maybe halfway through the MSR program, which is good,” said Pollack, who has been with Zoox since 2019. “That’s the point of the program — to explore the different parts of robotics. Robotics is a massive field. There are so many different things you can do.”

What Pollack is doing aims to revolutionize urban transportation and reduce the harmful effect motor vehicles have on the environment.

Zoox, purchased by Amazon in 2020, focuses on creating what amounts to a robo taxi service with a fleet of electric vehicles that reduce traffic and the need for car ownership among city dwellers. That makes Pollack’s job — enabling the vehicles to recognize and react to the enormous variety of obstacles and issues it might encounter in an urban area — of paramount importance for rider and pedestrian safety.

One of the challenges is thinking through those potential obstacles and issues.

"From a perception standpoint, there are a lot of long-tail detection cases where it's hard to tell that the objects are there because they're not standard," Pollack said.

Another challenge lies in how quickly technology is changing, but Pollack said the MSR program’s structure encourages the type of learning necessary to keep up with a rapidly evolving field. His classes and projects helped him adopt a learning framework that allows him to continue to upgrade his skills as new technology emerges and older technology changes.

“The density of the program, how hands-on the program was – it was a very practical experience,” Pollack said. “I really wanted to hit the ground running and get into industry. Through the MSR program, I learned a lot in just 12 months and then immediately went into industry, so that worked out really well.”

For his final project, Pollack worked to develop a software package that would identify the pose of an HTC Vive tracker. His winter project focused on machine learning and creating an object classifier that could be used for a Baxter robot to locate and pick up objects. Pollack sees a clear line between the computer vision he relied on in that project and the work he's currently doing.   

One of the most important lessons he learned through those projects was to be realistic about timelines. The autonomous vehicle industry is still in its infancy, and its impact on consumers remains to be seen. Zoox’s robotaxi went from a sketch on a sheet of paper in 2016 to taking its first completely autonomous voyage on open public roads in 2023.

Pollack said he is happy to be at a company where he can play such a vital role in crafting the future.

“We're at a nice middle ground between a startup and a more established company,” he said. “It stays challenging all the time. There are always new problems to solve."