The Optimizer

Jorge Nocedal can’t stop thinking about optimization. Even when gazing at Lake Michigan or lying in bed, he constantly imagines new approaches to new problems in new areas, like deep learning.

“I don’t think that we have found the best optimization for machine learning—that’s my main motivating force,” says Nocedal, Walter P. Murphy Professor of Industrial Engineering and Management Sciences and (by courtesy) professor of engineering sciences and applied mathematics at Northwestern Engineering. “Apart from the impact that new optimization ideas could have, I’m drawn to the fact that machine learning poses problems that I’ve never seen before.”

Nocedal’s highly cited research stands at the intersection of computer science, operations research, and applied mathematics. His career achievements, including significant contributions to the theory of nonlinear optimization methods and the creation of widely applied algorithms, earned him the John von Neumann Theory Prize in 2017, regarded as the most prestigious theory prize in operations research. In early 2020, he was named a member of the National Academy of Engineering, among the highest honors in the field.

“Jorge’s research over multiple decades has fundamentally transformed the landscape in large-scale nonlinear optimization,” says David Morton, professor and chair of the Department of Industrial Engineering and Management Sciences. “His research is shaping our understanding of deep learning, which has led to truly remarkable advances in artificial intelligence.”

The potential of machine learning

Nocedal began his career in theoretical physics, a subject that helped him discover applied math and optimization, which in turn led him to computer science. He joined the McCormick School of Engineering’s computer science faculty in 1983. As he observed the rapid advances in the industrial engineering department, it felt to him like the natural home for his research. In 2013, Nocedal became chair of the department. During his three-year term, he championed the idea that certain models and ways of thinking had run their course, and that industrial engineering had to move forward to embrace data-driven models and machine learning.

Under his leadership, the department created the Center for Optimization and Statistical Learning, which Nocedal now directs. “We were one of the first industrial engineering departments to foresee that machine learning would become a major direction for industrial engineers,” he adds.

The hunt for intelligent algorithms

This foresight helped attract top talent like Zhaoran Wang, assistant professor of industrial engineering and management sciences. The two now work together to find answers that lead to more intelligent algorithms in the area of reinforcement learning. 

“Can a robot learn to feed a person or help with some tasks?” Nocedal asks, then explains. “Zhaoran and I are looking at improvements in that area, which uses deep neural networks and relies heavily on optimization. Our contribution is to find the right model and optimize it efficiently.”

Wang appreciates the rare opportunity to have such a renowned researcher as a mentor. “Jorge is open minded about emerging, challenging problems, such as deep learning, when most researchers are hesitating about whether to get into this field,” Wang says.

A passion for collaboration

Nocedal enjoys how human connections can lead to learning and discovery. “Insights clash, intuitions may diverge, and people may get passionate. Then suddenly, out of the fog, you get some clarity,” he shares. “And you feel there was a team of people who did it.”

Nocedal, named Bette and Neison Harris Professor in Teaching Excellence from 1998 to 2001, thrives on the collaborative energy his students bring to his research. “My students and I try to find not only the solution, but also the most beautiful solution,” he says. “We aim to develop something that people can’t improve upon—this is the ideal.”