Strong Northwestern Presence at EC’25

Northwestern Computer Science had a strong presence at the Twenty-Sixth Association for Computing Machinery (ACM) Conference on Economics and Computation (EC'25). Held July 7-10 at Stanford University, EC’25 is the annual forum for advances in theory, empirics, and applications at the interface of economics and computation, convened by the ACM Special Interest Group on Economics and Computation (SIGecom).

"Our presence at ACM EC 2025 highlights the amazing strength of Northwestern in the CS+Economics space and is also a model for what CS+X interdisciplinarity can be when executed masterfully and strategically,” said Samir Khuller, Peter and Adrienne Barris Chair of Computer Science at the McCormick School of Engineering. "I was amazed by the breadth and depth of contributions by our faculty and PhD students."

Northwestern Engineering’s Jason Hartline presented the EC’25 closing plenary talk, advocating for the CS+Economics research community to develop AI theory through the lens of proper scoring rules.

“Fundamental challenges and frameworks in AI can be understood as problems of statistical decision theory, which are equivalent to scoring rules,” said Hartline, professor of computer science at the McCormick School of Engineering and director of Northwestern’s Online Markets Lab.

During the plenary talk, Hartline outlined four recent projects demonstrating the application of the scoring rules model.

In the 2022 paper "Optimization of Scoring Rules,” co-authors Hartline; Yifan Wu, a PhD student in computer science at Northwestern Engineering; Yingkai Li (PhD '22); and Liren Shan (PhD ’23) developed optimal scoring rules and determined that classical approaches like the quadratic scoring rule are not optimal. Building on this work in the 2024 paper "Calibration Error for Decision Making,” Yifan Wu and Lunjia Hu (Harvard University) showed that the optimization of scoring rules enables trustworthy quantification of the error of predictive models.

A Northwestern Engineering team including CS PhD student Ziyang Guo, Yifan Wu, Hartline, and Jessica Hullman, Ginni Rometty Professor of Computer Science, then demonstrated how the theory of scoring rules results in more rigorous human-AI collaboration experiments by identifying sources of loss, such as an under- or overreliance on the AI prediction or lack of understanding of the AI explanations. In the paper "Aligned Textual Scoring Rules," Northwestern CS visiting student Yuxuan Lu, Yifan Wu, Hartline, and Michael Curry (University of Illinois Chicago) developed a method for aligning a proper scoring rule for LLM-generated text. Their approach fits a proper scoring rule to a human user’s non-proper reference scores.

"Mechanism design has an important role to play in developing a theory of AI,” Hartline said. “Since mechanism design is a central research area for the ACM Conference on Economics and Computation, this connection provides an opportunity for this community to lead the way in developing such a theory."

Robust forecast aggregation

In another project featured at EC’25, Hartline collaborated with computer science PhD student Anant Shah to optimize the aggregation estimate of multiple forecasters without knowledge of how they arrived at their predictions. In the paper “Algorithmic Robust Forecast Aggregation,” the team — which included collaborators Yongkang Guo, Zhihuan Huang, and Yuqing Kong (Peking University) and Fang-Yi Yu (George Mason University) — presented an algorithmic framework that provides efficient approximation schemes for general information aggregation with a finite family of possible information structures.

“The team takes a computational perspective to this well-studied problem, providing an efficient algorithm to compute a near-optimal robust aggregator,” Shah said. “This has important applications in a variety of areas, including weather forecasting, medical prognosis, and finance.”

Shah studies social and economic networks, mechanism design, and online algorithms and is coadvised by Hartline and Benjamin Golub, professor of economics at Northwestern’s Weinberg College of Arts and Sciences and (by courtesy) professor of computer science at the McCormick School of Engineering.

Transferability of economic models

Chaofeng Wu, a fifth-year PhD student in computer science, also presented at EC'25. His work considers the following fundamental question: If a researcher uses a model estimated on data from one domain to make predictions in a new one, what guarantees can we provide about its prediction quality? For example, can a model of risk preferences estimated on insurance data from one population reliably predict insurance take-up in another?

Chaofeng Wu, his adviser Annie Liang, and their research collaborators addressed this question by introducing a new theoretical framework based on conformal inference and constructing finite-sample forecast intervals to quantify transfer performance. They apply these methods in a classic economic application---predicting willingness-to-pay for risky outcomes---and compare the transferability of economic models with that of machine learning algorithms. Using risk-preference data collected from multiple regions, the team found that while machine learning methods, such as random forests and kernel regression, outperform the economic models within their training domain, they struggled with generalization. In contrast, economic models such as expected utility and cumulative prospect theory proved more robust when applied to new settings.

“Our theoretical results provide a general all-purpose toolkit for evaluating transfer performance, and our empirical findings highlight the value of economic models for robust, cross-context forecasting,” Chaofeng Wu said.

Liang is an associate professor of economics at Weinberg and (by courtesy) associate professor of computer science.

Strategic cloning in elections

Northwestern Engineering’s Edith Elkind, Ginni Rometty Professor of Computer Science, also contributed to the EC’25 technical program with the paper “From Independence of Clones to Composition Consistency: A Hierarchy of Barriers to Strategic Nomination.”

Elkind and her team examined preference aggregation in the case of so-called spoiler effects in political elections, where every voter ranks very similar political candidates (called clones) together as a group. The researchers compared different definitions of clones and explored which voting rules are resistant to cloning, as indicated by whether the introduction of clones changes the outputs in a predictable way. They also analyzed how this work relates to the framework of obviously strategyproof mechanism design.

“Our work is relevant to fine-tuning AI models by means of reinforcement learning from human feedback, as many potential responses that an AI model may produce are similar enough to be treated as clones,” Elkind said.

Northwestern contributions

Additional Northwestern contributions to the EC’25 technical program and workshop sessions included the following papers, posters, and talks:

• “Aligned Textual Scoring Rule” — Yuxuan Lu
• “Artificial Intelligence Clones” — Annie Liang
• “Behavioral Study of Dashboard Mechanisms” — Jason Hartline, Jessica Hullman, and Paula Kayongo, a PhD student in computer science at Northwestern Engineering
• "Benchmarking LLM's Judgements with No Gold Standard" — Yuxuan Lu
• “Budget-Constrained Auctions with Unassured Reports: Strategic Equivalence and Structural Properties” — Chang Wang, a PhD student in computer science at Northwestern Engineering; Zhaohua Chen (Peking University); Xiaotie Deng (Peking University); Jicheng Li (Columbia University); and Mingwei Yang (Stanford University)
• "Confusion Matrix Design for Downstream Decision-Making" — Yiding Feng (PhD '21)
• “Evaluating the Efficiency of Cap-Based Regulation in Matching Markets with Distributional Disparities” — economics alum Kyohei Okumura (MA ’21, PhD ’25), Kei Ikegami (New York University), Atsushi Iwasaki (University of Electro-Communications), and Akira Matsushita (Kyoto University)
• “Games on Endogenous Networks” — Benjamin Golub and Evan Sadler (Columbia University)
• “Human-AI Interactions and Societal Pitfalls” — Sébastien Martin, assistant professor of operations at Northwestern’s Kellogg School of Management; and Francisco Castro and Jian Gao (University of California, Los Angeles)
• “Incentive Design With Spillovers” — Benjamin Golub, Anant Shah, and Krishna Dasaratha (Boston University)
• “Mutual Information from Samples” — Matthew vonAllmen, a PhD student in computer science at Northwestern Engineering
• “Opening the Human Blackbox in Model Assisted Decisions” — Jessica Hullman
• “Optimal Membership Design” — Piotr Dworczak, associate professor of economics at Weinberg; Scott Duke Kominers (Harvard University and a16z crypto); Changhwa Lee (University of Bristol); and Marco Reuter (International Monetary Fund)
• “Perfectly Truthful Calibration Errors” — Yifan Wu
• "Persuasive Calibration" — Yiding Feng
• “Proper Scoring Rules for Text and Superalignment” — Jason Hartline
• “Regulation of Algorithmic Collusion, Refined: Testing Pessimistic Calibrated Regret” — Chang Wang
• “Relative Monte Carlo: Scaling Reinforcement Learning to Large Systems with Local Structure” — Sébastien Martin; Audrey Bazerghi, adjunct lecturer of operations management at Kellogg; and Garrett van Ryzin (Amazon)
• “Reputation Effects with Endogenous Records” — Harry Pei, assistant professor of economics at Weinberg
• “Robust Market Interventions” — Benjamin Golub, Andrea Galeotti (London Business School), Sanjeev Goyal (University of Cambridge), Omer Tamuz (California Institute of Technology), and Eduard Talamas (IESE Barcelona)
• “Robust Online Learning with Private Information” — Kyohei Okumura
• “Robust Procurement Design” — Alessandro Pavan, HSBC Research Professor of Economics at Weinberg; and Debasis Mishra and Sanket Patil (Indian Statistical Institute)
• “Scoring Rules for a Theory of AI” (plenary talk) — Jason Hartline
• “Statistical Decision Theory for Human-AI Collaboration” — Jason Hartline
• "Streamlining Equal Shares" — Edith Elkind and Sonja Kraiczy and Isaac Robinson (University of Oxford)
• “Threshold Scoring Rules: A Generalized Decomposition for Optimizing Scoring Rules with Applications to Decision-Theoretic Calibration Error” — Jason Hartline and Matthew vonAllmen
• “The Value of Context: Human Versus Black Box Evaluators” — Annie Liang and Andrei Iakovlev, a PhD student in economics at Weinberg
• “When Are Forecasts Trusted? The Role of Calibration and Visualization” — Paula Kayongo