Faculty ProjectsCS296 Ethics and Fairness in Machine Learning
Project Manager
Zach Wood-Doughty, Assistant Professor of Instruction, Computer Science
Amount Requested
$5,000
Summary
There is an urgent need to teach ethics within our computer science curriculum, both to empower our alumni to address ethical decision-making in the workplace, and to meet the high demand from current students. Currently, many of our department’s ethics initiatives are led by students. Two PhD students co-teach the Computing, Ethics, and Society course that is otherwise without an instructor. Several undergraduates have worked as Ethics Fellows to develop and incorporate ethics modules into existing courses. I currently teach several courses on machine learning and have worked with the Ethics Fellows to incorporate ethical thinking into my Winter and Spring 2024 sections of CS349 Machine Learning. To build on what I am learning from these extremely passionate and knowledgeable undergraduates, I propose to develop a new 200-level course that focuses specifically on ethics and fairness in machine learning (ML).
Many of the most pressing ethical questions in ML arise from high-stakes environments such as medical diagnosis and credit scoring. To engage with these ethical questions outside of thought experiments, students in my proposed class will use ML libraries to build an automatic grading classifier (autograder) that will assign grades to their written reflections. Assignments will involve writing code to build an autograder and methods to evaluate autograders’ performance. Students will also write short essays that reflect on real-world case studies and their own perceptions of autograders’ fairness and accuracy. While students’ work will also be graded manually, students’ autograders will play a role in determining one another’s grades. Later assignments will ask students to conduct “fairness audits” on autograders to explore the trade-offs between accuracy and different metrics for quantifying fairness. Near the end of the quarter, students will collectively write, debate, and vote to adopt a policy to which their autograders must conform.
Throughout the course, the investigation of autograders will serve as a microcosm for the broader ethical questions around ML. Students’ own perceptions of autograder classifications will help ground discussions of complex issues.
Planned Activities/Investments
Matching funds from the department will allow me to dedicate a portion of my summer to preparing initial drafts of the course structure and assignments, including the architecture for autograding students' responses and allowing students to deploy their own autograders.
Funding provided by the Murphy Society will allow me to hire undergraduate ethics fellows to work through the first drafts of these assignments and then act as peer mentors to consider what challenges would arise in grading the assignments. Because this class structure is much more experimental than a standard CS course, I will rely on ethics fellows' expertise to design effective course discussions that we can tie into both the written and coding assignments.
Impact
I intend to have this class require only CS150 ("Fundamentals of Computer Programming 1.5"), to make it maximally available to as many students as possible. For the first time this class is offered, I plan to keep the course enrollment relatively small (e.g., 30 students). However, because of the intentional focus on autograding, I hope to scale this class to much larger class sizes (e.g., up to 300 students). I have had success with scaling CS349 machine learning to very large class sizes, including with ethics discussions in my most recent offering of CS349 (154 students). Given the immense demand for classes covering both ethics and ML, I expect this course to reach a very wide audience.
This project will also provide support for the continuing CS ethics fellows program, as I hope to recruit them (and peer mentors from CS349) as the students I hire for this work.
My standard practice while teaching is to collect copious surveys from students about their experience and their suggestions for improvement. I find this extremely helpful as I experiment with different grading structures and lecture material. This course would be my most extensive experiment yet, but I would use student feedback as my primary evaluation tool for understanding what went well and what needs improvement.
Sustainability
The initial development of the course materials is a one-time expense. As with all courses it will continue to develop as it is taught. Ethics fellows (supported by a separate Alumni Curriculum Award grant) and peer mentors (supported by the university to provide course support) will help refine the course materials.
Deliverables
The major deliverable is the new course that will be offered for the first time sometime between Spring 2025 and Spring 2026. I plan to teach this class once a year for the foreseeable future.
Budget Overview
$5,000 requested — 200 hours of undergraduate student time at $25 per hour
$5,700 matched — faculty summer salary
Total Budget Amount: $10,700
Matching Funds
McCormick has approved $5,700 in matching funds to support my summer salary.
This project is approved by department chair Samir Khuller (samir.khuller@northwestern.edu).