CS+Law Innovation Lab Teams Demo Client-Focused Technology Solutions

Transformational technologies like artificial intelligence and machine learning profoundly impact the legal landscape, both in terms of how computation is changing the practice of law and how legal principles govern technology.

At the intersection of computer science and law, the COMP_SCI 397, 497: Innovation Lab: Building Technologies for the Law course, co-taught by Daniel W. Linna Jr. and Kristian Hammond, immerses multidisciplinary student teams in a collaborative effort to develop practical solutions for problems posed by external client partners.

“People tend to think about the law of technology, but this class focuses mainly on thinking about technology for law and how technology can change the way law firms, legal departments, legal aid organizations, courts, and governments deliver legal services,” said Linna, senior lecturer and director of law and technology initiatives at Northwestern, who has a joint appointment at Northwestern Engineering and the Northwestern University Pritzker School of Law.

On April 11, eight Innovation Lab teams presented live project demos during a virtual event, the culmination of their entrepreneurial process of understanding stakeholder needs, conducting market research, brainstorming ideas, testing solutions, and prototyping tools.

Innovation Lab students include McCormick School of Engineering students pursuing undergraduate and graduate degrees in computer science, as well as Pritzker School of Law students earning juris doctor, master of law, and master of science in law degrees.

“These are cross-functional teams. They speak different languages. They have different goals. They have different ways of breaking the world into pieces and putting things back together again,” said Hammond, the Bill and Cathy Osborn Professor of Computer Science at Northwestern Engineering. “Every single team was able to find a way to turn the goals and values of the law into something that a machine could actually do something with and I’m wildly proud.”

Developing computational solutions

Teams deployed computational technologies including OpenAI’s ChatGPT and other large language models to augment and automate a range of legal tasks, such as drafting and reviewing contracts and providing legal guidance to businesses and individuals.

Building on a previous study by global news and information services company Thompson Reuters, one student team aimed to evaluate the legal reasoning capabilities of ChatGPT. The team built a web application called the Automatic ChatGPT Prompter to automate a process whereby prompts are generated based on data extracted from the Competition on Legal Information Extraction/Entailment. The tool prompts ChatGPT to respond to the hypotheses suggested for specific statute law premises using a pre-selected legal analysis methodology.

The team created an evaluation rubric to judge the legal reasoning accuracy of each response and examined ChatGPT’s accuracy across different legal categories, including civil, contract, and property law. The team found that different prompting approaches significantly impacts the quality of ChatGPTs response, underscoring the importance of qualitative analysis when evaluating the large language model’s legal reasoning and applying it in a practical setting.

Another team worked with the Law Center for Better Housing (LCBH) to improve the natural language taxonomy of the LCBH Rentervention chatbot. A joint project of LCBH and the Lawyers Trust Fund of Illinois, Rentervention is a free resource designed to provide aid to Illinois renters in diagnosing their legal housing issues, understanding their rights, and exploring solutions. The student team aimed to advance the chatbot’s ability to categorize basic user inquiries – such as eviction, discrimination, and housing conditions – and utilized ChatGPT to train Rentervention to better recognize a user’s intent and thus provide the most relevant, detailed legal advice.

The team also created new categories to allow Rentervention to aid with issues like the state of homelessness and trained the chatbot to make better use of Google Dialogflow to recognize tense and context differences in user statements such as “I am homeless” and “I am going to be homeless” to ensure that the bot provides the correct resources.

A third team collaborated with the Northwestern University Feinberg School of Medicine Institute for Augmented Intelligence in Medicine to develop a program to aid the Northwestern Office of Sponsored Research (OSR) in more efficiently processing data sharing agreements. The platform automatically reviews and annotates data use agreements (DUA) from external organizations to ensure compliance with state and federal laws as well as the standard policies of Northwestern Medicine (NM).

The tool highlights key terms and keywords in the DUA based on functional categories, such as “liability,” “severability,” or “dispute resolution” using a color-coded system. The program also identifies whether NM requirements are satisfied or not satisfied by the agreement, including clauses related to publicity, security, and language related to third-party beneficiaries.

Other projects with client partners included:

Adobe: The student team built a web-based application integrated with the OpenAI application programming interface (API) called Adobe ContractBot to assist small to medium-size business owners without any legal background that they need to review and understand contracts. The ContractBot provides a concise summary of key contract clauses, highlights and defines legal terms, and allows users to ask questions via a chatbot, such as “as a purchaser, what am I on the hook for?” or “what are the governing laws of this contract?”

Berkeley Research Group: The student team developed a tool called Contract Genie, which leverages ChatGPT 3.5 to automate the drafting of compliant, industry-standard employment contracts within minutes. The user-friendly interface guides users through the process of creating a customized employment contract that meets unique business requirements while factoring in jurisdiction-based employment laws. Through a prompt engineering feature using the OpenAI API, a user can also request specific changes to the generated contract.

LCBH: The student team built a tool called the ResponseGenie to help LCBH paralegals and attorneys respond more efficiently to inquiries from individuals experiencing housing-related legal problems. Integrated with LCBH’s HelpScout system, ResponseGenie uses machine learning to identify the best matches to a user inquiry from a database of previous help desk conversations.

LexShift: The student team developed Omnibuster 2.0 — expanding the functionality of the tool prototyped by an Innovation Lab team last year — designed to enable legal professionals to efficiently perform a comprehensive review of omnibus bills. Omnibuster 2.0 pulls, parses, and formats legislation from Congress.gov and adds user experience benefits and web accessibility features, including keyword search and a table of contents to allow users to quickly scan and navigate the document, and comment and highlighting features for collaboration with colleagues.

McGuireWoods: The student team developed a Food and Drug Administration (FDA) action risk-prediction tool called FDAnalyzer to assists attorneys working with companies regulated by the FDA in answering compliance questions and investigating liability of risk enforcement. The FDAnalyzer system allows a user to more easily gather and evaluate relevant data from the FDA website by searching company name, key term, inspection ID, Code of Federal Regulations (CFR) rules, and enforcement actions. The tool outputs a timeline of FDA inspections and issuance of letters with specific CFR codes.

Evaluating solutions

A panel of three experts provided feedback on the students’ solutions, asked questions to explore and evaluate functionality of the tools, and suggested potential future directions for further development of the prototypes:

• Amol Bargaje, Global Chief Innovation Officer, Mayer Brown
• Joy Heath Rush, Chief Executive Officer, International Legal Technology Association (ILTA)
• Hanna Kaufman, Counsel for Strategic Initiatives, Lawyers Trust Fund of Illinois

At the conclusion of the event, Linna invited the panel to share their perspectives on the product demos.
Bargaje, who has participated in the Innovation Lab and viewed the annual demos for several years, underscored the collaborative efforts of the teams.

“This forum is very inspirational,” Bargaje said. “Bringing disciplines together is where innovation happens.”

Heath Rush expressed how impressed she was by the level of problems the student teams chose to solve and their approach to developing solutions related to legal drafting, pattern analysis, and data accessibility.

Kaufman praised the teams for their client-focus.

“You all did a great job of paying really close attention to what your clients are looking at, and not solving problems with a tool that you think is cool or trendy, but really attending to the specific needs, problems, and challenges you’re trying to address from the people who are working on the ground,” Kaufman said. “You’re obviously thinking not only about the problems at hand but how to make it better in the future.”

Hammond thanked the students and expressed pride for their efforts to consider the impact of technologies more broadly.

“I could not be prouder of this class,” Hammond said. “Even in those instances where the systems were not aimed at the particular problem of access to justice, there was an awareness among the teams of the impact that their technologies might very well have in the world and how they should be thinking about the need to build something that will improve the way we do our work and the way we think about our goals and values. And I’m stunned by how well these teams have embraced this.”

The Innovation Lab is one component of the Northwestern University Law and Technology Initiative, a partnership between Northwestern Engineering and the Pritzker School of Law, which also includes joint research projects, workshops, executive education, and academic conferences.