Informing Accountable Software

Agent-based modeling at the nexus of software, law, and society

“Agent Based Modeling at the Boundary of Law and Software” was a collaborative project between ICSI and New York University’s School of Law and Agent-Based Modeling Lab from 2021-2025. We studied how agent-based models (ABMs) can address the gap between legal requirements and software design by helping regulators, domain experts, software designers, and other stakeholders assess the potential societal implications of particular software and regulatory systems.

The interactions between software design, legal requirements, and social contexts can have unpredictable and sometimes problematic results. We used ABMs—a research method that involves bottom-up modeling of complex systems and computational determination of their emergent properties by running simulations—to model elements of the social and regulatory environment in which software systems operate. Much of the work was aimed at operationalizing privacy requirements, as these are notoriously challenging points of interaction between software, law, and society. Project outcomes reflected a maturing understanding of the problem and solution space.

Modeling Frameworks and Technologies

Agent-based modeling (ABM) can refer to a wide class of modeling techniques, ranging from automata, to heterogeneous agent economic models, to multi-agent systems. We explored many paradigms of ABMs and their ability to handle bounded rationality conditions that are characteristic of software-heavy domains (Benthall et al., 2022a). We settled on a powerful and expressive system of composable, dynamic causal games (Benthall and Lujan, 2026). We implemented an open source prototype of a software system for implementing these models of software, society, and regulations: scikit-agent. 

Software Design Principles

Our original proposal was that ABMs could be used as testing environments for the societal impact of software systems, and that this would be an input to regulatory design (Benthall et al., 2021). We published a case study applying this method to one design decision: privacy preservation in contact tracing systems (Benthall et al., 2022b). Ultimately, we developed a more robust modeling procedure based on causal games, and have demonstrated its use in mechanism design for privacy enhancing technologies (Benthall and Cummings, 2026)

Law and Policy

The project also produced notable public policy recommendations. Most broadly, it enabled the formulation of agent-based modeling’s contribution to legal theory (Benthall and Strandburg, 2021), which has played a part in the formation of the larger emerging field of Law and Complexity. It has also informed a more applied proposal for how modeling can be used in the adaptive regulation of privacy (Benthall and Sivan-Sevilla, 2026). Our work moved from scientific and legal theory to the more concrete challenges of institution design.

This includes: 

• Creating software specifications using models of regulations and the social environment in which software operates,
• Testing software systems for compliance using simulations of their social impact, and 
• Designing regulations that reflect these new tools.

Project Team

About

Outcomes