Episode #84
Who Decides? AI, Accountability & the Hidden Choices Behind Every AI System | Prof. Renée Cummings
Who decides when an algorithm gets it wrong? That's the question at the heart of this conversation with Professor Renée Cummings —a criminologist who spent two decades in criminal justice before becoming one of the world's leading voices in AI governance. Every day, AI systems influence decisions that affect people's lives: who gets hired, who gets approved for a loan, who receives medical care, and who gets flagged as a potential risk. We often assume these systems are objective because they're built on data. But data carries history—and history carries bias. Renée explains what happens when that history becomes training data, why nobody walks away clean when an algorithm causes harm, and why she believes some AI systems should never be fixed—they should be deleted. This isn't a theoretical discussion. It's about the decisions increasingly being made about our lives by systems we rarely see and often cannot question. • What "data trauma" means and how historical bias becomes part of AI systems • Why algorithms can amplify past mistakes at unprecedented speed and scale • The real accountability chain when AI causes harm—and why responsibility doesn't stop with developers • When a broken AI system should be fixed versus removed entirely • Why Renée says "Code is the new cop" • How AI is changing hiring, lending, healthcare, criminal justice, and public trust • The emerging risks of algorithmic grooming, AI companions, and child safety • Why every AI system is ultimately a reflection of human values • What founders, product leaders, and CTOs should consider before deploying AI into high-impact decisions • How governance must evolve as AI moves from assisting humans to acting autonomously Professor Renée Cummings is Professor of Practice at the University of Virginia School of Data Science, Nonresident Senior Fellow at the Brookings Institution, former member of the World Economic Forum's Data Equity Council, member of the Global AI Governance Alliance, Founder of A.W.A.R.E. (Algorithmic Wellbeing & Responsible Engagement), and is recognized among the world's Top 100 Women in AI Ethics . She has advised policymakers, including the U.S. Senate, bringing a criminologist's perspective to AI governance, ethics, and public policy. Subrata Kar is the host of The Spark & The Forge , where he explores how founders, researchers, policymakers, and enterprise leaders are building the future of AI through thoughtful, long-form conversations that go beyond headlines and hype. Listener Advisory: This episode includes discussion of child safety, emotional harm, and youth suicide in the context of AI companion technologies. Listener discretion is advised.
