Satwik Mishra

Hello everyone! Welcome to another episode of Trustworthy Tech Dialogues. I’m Satwik Mishra, the Executive Director of the Centre for Trustworthy Technology. Today, we’ll dive into some of the foundational issues shaping our technological landscape. The curation of inclusive data structures. The value of ethics for aligning our tech architecture to a human centered future and building societal trust in artificial intelligence.

Today we’re confronted with a choice, will we permit ourselves to be captivated by the simplistic binary narratives of doomsday and utopia, or can be instead turn our focus to a nuanced, shared, and profoundly human reality of the people that AI is truly meant to serve. At the centre, we believe it’s about this human reality, a path of hope grounded in authenticity.

But hope here is not naive. It doesn’t ignore alarming realities or troubling statistics. Instead, it embraces the urgency of today by pushing for systems that are just inclusive and trustworthy. Hope, which is grounded in authenticity, allows us to look at AI not as a silver bullet or a looming threat, but as a tool with the potential for transformative change.

It keeps us moving towards a world that prioritizes fairness, dignity and justice. Today AI is woven into us shared realities, creating powerful hubs of influence while also enabling millions to learn, innovate and access new opportunities. Our discussion today centers on this duality. AI may concentrate power, but it also holds the potential to amplify voices and empower individuals at every level of society.

With AI’s great power comes at corresponding peril if we proceed without due diligence, ethical guardrails and a commitment to human rights. Every tread off innovation decision on our data systems and the ensuing policies should hold up to a standard of reasonability and fairness to maintain public trust. But today, we’re asking some fundamental guiding questions. How do we ensure that emerging technologies like AI enhance our agency?

How can we balance innovation with accountability? Securing a future with tech truly acts as a force for good. What would it take to be hopeful with purpose to navigate forward with a vision grounded in authenticity and inclusivity. To explore these pertinent themes, I’m joined by Professor Renée Cummings. Professor Cummings is a world-renowned AI data and technology ethicist.

She’s currently serving as the first data activist in residence at the University of Virginia School of Data Science. In addition, she co-directs the Public Interest Technology University Network at UVA and holds the position for Non-Resident Senior Fellow at the Brookings Institution. Professor Cummings is a member of the World Economic Forum’s Data Equity Council and of the AI Governance Alliance.

She also serves on the Advisory Council for the AI and Equality Initiative at the Carnegie Foundation for Ethics and International Affairs. Her work delves into the nuance, ethical implications of data on society. Through her research and advocacy, she has elevated our understanding of AI systems, guiding us towards a future which is ethically resilient, responsible and informed for the benefit of all.

Professor Renée Cummings, welcome.

Prof. Renée Cummings

Thank you so much. It’s such an honor and a pleasure to be with you.

Satwik Mishra

Hey, to start off, tell us about your journey into AI ethics. How has your background in data science shaped your perspectives on developing trustworthy AI?

Prof. Renée Cummings

I will say that my background begins with my work in criminology. It is working as a criminologist, engaging with these risk assessment tools. Algorithmic decision-making systems are data driven systems that I realized that the predictions, the forecast that were being made with the zombie prediction that were overestimating the risks of black and brown defendants in the criminal justice system. It also had me thinking about risk and what do we define as risk in criminal justice? and looking at the connections between risks in criminal justice, risk in the financial sector, risk in the insurance sector, risk when it comes to credit scores and realizing that I was seeing some patterns and some trends that really needed, some critical analysis.

So that is how I got into AI, really thinking about the ways in which we were using a new technology, but we were actually deploying old, discriminatory, stereotypic stories steeped deep in systemic biases and wondering, what did we need to do as a collective at interdisciplinary thinkers to really amplify, this conversation, to really look at those risks, because I am so committed to, technology and advancing systems with technology that I believe the only way for us to truly reap the rewards of this extraordinary technology called AI.

We had to have a really sophisticated understanding of the risks, our rights, and most critically, what were our responsibilities? For those of us using this technology, designing, developing, procuring, adopting this technology. So it is through criminal justice, through criminology and criminal psychology and really working in the space of human behavior and the intricacies of the human mind.

Then I found myself really falling in love with data science and of course, the building of algorithms.

Satwik Mishra

That’s wonderful. Like, it has been such a privilege over the last two, three weeks going through your work and preparing for this podcast. It’s been absolutely wonderful to see your journey and listen to the things that you’ve spoken about from a very societal lens.

But on an emerging technology like AI and at the root of many of the largest questions around data and AI, you have explained a foundational what you call data dilemma like as powerful and I’m quoting you here “As powerful as data is, it is also very vulnerable”. Now explain to us what is this data dilemma and how does this data define the challenges of inclusive AI growth as we know today?

Prof. Renée Cummings

Most definitely. So, we have something that we cannot see which is data, but it is so powerful that it has been the greatest game changer when we think about geopolitics, when we think about the global power structure, you know, we’ve heard the saying, those who own the data, own the world. We’ve heard the saying our data is becoming our DNA.

We’ve heard this saying, you know, data power is a global power, it’s economic power. So, when we think about something that is so powerful, then we’ve got to understand how vulnerable it is. When we think about privacy risks, when we think about safety and security, when we think about questions around data governance or the lack of data governance, when we think about something that is being amassed in such great amounts and we cannot see it, we cannot feel it, we cannot touch it, but we know it’s growing and we know it carries an extraordinary value, not only a societal value, for the extraordinary things that it can do.

But it carries a monetary value because everyone wants to own it, to have access to it, to use it to rethink, redesign, reimagine, and reposition. If not their business model, their strategies, and of course, use data to design their solutions. What is so critical is that we’ve got to think of the question of equity. I always say the data equity is good economics.

Data equity is not only about equal rights, it’s about equal opportunities. And at the World Economic Forum, I was really privileged to be a part of a group, the Data Equity Council, that put forward a really brilliant document on data equity and also situated within the context of indigenous approaches to understanding equity, understanding data governance, understanding questions around data serenity and understanding something as critical as data provenance.

How do we look at the history of data? How do we You know, look at the ways in which, the trajectory of data is, being developed and, and the kinds of ways we need to maneuver that or how data is being manipulated. So, inclusion is critical because inclusive data gives you more opportunity. Inclusive data gives you more approaches.

Inclusive data gives you more dynamic ways in which to think about the solutions and the strategies and the system that you want to develop. So, when we think about this question of the data dilemma, we’ve got to think about questions around power, questions around policy and questions around politics, because we’re seeing the data has a political context, a social context, a cultural context, an economic context.

And all of these contexts, lets us know how particularly sophisticated we have got to become. You know, our own understanding and why critical thinking is also crucial to this data dilemma.

Satwik Mishra

We will definitely link, the Data Equity Council report in our, in our show notes, the wonderful report, and I encourage everyone to read it.

You have this thing around coming up with slogans which just inspire the community. And from the data dilemma, you also have something you call the “Duty of Care”. And you said around, let’s move from the data dilemma to AI ethics. You talk about like navigating AI ethics requires a duty of care across a range of stakeholders.

Now, can you share your vision around what is this duty of care and how should it be exercised across like governance right now?

Prof. Renée Cummings

Most definitely. You know, when we think about risks, we think about responsibilities, and we think about rights, and we also have to think about questions around protection and liability. Liability is critical to AI safety, to AI governance, to AI ethics as well as data.

I always say there is no AI without data. So, when we think AI we got to be thinking data, when we’re thinking data, we’re also thinking AI. So, duty of care is our responsibility as data scientists, as technologists, as leaders, as social scientists. So, when we call for this interdisciplinary approach to data, we’re looking at interdisciplinary duty of care as well.

So, we all have a responsibility, it is all of us who have this duty to ensure the work that we are doing in this space really presents an attitude of care. Caring about not only the communities but caring about the long-term impacts of this technology, caring about the, global, geopolitical, relationships, caring about the future, caring about the environment.

We have a technology as super as it is. It is also super destructive when we think about the environment, when we think about the resources required to create these massive data centers or for compute power or for building the, you know, the, the semiconductors that are, that are required. So, when we think about our duty of care, it is our responsibility.

And it is also the liability that speaks to the fact that those who may be harmed, because we did not deploy that due diligence or that duty of care must have redress or questions around restorative approaches, using AI or questions around reparative approaches using AI. So, duty of care is this care and concern and liability that must be a part of any conversation around data, AI, safety, ethics. Of course, the future, the ways in which we are building our future using data and AI.

Satwik Mishra

So, this is a fascinating topic and we’ve, all in the community been working at it and made some progress, not to the level I think any, any of us would want, but we are still at it, in this duty of care and building these inclusive structures like across our domain, what progress do you find encouraging

And where do you think more of our energy needs to go?

Prof. Renée Cummings

I will start with the the end of the question. I think our energy needs to be focused on the governance, and we need to understand that governance is local as much as it is global. So, we’ve got to have these local conversations. You know, when we think about local governance, when we’re thinking about city council, when we’re thinking about universities, when we’re thinking about, communities have got to become part of the conversation for us to have, the kind of global governance that’s required, it cannot only happen at that level.

It must be, a bottom up, you know, a ground up approach. We’ve got to have these conversations happening. We’ve got to ensure data and AI literacy are critical to the ways in which we are thinking, when we think about that, that is so important. I think what we have done really well is that we have built these brilliant communities of practice around data, around AI.

So, I think, when we think about many of the scholars in this space, the ethicists, the technologists, the leaders, public policy, there’s a passion that we are all bringing to this because we want to see it done well. But we have got now to find ways in which we can distill these conversations from the C-suite and the boardroom to Main Street.

And that is what the challenge is. How do we lift up, you know, communities and, and, and members of the public, how do we provide the right kind of public information, public education? And how do we include these voices in public oversight, which is now even more critical when we’re thinking about the advances being made with data and AI.

Satwik Mishra

You also speak like getting into the practical aspects of building this from the ground up, from high level conversations to local conversations around this and empowering people, at the at the lower levels where these systems are deployed, like you also use this term, which you called “Ethical Resilience for Professionals”. And tell us about it. How do you cultivate ethical resilience in your work specifically with data and AI and how would you encourage professionals to do the same?

Prof. Renée Cummings

So, I always say that there is no perfect data set. So, there is going to be no perfect algorithm. When we think about data, we know that data are people. Data are everything that we do our smiles, our eyes, you know the ways in which we are thinking, our likes, our dislikes we understand that. So, I always say that we have got to be ethically resilient, because so much is changing in this space.

When we thought we were governing AI, then we got GenAI. Now we’re thinking about Agentic AI system. So, every day, every week, every month, it’s something new when something often, more exciting, but something that also needs critical thinking, something that needs a sophisticated understanding. So ethical resilience means that we have got to be resilient, intellectually resilient in this space.

We have got to understand that the ethics questions and the safety questions keep changing, keep evolving and that is very important. And we have got to state ethically vigilant, which means as data scientist, as technologists, as policymakers, as leaders, we have got to continuously try to be one step ahead of the game. We know that it is said that the law is always behind the technology.

We’ve got to move away from that and we’ve got to start to prepare. We’ve got to think about the future of law. We’ve got to think about the future of just every discipline, every industry and we have got to bring the kind of risk management that is also futuristic, not reactive. At the moment, we’re doing very reactive risk management.

We’ve got to do proactive and futuristic risk management, which really creates, you know, what these hypothetical situations and scenarios could be. And what would be our response? So, it’s like, red teaming, you know, 5.0 is where we need to be at this moment.

Satwik Mishra

I agree, we need to do more of a scenario mapping and prepare as this technology just accelerates at a speed that we’ve probably never seen before in our careers.

But we are in towards the end of this year, and it has been a pivotal year in many ways, for a lot of reasons in the AI space. Where do you see some industry wins and government wins across the world, whether in the United States, or anywhere else where what would you identify as a place where we made some progress, either in an industrial ecosystem or even for governments?

Prof. Renée Cummings

I think we made an extraordinary amount of progress. When we think about 2024, when we think about what’s been happening globally, a lot of traction and globally when it comes to data governance and AI governance countries all over the world, big and small thinking about national AI strategies, thinking about data protection and data privacy. We’ve had a lot of conversation, from the global majority when we think about, what’s been happening, let’s say in India, in Africa, in Australia, when it comes to online safety and child protection, really forward-thinking approaches there.

When we think about the U.S., we have had a many, dynamic discussions in this country. When we think about what’s happening with AI safety, and we’re seeing that we’re coming together on questions around AI safety, questions around ethics. We know a few years ago, we got our blueprint for our AI Bill of rights. Then we had our executive order, and this year we had several federal memos really charting an ethical, responsible, and trustworthy course for the ways in which we interact with this technology and the ways in which we manipulate the technology for a full benefit of humanity, as opposed to the technology manipulating us all the time.

So, there’s been extraordinary progress. There’s been major growth, there have been brilliant conversations, there have been event that I have been involved in. I’ve spoken at I’ve participated in panels where we really try to raise consciousness and really try to come up with very practical approaches. Well, that’s not enough. We’ve got to do so much more.

And this call of action should really be a call now at this moment to have the broadest spectrum of communities globally, really having conversations about AI and about data. It’s about civil rights. It’s about civil liberties, it’s about human rights. It’s about equal opportunity, equal access, the ways in which resources are going to be divided and the ways in which we can engage in the future.

So, it is really, really important because this is a very, very powerful technology that is already creating some inequities and some challenges that we are struggling to deal with. So, we need all on board because, I think 2025, this ship is going to be this AI ship is going to be on some rocky, rocky waters.

So, we need to have all hands up there on deck with us.

Satwik Mishra

I started off with, yes, this is a very big collective issue that we all need to put our hands on. And you’ve shed light on it beautifully. But there’s also the other lens on the micro, on the individual agency which this technology enables. And one of the things that you speak a lot about are creative processes. And in creativity, how do you see AI influencing creative processes and in our creative processes, what are the ethical implications of AI’s role?

Prof. Renée Cummings

So, AI and creativity, the intersection of both, absolutely exciting for me. When you think about the imagination, when we think about the possibility, and now we have the ability to combine our imagination with this technology. So, there are two challenges there.

There’s a school of thought that believes that the technology is, making the imagination muscle very weak because many people believe that imagination is independent of, of this technology and there’s a school of thought that believes that collective augmented intelligence and collected augmented imagination is critical to the future. My approach is that we have got to ensure more and more individuals.

and communities play with this technology. This technology has the ability to really advance, so many of the challenges that we have had as a society and to do things in ways that we, have yet to imagine, you know, many moons ago, I worked in, disability rights and, I worked as a rehabilitation counselor.

And I remember thinking in the 90s about questions around accessibility, working with clients who were visually and hearing impaired or who had other developmental challenges, people who were neurodiverse and thinking about neurodivergent communities and what we are seeing within the realm of accessibility, just from a technical standpoint where individuals are able to do things that we never imagined, that’s that’s brilliant.

But then there’s the question of access and who gets to play with the technology, who gets to experiment with the technology, who gets to design and develop and deploy the technology? Those are the questions. And when we think about, the creative industry and many, creatives thinking, about the impact, the long term impacts of this technology when it comes to the future of their jobs and this technology actually taking away jobs from creatives and many of the lawsuits that we continue to see when it comes to intellectual property, when it comes to copyright issues, we’re seeing that, IP law are really taking, you know beating from the kinds of cases that are being presented and our, our judiciary and our criminal justice system having to stretch your own imagination to really understand the impact of large language models and frontier AI and Agentic systems.

So, this is the time for creativity in every discipline. And this is a time for us to even have a deeper understanding, again, of how we can use this technology, the kinds of new creative approaches that are being developed, the fact that we never thought a prompt engineer would have been a profession, because those who are more creative come up with the most creative problems and the most creative prompts give you the most creative kinds of outputs.

So, we have got to really reimagine imagining in the world of AI and that’s something I’m very committed to when it comes to the imagination, when it comes to play and when it comes to, the radical imagination that is now required, for existing in a world of AI, GenAI and Agentic systems.

So, we’re built for this task, you know, the human brain is what has created AI. So, when we think about imagination, for me, we have the ability to just imagine so much more possibilities. But an ethical imagination, an imagination that understand the, you know, the context of this technology and why accountability and transparency and explainability and all of these things are so important as well to the process.

Satwik Mishra

The reimagination of imagination that’s so wonderfully put, but also so while we talk about this inclusivity and, bringing more people, to play with this tool to realize its potential in a lot of your work you’ve developed and a lot of accessible pathways for people to participate meaningfully in this entire process, right from the design to the development to the deployment.

I want to get to a question that I think a lot of us think about very deeply, and I wonder if anyone has a very comprehensive answer. Every answer is right in its own way. But how would you define meaningful participation and like from your work, what are your key takeaways from building inclusive spaces such that we create what you talk about ethically resilient AI systems? What does meaningful participation in the world that we occupy look like to you?

Prof. Renée Cummings

I think it looks like access, it also looks like equity, and it also looks at the structures that are required to protect. I think if we don’t have access to the technology and we don’t have the creative confidence to play with this technology and understand how this technology can be used for the kinds of solutions that communities, societies, you know, globally, we will require, then you’re not a part of this movement.

So that access is critical, resources, also critical. We know that across this world there are billions of people who do not have access to the internet, who do not have access to this technology, who have, probably never engaged, with its technology. And that is also critical when we think about resources, when we think about the infrastructure that is required.

And also it’s about, the exposure as well to those ethical questions because when I think about data and I think about AI, I always think about data rights, and I always think about algorithmic justice, because the other part of that exposure is understanding although you may not have access and you may not have the resources, you already part of a large data set. AI is already being used to make decisions about you.

AI is already a part of your future. So, I think for me it is how do we broaden the access, how do we share those resources, how do we engage all communities and particularly when we’re thinking about young people, for me, it is a great concern that some groups may have more access than others, denying other groups and opportunities to thrive.

One of the things I always speak about is legacy and when I think of an algorithm, I always think about this as, algorithm can deploy a legacy because it either creates or denies it creates access, denies it, creates resources, denies it, creates opportunities, deny it. So, we’ve got to think, when we’re building these algorithms, questions around agency and questions around self-actualization and questions around identity development.

And this is why accountability and transparency and explainability and impact assessments and auditability are so critical to the work that we’re doing within the space of AI and data, because, to get that kind of interdisciplinary participation, we’ve got to think about questions around intersectionality, and we’ve got to think about how do we create the most open access to this, this critical resource called AI and of course, data.

Satwik Mishra

On intersectionality and the need for interdisciplinary lenses for these systems. I know that you currently teach a course on how to communicate effectively about ethical challenges in data driven technologies. While if we recognize the importance of meaningful participation and you shed light on it so beautifully, what would you think are the lessons for this communication, for this technology which is so important. What are we doing right in our communication strategies to inform society from industry, from governments, from academics, from organizations like Centre for Trustworthy Technology, or from any of the people working in this space

What are we doing right in communicating effectively the stake which we are operating at today, and what are the places that we need to work more effectively on?

Prof. Renée Cummings

I think what we’re doing right would be those conversations around ethics, safety, governance. We’re doing that really well, and I think we’ve built an extraordinary amount of momentum around the things that we need to do when it comes to really looking at research around AI risks when it comes to really looking for common testing practices when it comes to developing and promoting the kinds of protocols for for global inclusion and for evaluative processes.

You know, we’re doing that well and from that geopolitical level, we’re doing that. To me, I mean, there’s always room to do more, but I think we’re doing that very well. Our challenge continues to be, how do we build these conversations from the ground up, how do we communicate to just everyday people the kind of social responsibility that’s required at this moment

Because people are using this technology, people are engaging with AI systems and many individuals continue to find themselves in, in really challenging places while we think about criminal justice, when we think about health care, when we think about public benefits, when we think about, you know, even the financial sector, the housing sector, we’re seeing those biases and discriminatory practices there.

So, I think when it comes to communicating, we need to really bring together, the communities and bring them into this conversation. You know, I’m, I am so forth at it and so committed to that. But we also got to understand that many of the things that we’re communicating, creating these binaries, you know, the doomsday, versus whether or not, you know, the, the evil, versus the excellence, you know, we’ve got to understand this technology is a tool one

It’s something that you can press into service. And it’s also a style of communication. We now communicate using data. We communicate using AI. As there are many languages in this world, a new language that we are using is this language of data and this language of AI. So, it comes back to the literacy in real time. We need to upskill all of us globally in this new communication style.

And of course, we’ve got to ensure that people have the skills to use the technology for it to deliver the efficiency and the effectiveness and the excellence that it can deliver. But we also need to give that other critical education, which is around the ethics, which is around the bias and the discrimination and the risks and of course, we need to really lift up that conversation on rights.

And I think our legal system, or our judiciary have got to also upskill themselves in real time, because there are things that need to be said at the level of the judiciary. That is so critical. There are things that need to be said at the level of world leaders that they need to be a little more direct in articulating, those risks and of course, really encouraging people to engage and really looking at who can play with this technology and who cannot.

Satwik Mishra

Back in engineering more than a decade back, I had this professor tell me, one of the biggest things you should always fight for is indifference, not passion. People will disagree with you, but as long as they’re passionately disagreeing with you, it’s fine. And I think effective communication in this space is so important, so that nobody becomes indifferent and realizes the value of checking into the space and gets involved.

So, Prof Cummings, this was absolutely beautiful, my final question to you is, what is your most hopeful vision for this inclusive and trustworthy AI future? What are the practices that you really hope we will be able to embed and adopt in let’s say a 2-to-5-year horizon.

What are the practices you see embedded and you think that we would have done some good work, in building an inclusive and a trustworthy AI future?

Prof. Renée Cummings

Well, I will say this. I cannot even imagine what AI is going to look like in five years. I think we can’t even imagine two years. But my hope is that we truly commit as a world to ethical, safe, responsible, and trustworthy AI.

 My hope is that we understand to truly reap the rewards of this technology, we have got to protect rights, and we have got to evaluate and respond to those risks. We’ve got to provide the requisite systems to deliver redress, to ensure that there are reparative approaches to ensure that there are restorative approaches as well. We’ve got to also look at the resources and look at the environment and ask ourselves, do we need all that we are creating?

You know, when we think about this technology, when we think about the kind of capacity we are trying to deploy, “Do We Need It?” is a question we’ve got to ask ourselves and what is the cost? Not only the financial cost, but the cost on humanity, the cost on us, the cost and on our future? And what is this technology doing when it comes to our legacies?

And one of the most critical aspects of understanding AI is understanding the legacy it deploys and understanding dreams. I’m so committed to dreaming. And, you know, now we dream in a, a datafied world and my approach is that we must never allow data to defer a dream. We must never allow data and AI to undermine a legacy.

So that is why data equity is so critical. That is why equity is also very critical. And that is why we have got to understand how important AI and data governance is. And we’ve also got to understand that we have always, got to, to protect, some of those things that make us truly human and in protecting those things, that means that we understand why we need to protect each other and that is so critical to this work.

Satwik Mishra

Professor Cummings, to building a trustworthy future and creating a legacy that we all will be proud of. Thank you so much for participating in Trustworthy Tech Dialogues.

Prof. Renée Cummings

Thank you so much. It has been wonderful conversation. Thank you.

Satwik Mishra

Welcome everyone to another exciting edition of Trustworthy Tech Dialogues. I’m Satwik Mishra, Executive Director for Centre for Trustworthy Technology. In this episode, we will dive into the power of designing data architectures that have the potential to shape a future that is both trustworthy and innovative. Today, industry has emphatically embraced the transformative power of data to drive emerging technologies and unlock their full potential.

But is the public sector keeping pace? Governments are pouring resources into AI and other emerging technologies, aiming to transform the economies. However, there exists a challenge in effectively monitoring and measuring the spending. To truly understand its impact, a paper released by IMF this month brings out a stark analysis. In the 1980s, the total U.S. R&D investment was 2.2% of GDP.

Today, it stands at 3.4% of GDP. Conventional economic models suggest that this rise in R&D spending could have spurred faster economic growth. But contrary to expectations, the economy has experienced a slowdown instead of an anticipated growth. Between 1960 and 1985, the productivity growth averaged 1.3% annually. In the following 35 years. Productivity growth constantly fell below this historical average.

That brings forth the question how can we ensure that these investments are smart, strategic and backed by data and evidence to truly make a difference? Where is the public sector excelling? Where does it need to improve, and what changes are necessary to stay ahead? Today, we dive into the transformative potential of building data architectures which can ignite innovation, inspire fresh ideas and power economic growth.

We’ll also analyze the impact of artificial intelligence on the economy and the future of work. This remains a hotly debated issue, with predictions often pulling in opposite directions. Some argue that a productivity boost is on the anvil. Others warn of a possible hype. Some say we are headed for a jobless future, while others argue that AI will surely elevate the quality of work.

Increasingly, it seems that when it comes to AI and future of work, the only thing we can truly count upon is uncertainty.

That brings about another question, are we capturing the right data to truly understand what’s happening in our economy? If yes, what does this data actually indicate? And if not, what do we need to do to get the answers that matter?

To discuss all of this and more? I’m delighted to be joined by Dr. Julia Lane. She is a renowned economist and statistician who has brought her quantitative skills to public service. Dr Julia is a tenured professor at New York University’s Wagner Graduate School of Public Service. She recently served on the Advisory Committee on Data for Evidence Building and the National AI Research Resource Task Force.

She currently serves as the Secretary of Labor’s Workforce Innovation Advisory Committee and the National Science Foundation’s Advisory Committee on Cyberinfrastructure. She’s also the author of a fascinating book, “Democratizing Our Data: A Manifesto”, which I highly recommend to everyone. In her extensive public sector experience, she has developed innovative and practical approaches to design in varied data infrastructures. Let’s dive into the potential of building optimal data architectures. To help us navigate today’s uncertainties, enable smarter decision making and as importantly, advance societal trust.

Dr Julia. Welcome. 

Dr Julia Lane

Thank you so much. I’m thrilled to be here.

Satwik Mishra

Walk us through your journey from the quantitative background in economics and statistics to the world of public service. How did you come here?

Dr Julia Lane

I was really interested in answering the actually partly the same questions that we still have, which is what is the impact on investment on the job training on the economic productivity, the earnings of workers and the survival and growth of firms. So, this is 30 years ago. You can’t answer those questions with survey data because most workers work for big firms. But most firms are small. And you can’t get information about workers over time and firms over time. So, the only way to do it was to develop new sources of data.

So, I thought, okay, let’s go ahead and try to do that. That turned into a long journey. I created the LEHD program, Longitudinal Employer-Household Dynamics Program at the Census Bureau and that integrated data infrastructure in New Zealand, as you can tell from my accent, I’m a New Zealander.

And what I found was this foundational investment in data is way more valuable than writing a paper that maybe half a dozen people read, or even writing a book that thousands of people read. But that was really what hooked me into investing in data. That massive gap for firms, for businesses, for government, it’s a public good, and you can make a huge difference by investing in it. But that’s how I got into it, just by chance.

Satwik Mishra

I do want to dive into all the current affairs that are happening, and I would love to know your views on all of them, but I’ve spent the last two weeks reading your fascinating book, and I don’t want to, get into current affairs before getting you to speak about the book and its key ideas.

Give us a snapshot of, and I wish we could do an entire podcast on that book itself, but, give us a snapshot of that book’s recommendations and what is that vision for democratizing data?

Dr Julia Lane

The book really came out of a sense that despite the enormous potential, when I first started the LEHD program at Census Bureau, which was in the mid-1990s, you could see the world changing potential from data, from moving from surveys to new types of data.

I went from census to the National Science Foundation; I could see what the computer scientists were doing in security and access and converting texts was in 2004. So, I thought, oh my God, we could just securely make data available, we could democratize data. There’s all this potential that would galvanize the statistical agencies.

After 20 years of pushing and saying and opening the doors that the federal government couldn’t move and didn’t. It was still the same old stuff and I was brought to the White House to work on the federal data strategy.

And I was having a series of conversations with people, and they were saying, it’s never going to change. There are all these wonderful people. There’s a lot of money getting spent, but nothing is ever going to change. So, what the book is saying – “What are the consequences of not changing?” What does that mean for the people of the country when they don’t have the money to make decisions?

Not the big companies, they always kind of have the money, but economic intelligence, the small businesses, the students who are making career decisions, the policymakers who are trying to make informed decisions, one what are the consequences and how do we get out of it? How do we reimagine public data? So, that’s the notion of the book.

And the recommendation is very much the hap build within the government. You need an independent, trusted, data driven resource that is complemented by high quality research but driven by local needs. So, you get finely actionable information that can be used proactively, not just reactively. So that’s the book in a nutshell. 

Satwik Mishra

There are so may practical recommendations in that book. But if you are to look at today’s environment, give us one example around how informed, granular or dynamic real time data would have helped make better public policy decisions. What is that one aspect in the economy, you feel like if he had better data right now. I know there would be so many, but we would make better public policy decisions.

Dr Julia Lane

Well, I’m not going to take one. I’m going to take two. So, one is COVID, and I’m going to give you a concrete example in just a minute. But then the second is the point that you opened up with which how do we react to this and better manage the investments in R&D, which we hope are going to stimulate the economy. So, can I do the quick thing on Covid first and then the R&D?

Satwik Mishra

Absolutely.

Dr Julia Lane

The COVID example is a classic one. COVID hit and, we had this massive impact on the workforce. People went home. They were getting laid off.

How you went from .5% of all people, or 2% of people going and asking for unemployment insurance, to 25%. The workers who were administrating unemployment insurance were sent home. They didn’t have access to the data. A lot of them had the data on their physical machines. They didn’t have laptops, they didn’t have access.

They didn’t know how to produce information to the governor’s offices and the workforce boards at a local, granular, timely level so that you knew who was getting hit, how to get help to people and infrastructure laid bare, the inadequacies of a system that was designed a 100 years ago. Now, I had set up the Coleridge Initiative so, the states that had put their data into a secure environment, informed by my NSF experience on cyber security and so on, and they were able to get access to the data, produce dashboards that could be provided to the workforce boards and the governor’s offices. Illinois was the leader in this. They had they got the data about the layoffs and the unemployment insurance claims on a Monday.

The data were in the summaries, the tables, the dashboards were in the governor’s office by Thursday. That is transformational to answer your question, timely, local, actionable. So, take the same basic idea we had. I was on the National AI Task Force, asked to figure out what we need to spend money in order to do better industrial policy, essentially, what are we going to spend it on?

And the task force was asked to say, what are we going to spend? And of course, people were saying compute, data training and so on. And I said, how are you going to evaluate that?

Exactly the question you opened up with. We don’t have an infrastructure to answer those questions, proactively not reactively. You actually said that 20 years later, like with NAFTA, the North American Free Trade Agreement, 20 years later, we didn’t have enough data on there were these deaths of despair from the middle parts of the country that lost their jobs.

How do we avoid that?. So, as part of that national AI task force, I said, what data do we currently have? The data doesn’t make sense. The measures of AI that were put out, for example, by the Stanford AI ethics, which is what the committee relied on. They’re changing their ways, is based on Bibliometrics, which doesn’t tell you anything about the workforce.

In the MIT press piece that I circulated, and you’ll probably provide a link to it. The Census Bureau provides an estimate of 5%. What are the consequence of that 5% of firms who are using AI?

So, what are the consequences of, of not knowing the answers? Well, let me get very concrete. Since you opened up on productivity and the differences at MIT’s Daron Acemoglu will think that AI will increase by GDP by 0.93% over the next ten years.

Economists at Goldman Sachs say AI will increase GDP by 6%. Our task force had to figure out which one was right and where does the money go. In that order of magnitude between those two estimates is 1.6 times larger than the world’s third largest economy, Germany, over the decade. So, these are not small amounts. These are really, really big amounts.

And it means part of the issue with this is one of the questions is that is knowable. The data could be created and figured out, what proportion of firms are using AI, what proportion of jobs are going to be replaced, what types of new jobs are gonna be created and finally, how skilled is workforce and how can it be retrained?

All that’s knowable. So, building an infrastructure that can answer the question of what is the impact of R&D investment on productivity and the workforce not currently known. Big differences. But knowable and in relatively short time period.

Satwik Mishra

On this. This is a perfect segue way to, speak to yours industry of ideas, which is such a fascinating idea again and I’ll link again that paper in our show notes. Speak to us about the industry of ideas, not only its potential, but also where do you see it making a difference, and what do you think are its pitfalls or challenges which may come about if we are to go down that path?

Dr Julia Lane

So, here I am on the NAIRR task force. We’re trying to figure out where should, we were charged by the President, Congress writing a report on where to invest. Many economists say “you need data to decide”.

You don’t just make big numbers up. So think about it. AI quantum computing or not, all of these new areas of investment and industrial policy, none of them has an industry code. None of them is a scientific field. So, if you can’t even measure what you’re investing in, when will you get it? So what you measure is a management saying what you measure is what you get.

So again, I’m an economist. You think, okay, how do you think about what these terms mean? And really it is a new form of economic organization. So how are we going to describe that. And I’m going to argue industry of ideas. And let me tell you how I got to that piece. A 100 years ago, which by the way, as I said before.

Satwik Mishra

Can I just intervene just one. One sort of quote from your nature article, which I found fascinating. And then we can absolutely get into this idea, which will set it up for the audience. So, in the nature article, you wrote, which we’ll link in the show notes, the North American Industry classification system was modified in 2022 to add a single category for AI Activities, AI Research and development laboratories. In February, Adam Leonard, the Chief Analytics Officer of The Texas Workforce Commission in Austin, applied this new NAICS classification to Texas data and found a mere 298 AI research and development firms employing just 1021 workers in total, real workforce,

Dr Julia Lane

Entire state in the entire state of Texas

Satwik Mishra

Exactly. The real workforce involved in AI related activities, meanwhile, is likely to be much larger and spread across multiple industry sectors, ranging from hospitality and healthcare to oil exploration and more. So yes, what is the industry? Let’s segway into the industry of ideas.

Dr Julia Lane

And so, thank you for jumping in right there to make a very concrete. So, 100 years ago, we classified firms into what they produced, and that was mainly manufacturing and agriculture. So, we had thousands of manufacturing, agriculture industries. Then 50 years ago, it was, we’re not just producing physical goods anymore.

So, we need to change from SIC codes to what you referred to as NAICS codes, NAICS codes that are based on services. So, how things are being produced. That was 50 years ago, and it took ten years to get those categories. Now, as I’m looking at it, those new industries – AI, quantum computing, these are industries of ideas. The basic idea, the innovation is ideas, new ways of putting things together.

As Heidi Williams has pointed that innovation is what breaks the shackles of scarcity because you no longer have to just invest in capital, labor, energy, material services which we have. But if I have a better idea, you can put the same inputs together in a different way, like Steve Jobs did with the iPhone. That was nothing new. It was just pure innovation.

There are so many things, that’s what AI is. So how do we track those ideas? the reason that the industry of ideas is such an important one is ideas are embodied in people. It’s not things, it’s not services. Its ideas embodied in people. So, the whole idea between investing in R&D is investing in great ideas in great people.

So, and not just to give you credit, Paul Romer got the Nobel Prize for this Endogenous Technological Change back in 2018. He called it the “Economics of Ideas” that you think about that is, ideas don’t get consumed like capital. Right. Physical capital that one of the key ideas that comes out of that is, first of all, that creating a market with incentives for people to share ideas is massive because you have a great idea.

The more you share it, the more impact it has. Unlike steel when I consume it, it’s gone. The second thing is, going back to your opening statement, which is then you need to figure out where to invest in people. That has demographic consequences, geographic consequences, workforce consequences. And then it tells you how to build the data infrastructure that has been being built over the past ten years, about which we wrote the Nature piece.

So that’s the key point here. It’s a new economy. Firms are organized around ideas much more than services and industry. And so, the data infrastructure needs to reflect the economic organization.

Satwik Mishra

So, from the industry of ideas, give us an analysis of the issue, which everyone’s debating right now. What would the future of work look like? How would you think about the measuring and monitoring of how jobs are changing?

What needs to change in our measurement of jobs so that we’re able to analyze artificial intelligence and impact on our workforce, our industry of ideas translate into that measurement?

Dr Julia Lane

My first answer is going to be you don’t send out a bloody survey that no one’s going to answer. But, you think strategically about how are you going to trace that through digital fingerprints? Right. So, what has been being done on AI is the first thing is the shock to the economy is coming, as you just pointed out from R&D spending.

That’s the first thing that they’re going to need to figure out. How do we track that? Well, that part is straightforward, because when a government or a grant, a contract is issued, there’s a code that’s generated “Abcdefg”. When that contract hits a firm or a university, I’m going to focus it on universities for right now, a code gets created within the university or within the firm.

Everyone who charges time to that project is captured, and so are the vendors who are paid. The people who are producing new types of computers, the GPUs, the Nvidia’s. All of that is tracked in the HR and finance system. So, immediately you have a data infrastructure that describes the scientific production function and who is working on these new ideas.

And this is what we talk about. Now, how did those ideas that are in the university get transmitted to the private sector? Well, it’s not just the principal investigators.

Our research shows using these data, which have been built up at the University of Michigan, the Institute for Research on Innovation and Science over the past 15 years. So, this again, and it goes back further because universities were able to pull it out.

So, it goes back 20 years. We show that there are 22 graduate students, postdocs, undergraduates, clinical researchers, the every principal investigator. You can see those people. Those are the sparklers like Larry Page and Sergey Brin. Those are the sparklers that go out and create new fire into the economy. So, what those people were taught because they described it in their proposal.

The funding agencies know what’s going on. You can then track that. You could see the firms that they landed. Many of them get hired in at high wages because they’re expected to create that spark and then you can see what happens to the jobs in those firms and the skill contours of the workers in those firms, because that skill information is available from additional sources, both the links and to the educational information from the public schools, as well as what’s hiring information with the job vacancy data.

That immediately tells you, its local, it’s timely, it’s actionable, and it enables you to figure out proactively what’s happening to the workforce and the hiring patterns of firms that are affected.

Satwik Mishra

Absolutely. So, this answers the question around why the industry of ideas. Another fascinating piece of intervention that you make in your writing is, answering the “How”. And you speak about your proposal for a Centre for data and evidence.

The institutional intervention, which is, if I may say, which is required in our data ecosystem. Walk us through, what would what is that proposal for Centre for Data and Evidence? Why is it important and what do you think should be the structural arrangements around it?

Dr Julia Lane

So, I think what’s important is getting new eyes in to solve these problems. There are so many smart people out there who are already cited MIT people, but also people in other institutions throughout the country. We shouldn’t just focus on the top institutions that need to be many voices. So, the basic idea is to get new ideas in, there can’t be the Federal Statistical System. Just to be clear that survey methodologies and statisticians are not the ones with the ear to the ground.

It’s the researchers, the local researchers, the people who are on the ground trying to provide services, community colleges and so on. So, the questions need to be driven from outside. That centre should be independent, nonpartisan, trusted think tank, but a different think tank from the ones that we historically see it needs to be, grounded in the idea of bringing together data from multiple sources to make sense of it, just like IRIS has done, for example, at the University of Michigan.

But many other things like the multi-state data collaboratives, that have been set up by the NASWA, the National Association of State Workforce Agencies. So, people who know how to analyze data and people who want to understand why it needs to be built on top of an institutional infrastructure. So, you’re not building castles on sand. You’re building a firm foundation.

What does that look like? In what I’ve argued is philanthropic foundations are going to be central to getting that set up. They’ve been very successful in setting up institutional infrastructures in the past, but they have to be pure of heart.

You could also have funding lines that go to the states through federal agencies that also want the answers to the questions. Department of Labor, Department of Education, Department of Commerce, Department of Defense, Department of Energy. All of those could provide funding to this independent centre, but the centre would be demand driven, agile, and had the right incentive structure.

Satwik Mishra

We answered the why, we answered the how. All from your writing. But there’s an old piece of yours which I went through last week. It’s a piece you had written in science and technology, issues in science and technology where someone who advocates, as you do most importantly for building a robust data ecosystem, you also identify three challenges of relying a little too much on data and being wary of that.

And I’ll give you the headers, and I would love for you to expand on them. The first one is what you call “Campbell’s Law”, which I didn’t know about but is very interesting to hear. The second one you speak about the lag in investments in things like science & technology and education. That is something that I’d love you to explain the readers.

Why that lag happens in science and technology and education? And thirdly, obviously on privacy and confidentiality. So, expand on a little bit on the challenges on all three of these challenges and what what do you see? How do you analyze these three, particular challenges?

Dr Julia Lane

Why do you start with number one? Why don’t you spell out your take on that law that you’d never heard of?

Satwik Mishra

So as much as, for a novice in stake, I understood Campbell’s law, as the more we rely on a quantitative analysis or to make decisions the more they will be proclivity from society to possibly corrupt that ecosystem because of decisions are made from it. There will be a tendency or a proclivity to corrupt that ecosystem.

And I found that like, it was almost like a light bulb moment when I read about Campbell Law. But this is as I understood it. But I would love for you to expand a little bit more on it. How do you think about it? How do you envisage getting around something like that?

Dr Julia Lane

So, and I wanted you to say it so that I wasn’t I, I feel like if I talk all the time, I’m talking at people. So it’s helpful. So, I’m going to ask you to also summarize the other two. But once I go through that, so, again what you measure is what you get. So, the challenge is coming up with a measure that can’t be gamed.

And people are really smart. So, anytime you set up a measure, you’re going to get gamed in some way. The advantage of investing in ideas and people is harder to game that, right. You immediately switch the attention away from mindlessly producing bibliometrics and papers and patents. You’re swamped and the stuff to really investing in your students and your trainees and pushing that out.

But that can still be gamed. So you just have to pay attention, basically, and change, if you’re getting the wrong results for not the wrong results. But if you feel like there’s bad actors in there, just be aware and change, because people are always going to figure out how to mess things up.

Satwik Mishra

So yeah. And the second one, which is investments in science and technology and education, take time from the time of investment to its impact in society. And it’s, we need to make space for assessing that complexity and almost acknowledging that complexity of those investments.

So, we need to make space in a data ecosystem, walk us through that risk as well.

Dr Julia Lane

So, it comes back to a very basic concept, which is what is your theory of change and what kind of lags do you think?. You’ve got these inputs which are the spend a new immediately in the introduction you said, well it was 2.4.

Now it’s 3.2. So, everyone focuses in on the inputs and the Lisbon protocol said every European country should spend 3% and you’re like, well, why 3%? Why not 2.91 or 3.1? What’s the magic number? So, the theory changeṣ. Okay, we’re going to spend money. Also, the charge to the National AI task force that I was on.

What activities are going to be generated? What are the initial outputs, what are the outcomes and what are the goals? And then how are you going to measure at each step of the way and then, what are the compounding factors? Now for a lot of things, people are complex, organizations are complex. There are a lot of compounding factors.

And so of course, what scientist tend to say when they’re going to be evaluate it but it sounds so complex, you can’t possibly measure how wonderful we are because there are so many things that can happen. And there are two answers to that response. One is, this is why they call you welfare queens in white lab coats on the hill.

The second point is, if you think that ideas are random and serendipitous, you get that argument a lot, you don’t need to send money. If, however, you think, put it simply, why which is an idea is equal to X, which is money beta plus epsilon. Epsilon is your noise, The serendipitous part.

You’re going to say money makes a difference. Then you have to give me some idea of what beta is and the process whereby that beta and that X generate innovation. That’s the theory of change. And as a scientist, you should bloody well be able to write that down. That’s the core idea here. And yes, science is complicated.

So is investment in early childhood, so is investment in health, so is investment in many areas. But you have some idea. And as you go through the process you write, you learn, and you have a better understanding of how that money should be structured to be most helpful.

Satwik Mishra

And the third one the final one which is a little more straightforward. But leads me to my final question. The third one, around is the insufficient protection of privacy and confidentiality, including disclosure of intellectual property and national security risk could lead to a loss of trust and, more importantly a consequent disincentive to participate in sharing knowledge, which is so important.

Dr Julia Lane

This is what I’ve spent my life worrying about these things that privacy and confidentiality issues are massive. That’s partly why setting up secure environments are so important. What I learned from NSA about security is you can have multiple tiers of access. So, as long as your biggest risk of security is people sharing information.

So, figuring out how to indemnify individual behavior, which is what they do in the Defense Department, for example, is going to be critical. And that’s by the way, going back to the Illinois example, there was specialized customized tabulations that were sent to the governor’s office and the workforce board because they were trusted partners. So, you have levels of trust that are institutionalized.

There can be standards that can be set up around those that get that provide that trust. FedRAMP is an example that is being put in place so.

Satwik Mishra

FedRAMP is a fascinating example and how it functions. Right. Yeah. My final question is always looking ahead. As we said in 2024, with all of these debates happening around us as to what’s happening in the economy. What is the best case outcome, as you see going forward and what are the pitfalls that you worry about that we may need to work more on otherwise we are going down that path?

Dr Julia Lane

I’d like to see the centre set up immediately. It’s been 30 years, there’s lots of reasons kicking cans down the road. A philanthropic foundation or a set philanthropic foundation dedicated to the public good should set this up and get it done. All the pieces are there, join together, make massive opportunity that I don’t think we can miss.

So that’s the opportunity that I think can be capitalized on. What are the challenges? I think the incentive structure needs to be rationalized, it’s not publications and patents that rich publishers, it should be investing in people. Immigration is going to be a big issue because it is just a few ideas or you need as many of those bright ideas opened up as possible.

So, that’s going to be a major challenge.

In terms of the privacy and confidentiality, there’s been a big focus on things like differential privacy and synthetic data. The secure environment having multiple eyes on the data is what’s going to generate trust, not black box fake data. So. And all of those things can get addressed in this new set of the data and evidence.

Satwik Mishra

Dr Julia, from the book to the Centre for Data and Evidence to the Industry of ideas, all of these would be individual podcasts. And I could just keep asking you questions because it’s such a fascinating body of work. Thank you so much for being here. 

Dr Julia Lane

You’re a joy to work with. Thank you so very much.

Satwik Mishra

Welcome everyone to another episode of Trustworthy Tech Dialogues. I’m Satwik Mishra, Executive Director, Center for Trustworthy Technology. Today, we are exploring a topic that affects each of our professional lives, ‘The interface of artificial intelligence with human resources.’

In a world where AI increasingly interfaces with workplace operations, particularly in HR, the question arises how do we ensure that this technology is used ethically, responsibly, and with a clear commitment to civil rights to advance society trust?

Now, on the surface, AI offers tremendous benefits. It can streamline recruitment by efficiently analyzing resumes. It can remove unconscious bias in hiring decisions, and it can create more tailored, and data driven approaches to employ management. But with these capabilities come significant challenges and considerations. How do we develop AI systems that are human centered? How do we ensure that AI in HR (Human Resources) respect civil rights, promotes equity, and avoids perpetuating discrimination?

Trustworthy AI adoption isn’t just a technical challenge, it’s a deeply ethical one. It requires the collaboration of technologists, policymakers, business leaders and civil societies to ensure transparency in design, development and deployment. It demands a vigilant focus on safeguarding the dignity, integrity, security and privacy of data that AI systems are built upon. Because at the heart of every HR decision is a person, and that person’s rights and autonomy must be respected.

Today, we will explore all of this and more. We’ll talk about the importance of building AI systems that not only work for businesses, but also enhance fairness and protect civil rights within the workplace. And of course, we’ll dive into practical aspects like – What does AI mean for the future of HR? How can we harness this power responsibly? And what are the steps that we need to take to ensure that AI is used to empower, rather than undermine, the human experience at work?

To discuss these pertinent and timely issues, it is my absolute privilege to welcome the Commissioner of the U.S. Equal Employment Opportunity Commission (EEOC), Keith Sonderling, in today’s podcast. Commissioner Keith was confirmed by the US Senate with a bipartisan vote, to be a Commissioner on the US EEOC in 2020. Prior to his confirmation, he served as the Acting and Deputy Administrator of the Wage and Hour Division at the U.S. Department of Labor.

Before joining the Department of Labor, he practiced Labor and Employment law in Florida. He was a partner at one of Florida’s oldest and largest law firms, Gunster. At Gunster, he counseled employers and litigated labor and employment disputes. Commissioner Keith also serves as a Professional Lecturer in the Law at The George Washington University Law School, teaching employment discrimination.

Since joining the EEOC, one of Commissioner Keith’s highest priorities has been ensuring that artificial intelligence and workplace technologies are designed, developed and deployed consistent with longstanding civil rights norms.

Commissioner Keith, thank you so much for being here.

Keith Sonderling

Thank you for having me.

Satwik Mishra

All right. So, let’s dive in. I want to start off with a little bit about your journey at EEOC.

You’re currently wrapping up almost four years as the commissioner. Walk us through your path to this role. How were you introduced to the intersection of technology and politics?

Keith Sonderling

Well, it’s a great question. And to get there, you have to take a step back and learn a little bit more about my agency here.

So, the US EEOC, for all intents and purposes, we are the regulator of human resources. So, our mission is to prevent and remedy employment discrimination and advance equal opportunity for all in the workplace. So, when you think about some of the larger issues the workforce faces with discrimination, preventing discrimination, all the issues related to the #MeToo movement, diversity, equity, inclusion, social issues in the workplace, pay — That’s my agency.

So, you may ask, what is so significant about this agency and why we’ve been able to take such a global role in ensuring workers have protections related to not being discriminated against in the workplace, which are civil rights protections in the United States? That is because of our founding.

And this agency was founded out of the civil rights movement in the 1960s. So, Martin Luther King Jr. marching in Washington DC, led to the Civil Rights Act of 1964 that created Title VII of the Civil Rights Act, which gives employees, applicants and even former employees, civil rights protections in the workplace to not be discriminated against. Such as characteristics, their sex, national origin, color, race, religion, age, disability, pregnancy — all these very big-ticket items.

So, the United States was one of the first to create these civil rights protections in the workplace. Because of this very storied history, a lot of governments around the world have modeled their own employment laws and civil rights protections for workers in the workplace. So as the global leader, when it comes to these important issues, we really can set the pace for not only the compliance side, but also the enforcement side, because at our core, this is a civil law enforcement agency.

So, that’s a little bit about our mission and how we’re here. And I would like to call that the day job because, what’s also unique to US based employment law is that for most claims, you cannot just go sue your employer in court. You have to come to the EEOC first. Whether you work for a private company, state or local government, or even the federal government.

We see every case of employment discrimination for the most part in the United States. So, it really puts us in a very good position to see what those trends are and to see what the hot issues are. So now, to answer your original question, how did I become interested in the intersection of technology in workplace laws and workplace rights?

The answer is – when I got here, I really wanted to focus on an issue that was going to be proactive, focus on an issue that would help not only workers, but employers, be able to comply with these very complicated federal laws. And if you look at just at the recent history, the last 10-15 years of this agency, very much like HR departments, very much like corporate governance departments, we’ve had to shift what our priorities are, based upon what’s in the news.

So, let’s go back to the recession post 2008, there were a lot of claims of employment discrimination related to age discrimination. Because a lot of older workers were disproportionately laid off after the recession because they were making more. So, we started focusing on age discrimination. And then happened, the global #MeToo movement. And we had to stop everything and then focus on preventing sexual harassment, which of course has been illegal since the 1960s.

But that raised awareness. Then the U.S. women’s soccer team with pay equity became national news. Then, COVID happened. And everything related to vaccines and working from home and those accommodations were right in the center of our agency. Then George Floyd and the social movements within, then racial injustice within the workplace and a lot of those movements. So, we had a focus there.

So, I say this is that there’s always going to be something, for us, for companies on the compliance side, that is going to distract based upon what’s happening in the news. So, when I got here in 2020, I said, how do we get ahead of the next #MeToo movement? How do we start thinking about, proactively, what is the biggest issue for the workforce going to be? And how do we tackle that?

So, I started talking to corporate leaders and a lot of different stakeholders, both on the employee and employer side. And they told me that artificial intelligence in the workplace is going to be the biggest issue moving forward. And like many, especially in the government, I didn’t really understand what that meant. For me, it kind of took me to this dystopian vision of these robot armies replacing human workers.

And I thought, maybe that’s happening. You see that in manufacturing and logistics, some retail fast food where actual robots are doing the job. Well, here at the EEOC, we regulate every single industry across the board. So, I needed something big enough to tackle, and not just a couple industry specific things. But then when I started learning about it, I said, wow, it’s a lot more than just replacing workers.

Which is funny because four years later, after generative AI, now we are actually talking about replacing a different kind of worker with our knowledge workers, which we can talk about. Then I realized how artificial intelligence, machine learning, natural language processing, all these buzzwords I had to learn, were already being vastly used within the HR function, not to displace the workers, but to actually perform HR functions that HR managers have been doing in their entire career.

And to do that was really having software designed, developed and deployed to make decisions better than humans making them. Not just on being efficient, not just being more economical, but dealing with the biggest problem in human capital management, in human resources. And that’s the H (human), because a lot of it was based upon and still is. If we can remove the H from these decisions, if we can remove the humans from these decisions, then we will be without bias.

Then, humans can’t inject their bias into employment decisions, which we know have plagued employment decisions before, during and now where even though these laws are here and taking a step back again. In the last two years, this agency has collected $1.2 billion from employers for violating these laws. So, as entrepreneurs and technologists look at stats like that and they say, well, how can we have artificial intelligence, computers, do that better? And as I started to explore it, I said, yeah, if properly designed and carefully used, AI can potentially make decisions without bias. But then I also realized at the same time, if the AI is not properly designed, or if it’s not carefully used, it can scale discrimination to the likes we’ve never seen before.

So, it really took me to understand not just the AI industry, not just technology, but how impactful it has been, it will be and it is to our space. So that’s why we decided, and I decided to make this a priority for the agency. And a lot has happened since then.

Satwik Mishra

The EEOC absolutely has a fascinating history.

And like you said, with your term, I feel like building that proactive lens of getting ahead of the curve, you really embodied that. And it’s fascinating. And might I say you are being too humble when you say that you didn’t know enough, because I’ve read everything that you have published on the topic, and we’ve all learned so much from your work.

So, it’s absolutely fascinating. Let’s talk a little bit about your work at the EEOC now. As you say that over the last three-four years, the focus exclusively has been on artificial intelligence and the changes which have come about. And now you’ve focused extensively on the balance between the benefits and the harms of using this, at the workplace.

Walk us through the factors which go into weighing this balance. And secondly, it would be really interesting to know from your perspective what potential benefit of harm do you think is most misunderstood or mis-valued at the workplace right now?

Keith Sonderling

Not being a technologist myself and being in the executive branch and not in an agency, like Congress, that can create new laws, I’ve always understood the restrictions and the confines that we’ve had, and that’s how I knew we had to operate to be impactful. So, just taking it a step back. At the end of the day, there’s only a finite amount of employment decisions – Hiring, firing, wages, training, benefits, promotions, etc. And that’s what we regulate.

And that’s what we’ve regulated since the 1960s. And from our perspective to look at how to analyze these AI tools, don’t forget companies are making those decisions with or without technology. And they have been making those decisions, since companies have been in existence. So, in a sense, I try to simplify it and say like, we regulate the outcome of the employment decisions, whether you’re using AI to completely make that employment decision, whether you’re using AI to supplement or augment that employment decision.

There’s still going to be an employment decision. And that’s what we know best. And just that the employment decision is based upon the candidate or the employee’s ability to perform the job, based upon their skills and their merit? Or is it based upon bias, which is unlawful so that just, to simplify things, I always went back to the basics there.

And, when you talk about artificial intelligence making all those decisions in the HR space just so everyone understands, there’s AI out there related to every type of employment decision an employer is making. There’s software out there that will completely draft the job description, a job advertisement. There’s software out there that will do the advertisement or software out there that will collect the resumes, that will review the resumes, that will rank the resumes, that will determine which resumes to interview.

And then, of course, there’s software out there that will completely conduct the interview from scheduling to substantively asking the questions to grading the interview, and then software out there to determine whether or not to make an offer, how much to pay the person. And then once they accept the offer, there’s AI software out there that will look at your social profile and tell you where you should physically sit within the office, what colleagues you should interact with based on AI.

And then when you start working, there is software out there that will be your boss, that will tell you what your assignments are for each day. They will grade you. It will determine what your performance reviews are. They’ll determine if you get promoted, or demoted. And then there are AI algorithms out there that will tell you – you’re fired, if you’re not meeting expectations. So, this isn’t hypothetical. This is out there right now related to every type of employment decision that employers are making, and they’re making it with or without technology. So, when it comes to using technology for these decisions, we’ve historically relied on humans.

I’ve taken the approach that – Can AI absolutely help you make better, more transparent, more fair employment decisions based upon the ability to do the job based upon neutral metrics, based upon industry standards without the human bias?

Absolutely. Is it the best thing in the world? Is it going to completely revolutionize everything? Maybe! Is it the worst thing in the world? Is it going to discriminate against everybody? Maybe! At the risk of sounding like a lawyer, which I am. It all depends. And that’s the trickiest question. And I always felt here at the EEOC, we shouldn’t be in the position to tell employers not to use certain software, because if we are, then they’re going to rely on some of the issues that we’ve been dealing with – Humans – which has led to the creation of this agency.

The decision you have to make is what software are you going to use? What purpose are you going to use it for? What are your existing policies and practices for that? And when you’re using these tools, how are you going to comply with the longstanding civil rights laws?

And that is so complicated because every use of AI technology has a different analysis. So, if you want to use facial recognition in a zoom interview to determine a candidate’s score, whether how much they’re looking at the camera, how much they’re smiling. Grading that on a score, you are allowed to do that except for, in certain states, of course.

But is that going to give you the best candidate or will it potentially lead to disability discrimination because the person can’t smile, or national origin discrimination because it’s not culturally appropriate to smile in that situation, even though that would be the best employee. That is something you need to account for and assess.

And even with some of these programs that look for the best candidates, are they going to be sophisticated enough to deal with some of the very classic discrimination examples we’ve seen in the AI space and specifically employment saying – we can’t take into factor whether you’re a male or female and, you’ve seen cases where the vast majority of applicants are men.

And the AI thinks that must be the number one qualification for that job because there are so many men applying. That must mean something more than skill, more than the ability to do the job, and that will then play an unlawful factor in making a determination if the AI isn’t sophisticated enough to exclude those characteristics, that you’re not allowed to make an employment decision on and actually look at merit.

So much of it is what I said earlier. It’s the design side which the vendors need to deal with, and then it’s then the use side, which the companies can use, and have to deal with. Because in both of these examples, whether it’s the data, SAT discrimination or in the use of not being able to account for disability or national origin or color, if the vendor can then design the software to say, well, I’m not going to be able to have the artificial intelligence look at any of those protected characteristics, such as male or female. And even though 90 out of 100 resumes are male, the AI is not going to look at that. It’s going to exclude all that. It is going to mask all that, and it’s going to look what are the underlying actual skills that these potential men, who were the more qualified applicants in the first round, had? And then where do the females rank within that?

Where does everyone rank neutrally based upon that skill, which is how hiring works. Or in the other example, how are these tools, especially when you start dealing with natural language processing going to account for foreign accent? How are they going to account for disability to allow those individuals who are in that class to be treated equally, which is what the US law requires, ‘Equal’. It’s in our name. And that is so much on the design of the program that the vendors can do. And then on the flip side, you can have the most perfectly designed algorithm, you can have the most perfectly designed tool. And if you don’t have those governance constraints within your organization, if you don’t have that training, the vendor can do everything they’re saying.

And then somebody within your organization can use these AI tools now, that is faster and more efficient than ever before to make an unlawful hiring decision whether that’s including certain people based upon characteristics you’re not allowed to or excluding them. I say that in the aggregate, it’s difficult to just say one broad brush about if AI is going to revolutionize every side, everything in HR space, or if it’s going to cause significant discrimination, it could do both.

But that is on this community, whether it’s the vendors, the users, the employers, developers, all these different stakeholders, we have to come together to deal with all these factors because unlike using AI in other areas, reviewing documents, making shipping routes faster in employment, you’re dealing with what you said in the intro “Fundamental Civil Rights”.

So, if the AI makes a mistake here, you’re excluding somebody from the workforce based upon their religion, based upon their age, based upon their national origin, based upon a disability, etc., and not allowing them to provide for their family because of that at scale, I think that’s the difference here. And why leading with the workforce, leading with the HR from our perspective here at the EEOC, has brought us that attention because everybody gets it.

Everybody has entered the workforce, everyone’s had a resume, everyone’s had a boss and the issues that we’re dealing with related to this AI software is being used equally, whether it’s in the United States, whether it’s in the EU, whether it’s in Asia, it’s the same core issues.

Satwik Mishra

You’ve said this in several other interviews and in your papers, but one of the things is to look behind the wheel.

And it’s not just technology, it’s affecting people, it’s affecting different communities. And it could elevate them, or it couldn’t. Basis on what you said, it depends on how we design it and how conscious we are in our decisions, why we are deploying these technologies. But in your role, what you’ve done is also engaged, like you said in the beginning, engage with a lot of corporate leaders, engage in society, engage with community to understand what is happening, walk us through that approach.

Like how does the EEOC approach and engage in multi-stakeholder dialogue to work these critical issues, and how important has that been in the framing of what you have thought about the current situation?

Keith Sonderling

I think it’s probably the most important thing that I was able to do here related to the AI governance world, related to regulations, related to the use.

And let me let me just explain why. Because, again, you have to take a step back and look at our history. Since this agency has been around, we’ve had a very small world. I mean, a very big world. Every employer, basically every employee, I don’t want to minimize that. But our stakeholders were limited.

They were employee groups, employer groups, unions and staffing agencies that was everyone in this world and we never had technology that nurses, making or assisting with employment decisions. So, we’ve operated in this world of people who understand the issues here. But now you have that technology coming into play and essentially making those decisions about being a big part of decisions.

And then I learned that, taking a step back, who is this new world when it relates to tech and AI, especially in the workplace, it’s such an expansion of those stakeholders. So where does it start? From the very beginning. The VC money that’s going into this that is used to going into tech, I realized that they need guidance and tools of what technology to invest in because they’re not steeped in federal employment law, and they do not want to invest in a product that’s going to violate civil rights.

Then you have entrepreneurs who are extremely talented in actually making the technology, whether it’s from actually being able to code and make the AI or the business ideas to be able to revolutionize some of these hiring processes. And most of them, the common theme is that they are entrepreneurs. They’re tech savvy. They’re not HR professionals.

They speak a different language as well and they don’t want to build with the money they just received a product that’s going to violate civil rights laws. Then you have what I like to say, the group that is in the toughest position, is in the middle, and that’s the companies who are needed to make the decision of what products to buy, what products to deploy. on the front end, dealing with the vendors.

And now they need to speak the language and understand the language the vendors are speaking, and they’re used to buying off the rack SaaS software, which doesn’t work in our space because, the rights and requirements for each individual employee are different. And then, after making the determination of what products to use, they have to then determine how are we going to use this on our applicant pool? How are we going to use this in our workforce?

Ultimately in our space, which we’ll talk about, the company using the product is going to be the one that’s 100% liable for those decisions, the final stakeholder. Arguably the most important are our consumers. And who are the consumers in our space that’s different than other spaces? The consumers are the applicants. The consumers are your current workforce.

They’re you’re going to speak a different language and there’s a different level of trust required for them when being subject to these tools. When I realized that’s our world and everyone is speaking different languages, this is going to be a lot of work getting everyone on the same page, although everyone is on the same page about not violating the law,

You’d hope. We’re not violating fundamental civil rights. But in all sincerity, everyone was looking at it from different directions. For VC – how do we invest in the right products and how do we return our money? For the entrepreneurs – how do we make these companies very large, the companies buying it, how do we use these properly and have more efficient hiring decisions with less bias?

So, it’s taken a lot of effort to bring everyone together to have those discussions to see what all the different, unique concerns are. But more importantly, explaining how these tools are, how we’re going to look at these tools. And in federal law enforcement investigation, which of course is the fear of using all these products. Here is what we’re going to look at.

So many people wanted to say, this is so novel. We don’t understand how an investigation is going to work. And that’s where I’ve been trying to change the narrative and saying, look, right now our investigators are trained to look at employment bias. And how do we do that? By interviewing people by, unfortunately, subpoenas and depositions and litigations.

But if this AI chain can be done correctly through transparency, through accountability, robustness, all these words that I know you’re very familiar with, here’s what they mean in the employment space. What is your data set? for most people in this world, they don’t know what a data set is. But to simplify that saying, your data set, let’s say, it’s your current workforce.

If you’re going to be using AI, what are the dynamics of that? Is it imbalance there, what going to lead to an imbalance, result potentially on one side and how do you potentially justify that. If not? We’re just talking about the scalability here when you’re using AI and then if you’re ever challenged on an employment decision, instead of, having to retrace that now you may have a contemporaneous record that’s defendable exactly what you asked the algorithm to do.

Let’s just say, in a hiring case where you have AI help you with the job description, and you have AI do those things based upon industry standards, based upon your own standards. And you can then show that the AI was only looking at these qualifications that are necessary to do the job, and not the other factor.

That’s how we selected these applicants, we didn’t know what religion they were. We didn’t know what their sexual orientation was or pick any of the factors. Then you potentially have an undeniable record of a proper employment decision in real time. And that’s taking a step back to explain how then you can use that to disprove discrimination.

I think it really helps everyone understand saying – now I understand why these need to be properly designed and carefully used, now I understand why these factors need to be accounted for. Because if starting with the end of a class action lawsuit or any EEOC investigation, which everyone fears, if this is what they’re going to look at, well, I can show that and I can build an algorithm around that.

That’s going to account for that. And I think that was just a different way of thinking. I think most people have never experienced a regulator trying to be proactive, because it goes to my theme of why I got involved in this. It’s because not only do I want this to work, but because it can help eliminate bias if it doesn’t work.

The scale of these lawsuits, the scale of potential victims of discrimination through AI, is just far greater than anything this agency or courts had ever seen before, and that requires learning different languages and dealing with different stakeholders. Many, especially when you start looking at how the global community, had never heard of the EEOC, yet they’re creating products directly regulated by us.

Everyone has heard of the FTC on the advertising side, but that’s just the first layer. When you actually get to the core, when you get to the bias side, which is one of the hottest topics in AI. We’re the core regulators. So, it really took almost injecting myself in that conversation and then have a stake in it.

And once we did, everyone was like, no matter where you are in the world we want to get your side right as well.

Satwik Mishra

So, one of my former professors is Defense Secretary Ash Carter at the Kennedy School, used to always, tell us, just look under the hood. Look under the wheel.

Keith Sonderling

It’s not that complicated. If you’re able to build out the pipeline of decisions which are being made by technology, and if you can create out those structures, you’ll be able to navigate through all of this very well. But you managed to do was absolutely fascinating in the sense of and also you mentioned this changing the narratives, changing the narrative when you came in and took over and went ahead of the curve, tried to go ahead of the curve about what the current marketplace challenges might be.

I wanted to talk one more thing about changing the narrative, because, it gets into the Washington D.C. part of this where it’s just easy for people in D.C. to say, this is bad. And before generative AI, people started citing the benefits. If you look at some of the oversight towards this agency, sort of the original interest from Congress and civil rights groups about AI was all bad.

It’s going to discriminate, It’s going to cause issues for older workers, for disabled workers. It’s going to discriminate based upon race, based upon national origin. And you had heard some of those original horror stories, with the discrimination against females, just the very basic data set discriminations, which are used largely in the AI space, whether it’s from the employment, for the housing and the finance credit. The original kind of thought leadership on how I can discriminate was all bad.

Because, when I realized early on, it’s no longer a question, especially for large organizations are going to use AI in HR that’s when we had to flip the narrative to, like I said earlier, how are you going to use it? What purpose are you going to use it for? Don’t use it because it’s going to be really bad and discriminatory.

So, I think that is also something really important where a lot of governments inherently, because we have a very important role to play to prevent fraud or prevent discrimination, to prevent violations of the law that you really need to take that compliance side of it as well. And saying, if this is where the markets are going, here’s what we’re going to require opposed to scaring you into not using it.

And that was a very, very different approach, that I tried to lead.

Satwik Mishra

This is absolutely wonderful. But on exactly this aspect, I’m going to ask a little bit, to step back and maybe put on a lawyer hat. Tell us, you’re the interface of artificial intelligence at the workplace and EEOC’s work, which has been absolutely fascinating.

But tell us, how does this role shape your perspective on the role of federal government in general, emerging technology guidance. There are a lot of issues that we’re dealing with at the marketplace. And like you said, there are several agencies looking at it. From your experience, what do you think about the federal government’s role in technology guidance?

Keith Sonderling

Well, this is a very complicated question. Again, I’m in the executive branch. So, I take the very narrow approach, we cannot make new laws. We have to enforce the laws that are in the books and the laws that are in the books of this agency are from the 1960s although they may be old, they’re not outdated.

They’re going to apply to all the decisions. And it doesn’t matter to us whether it’s an AI making decision or whether it is a human making the decision, because the employer is the only one that can make an employment decision under our laws. It’s easier for me to say this with the authority and power we have here, this is how the law works.

This is what Congress, all of us doing, and all we’re going to do now, I can’t ignore the complexities of the changing dynamics in how governments want to regulate AI and how they want to regulate AI in the HR space. So, taking out that side of the executive branch limitations. You’re right. I’ve learned a lot and I’ve seen a lot.

And I think the trickiest thing is understanding that without additional resources, without additional help, we are not technologists here. And if Congress decides to give us a lot more money to hire technologists to actually dive into the algorithms, dive into the development and the deployment from a technical side. Yeah, that will significantly change things.

And then it would be a different conversation than just the lawyers dealing with the results and dealing with the technologists on the design, which we just don’t have the ability, we don’t have the funds. Really most agencies aren’t able to do this. I think what you’re seeing now, in the meantime, is states and foreign governments trying to get involved, trying to fill some of those gaps that we can’t do without congressional authority.

And if you look at the HR space in particular, it’s one of the more active ones when it comes to AI regulation. There is just a quick brief history is that the first laws related to AI in the workplace were out of Illinois and then Maryland, essentially banning facial recognition in interviews. Then obviously New York City local law 144 – was the first comprehensive AI HR law anywhere in the world related to artificial intelligence being used to make actual employment decisions and hiring, promotions. Then obviously the EU AI act diving into some of the cases for employment being in the higher risk. And same with Colorado and a lot of proposals and California and New York state.

Recently, just last week in Illinois, they passed another law related to some of the disclosures, and then not being able to use zip codes as proxies for artificial intelligence because of the longstanding discriminations there. So, you’re seeing now everyone wanting to get involved in that space. And my personal thought of this, having that national perspective is that, I commend them for trying to dive into this very complicated issue, but without a federal standard, without these requirements coming from us, especially for national or global employers, it’s going to cause potentially more confusion than compliance.

And I would like to use the example of New York. So, New York’s local law 144 requires that if it’s their own definition of what an AEDT (Automated Employment Decision Tool) means, well, for them, it has to essentially, for the most part, make the employment decision, right. The employment decision versus, being able to say, we have some humans in the loop.

It’s not actually one of many factors. Then it doesn’t qualify as AI even though it’s clearly AI. And if it does qualify as AI need to do a pre-deployment test and yearly bias, not even post those results. But if you look under the hood, what does that bias audit entail on the front end? It allows you to use other employers or synthetic data, which the EEOC would never allow.

And then it only requires you to do the audits on race, sex and ethnicity where we have many more protected characteristics. So, I just don’t want people to lull into thinking that they’re fully compliant if they do these little one offs in certain areas where states care about whether it’s video, whether it’s certain disclosure. And then, I think if you’re completely compliant and be surprised if the EEOC comes out and says, well, you’re using it for terminations and show us that.

And, we didn’t think that was regulated. So, you see kind of the issues there. And then, you’re starting to see some common threads now across the board of whether it’s overseas or here in the US in the HR space of saying, well, what are the actual themes we’re starting to see? And what’s emerging is consent, disclosure of the vendors, of how it’s actually being used.

And also, that bias auditing requirement. And even in the EU, with the high-risk systems requiring those audits then that takes the step back of saying, okay, well, now we have to do audits if we’re using AI in HR, whether we’re in Colorado, California, New York City or in the E.U., how do we do that?

And then if these states and foreign governments are going to start diving that well, what’s the new standard for auditing bias in HR? what everyone said across the world will look to the EEOC. And if you look at EEOC, it is based upon our uniform guidelines and employment selection procedures from 1978.

All right. So, it all comes back to that as progressive as some of these new laws are, the basic standards of how you actually get there, are based on us and are not going to change. And I’ve argued too, look whether or not you’re subject to the EU AI act, whether or not you’re subject to New York local law 144. What are the potential benefits there that you can see and use as yourself as a risk mitigation tool such as the bias-lines? And I’ve encouraged self-audits, since I started diving into this. Because if you’re using the EEOC standards, which of course, all these places require you to do, you have the ability now to test these tools before ever making a decision on someone’s life.

Well, prior to this. When you do an employment test where you have some kind of assessment, you hope it doesn’t discriminate. You do everything in advance to mitigate it. And then if there’s discrimination, you’re liable for that now with these AI tools, you can test in real time to see if the skills and everything you’re asking it to do related to your business is actually finding you the best candidates, or if it’s disproportionately excluding certain groups, based upon something that you could potentially change that is not necessarily for the job which we see all the time. So, even though that’s not required under federal law, even though you may have to do part of it for some places, if you’re doing this in advance voluntarily, you get to root out employment discrimination, not have that risk that it’s going to do wrong and feel much more confident than you can with your regular employment tools.

So that’s a long-winded way of saying, if these state, local, and foreign laws are pushing people to do things that you could be doing voluntarily any way to prevent discrimination, I think that’s a really good thing.

Satwik Mishra

So, one aspect of this is definitely we are starting to see the emerging trends of regulation, what it will look like.

And you’ve done, this was great answer to understanding what those emerging trends would look like, what’s missing and what has been done well. Let’s talk about the two questions that I definitely want to ask you is one of the issues that all of us deal with is what would the future of work look like?

What do you think from your perspective going ahead? Let’s take a foresight. I would never ask ten years ahead because no one would know. But let’s say 2 to 5 years ahead, what would be not the regulation side, but the future of work side. So, what do you think should and will change? And more importantly, as you said, as we’ve been talking about the constants.

So, civil rights movement, the civil rights laws, the 1970s interventions which have stayed constant. What do you think when saying ‘consistent’? And what do you think would change in the next 2 to 5 years and then going.

Keith Sonderling

Well, I think this conversation has completely changed since generative AI. And everything we’ve been talking about so far is using machine learning to make human employment decisions.

The decisions that humans were making, whether or not to hire somebody, whether or not to pay somebody, how to pay somebody. But now we’re seeing generative AI come into place and the displacement of jobs is no longer related to factory workers where you actually have a physical robot. And it’s coming for everyone for the knowledge workers across the board.

So, I think that immediate is the most number one immediate impact in the workforce. And from my perspective on that, I have significant issues like I do with any kind of AI in the workplace. How is that going to be implemented? And I think now with the rush to implement some of these generative AI tools, because every CEO, every board of directors is reading those same studies, we read about – How it’s going to displace 300 million jobs?

Is it going to make everyone 80% more effective in work? You don’t need to hire for these jobs anymore. People don’t have to do what they don’t want to do. Work and they’ll be happier working. You’re seeing a rush to say, well, how do we get that with an organization? And, if that is not built in with not only these ethical and legal safeguards, but more importantly trust, it is not going to go anywhere.

It’s going to lead to more claims of discrimination, because workers are going to feel that they’re just really being pushed out and that they’re actually not the employers not using these tools in good faith. They’re actually using these tools to displace them, and they’re having their employees train their robot replacement. And that really goes to the messaging, the trustworthiness of the name of your organization, Centre for Trustworthy Technology.

And it really resonates in this context, because if you’re not going to account for how are disabled workers going to be able to use generative AI, whether they need more time, where they need more adaptive devices, how are older workers, going to be able to use these tools who didn’t grow up on their phones like the Gen Z workers?

And are they going to really believe that the company is trying to implement this to help them, or are they trying to push them out? And we’ve seen a lot of workers who’ve been doing their job great for 30 years, the highest performance review of their job. And now suddenly, if they don’t become a prompt engineer 30 years into their career, they’re going to be forced out.

So, I think a lot of that on the front end is just how we message this, ensuring that workers with disabilities, workers who are older, who are more vulnerable to this, these issues or job displacement are going to have the tools we need as well, or it’s not going to work.

So, before we can even get there, from our perspective, we don’t want to see this led to more discrimination. We want to have all those safeguards in place to ensure that everyone can use these programs properly and that there’s no fear and anxiety. There’s been some studies out there that most workers, a lot of them, are very anxious to these tools.

And that is leading to mental health issues in the workplace. So, you can see from my perspective, from the laws we enforce, my head immediately goes there and says, how are we going to do this right? And what are those safeguards going to be in place to make sure that’s happening? And like everything else switching gears here, when this inevitably comes, it makes people much more efficient, they don’t have to do the job they don’t want to do, etc., all those studies.

And then we start getting into, well, who are going to get those employment opportunities to reskill, to retrain, to start getting the newer jobs that are going to be created out of this. And look, historically, we see some of these job displacements impact certain groups over the others. And there are some studies just on the quadrants of workers who are going to be displaced immediately, are lower wage jobs, and it’s going to impact certain groups more than others.

So, I think these are just really important dynamics that have to be in play. Because the worst thing is for a company to spend millions of dollars on a generative AI tool that they think is going to help their workers make them more efficient and it leads to mass claims of discrimination because it disproportionately eliminates certain jobs that you can correlate to, national origin, race, etc., which we see happening.

So that’s where I think before we can get into this whole future of work conversation, I have immediate questions and a raising immediate awareness on the implementation of this.

Satwik Mishra

Here, a lot is at stake. And no one does. Like no one brings light to it better than you in this space when we are in HR space. My final question before I let you go.

And we’re coming on time. But this is one of my favorite questions I would love to ask different leaders in their own fields. How do you think about trust? What is your perspective on trust? Certainly, within this domain on AI in the workplace?

Keith Sonderling

Well for me it’s a little different because I’m a lawyer, I’m a regulator.

On the compliance side, there’s going to be no trust if you’re violating the law. There’s going to be no trust. Like I just said, if you’re disproportionately impacting certain groups, whether females over male, right. And like some of these classic examples of who is being discriminated by AI, not just in the employment space, but the housing space, the credit space buying in, you see a lot of these studies of some of the same characteristics of those who have been historically marginalized and discriminated against groups is just being replicated and amplified when it comes to, technology. So, I define trust more uniquely. I’m saying that, well, you can’t even get to the trust conversation, which is incredibly important. In some cases, more important than some of the issues we were discussing.

Because you can legally have everything solid and not have trust and you’re going to potentially lead to those issues. Well, how do you get the underlining right on this? And for me, that’s the civil rights protection. And before you get to define what trust is, show me how you’re going to comply and ensure that these are very important protected characteristics in the US and around the world are being protected before you even tell me the benefits of the technology. So, I think that’s how we get there in the HR space, showing that, like, look, most people don’t trust employment decision making made by humans. And there’s a really interesting study that I encourage everyone to read by the Pew Foundation about this issue in HR. And we live in a bubble.

The vast majority of Americans don’t know this technology exists in the workforce, right. So, at first glance, most people don’t even know the topic. But when you dive into it, you ask people, would you rather be rated by the human or machine? And a lot of people said human first because that distrust of the technology, that distrust of big tech, the distrust of algorithms.

So, when they flip the question, they said, does bias exist? Yes. Bias exists in employment and hiring and promotions. Most people said yes. Well, understanding that exists. Flipping the question now, would you rather trust that same hiring manager or same boss who just had a bias or, a potentially neutral algorithm, and then people said they’d rather trust the algorithm.

So, I think that’s really a good example of trust, because if you could build that trust, then people are going to potentially want to be subject to this technology more than humans. So, a lot of that is from the good work you’re doing in other organizations of trying to lead with trust, lead with ethics, lead with legal compliance.

And this is not a criticism of the tech industry. I just think it’s new because a lot of the products that have been developed previously, whether it’s making data transfers faster, cloud tech, all these things, you weren’t dealing with human, you weren’t dealing with civil rights. You were dealing with business issues of reviewing documents story that all these things and now you’re using this for good, in civil rights areas.

It’s just a completely different mindset of what that trust is.

Satwik Mishra

Sure, Keith. Thank you so much for being here. It’s always such a privilege to talk to you and I always end up learning so much.

Keith Sonderling

Thanks for having me.

Satwik Mishra

Welcome everyone to another exciting edition of Trustworthy Tech Dialogues. I am Satwik Mishra, Executive Director of the Centre for Trustworthy Technology. Today, we are diving into a topic of great significance, ‘Leadership in the era of emerging technology.’ As we navigate these uncertain times, exercising leadership has never been more critical. In today’s world, innovation, creativity, and curiosity are not only buzzwords, but the foundational pillars of a successful corporate strategy.

Leaders are increasingly recognizing that fostering these qualities within their organization is essential for staying competitive, driving growth, and navigating the complexities of a rapidly changing global landscape. Cultivating a culture that sparks new ideas and encourages out-of-the-box thinking has become the key to developing products and services that are not just relevant today, but future-ready. For this culture, curiosity is the engine that drives continuous learning and adaptability.

It is the key to staying connected with the ever-changing ground realities around us and staying ahead in today’s fast-paced, competitive world. Leaders who nurture curiosity within their teams create a culture of exploration and learning, leading to higher engagement and ultimately more satisfied customers. This commitment is particularly vital in industries where stability and trust are paramount, such as the insurance industry.

We find ourselves in an area characterized by profound economic uncertainty, unprecedented technological advancements, escalating climate change, and the imminent reality of looming global health challenges. In this context, the insurance industry is increasingly seen as vital in delivering the stability we so urgently need. It is an industry which uniquely stands at the crossroads of the challenges and the opportunities presented by rapid change.

At the heart of this intersection lies trust, which I believe is fundamental to the industry’s mission. To explore these themes and more, today, it is my absolute privilege to be joined by a global luminary of the insurance industry, Mr. Tapan Singhel. With over 30 years of rich experience in the insurance sector, he has been a key figure at Bajaj Allianz for more than 22 years.

He has served as the company’s MD and CEO for over 12 years. He is a member of the Insurance Advisory Committee of IRDAI and the Pension Advisory Committee of PFRDA. He serves on the board of the Institute for Insurance and Risk Management. He has been the president of the Indo-German Chamber of Commerce and continues to serve as a board member.

He is also a member of the Governor’s Council of the World Economic Forum. Under his visionary leadership, Bajaj Allianz has emerged as one of the largest private general insurers known for its growth, profitability, and customer centricity. He has been awarded innumerable awards and recognitions over his illustrious career, including most recently the CEO of the year at the India Insurance Summit 2024.

Beyond these astounding accomplishments and an enviable body of work, I have had the distinct privilege of knowing him for over a decade. Throughout this time, I have witnessed firsthand, leadership embodied by him, which is not only strategic and steadfast, but also profoundly humane. It is an absolute pleasure to welcome you to the Trustworthy Tech Dialogues. Thank you so much for being here.

Tapan Singhel

My pleasure, Satwik. It is amazing to be with you on this forum.

Satwik Mishra

Let us start off at the beginning. Walk us through your journey at Bajaj Allianz. What initially led you to the insurance industry and how has the industry evolved over the last few decades?

Tapan Singhel

If you go back to the era when I joined insurance, at that time, hardly anybody wanted to join insurance.

It was like somebody who did not do good in life would join insurance. That was the image. And the funny part was that when we had this JB (Joint Business) with Allianz and speaking to some of my Allianz friends, they said, ‘No.’ In Germany, it is a bit different. When you fail in life, you become a bartender, and when you fail as a bartender, you join the insurance industry.

Imagine! That is the image that the industry had in that era. I am a scientist by education, and I wanted to get the Nobel Prize for my country. That was the burning desire as a child growing up. But because of some challenges, my friends said – This is interesting, take this government examination. I joined, public sector, which in India, you know, is run by the government insurance company.

Very early on, I paid some claims, and, when I saw the difference it made to the lives of people, I realized it’s a much bigger thing than what we understand. You contribute to society, to people at a very, very different level. And you can make a huge difference to your own country also by being part of this industry.

And that’s why I stayed on in the insurance industry for over three decades. But if I look at the last decade, not only for insurance but across the globe. Earlier on, when we used to do an innovation or do something new, we would reap the benefits for a decade or two, and you feel like you have done something really amazing.

Today, you do something which you feel is amazing, in just about 20 days, 25 days, everybody else has done it just like that. The speed at which things are happening now is so phenomenally fast. It is exciting. So, if you are not used to this excitement, it can become overwhelming. But if I use excitement and constantly keep on thinking about what next, then you really enjoy this game.

The industry has gone through massive changes in the past 10 years. And we will talk about it as we move further. But I will leave you with the thought process that if you meet somebody on the street, they will have distrust in insurance.

And they would say, these guys do not pay claims. And this is happening globally. You ask anybody, any part of the country or world. You are in the U.S. right now, go and ask somebody what do you think of the insurance industry?

But look at the insurance industry, they pay millions of claims, and their profit margins are actually much smaller than any other industry which is known.

So, the thought is, why should this be the notion when they are paying claims so well? Why should there be distrust in the industry? I will leave you with this question to answer that later on, as we progress in the conversation.

Satwik Mishra

So, a little bit on the distrust. And you have seen the shifts in the economy and society over the last two, three decades in this industry. What I wanted to specially do and learn for myself, and the audience is your distinct leadership style over the last three decades, how has it shifted?

And you speak specifically about the last decade and the changes which have come about. How has the changing environment, the economy, society, changed your leadership style at the company?

Tapan Singhel

Well, is it the economy or my age or the years of being there? Satwik, I can’t define what would be the reason for the changes. But let me tell you how I saw myself some years back, how do I see myself now and how do I see myself going forward?

So, when I was younger, aggressively building the company, I would like to be, very, very strong like an alpha male, as you define it. That is how it would be. And I would feel that ‘this is it.’ Nothing can go wrong. And this is how the world should be, and this is how people should think, and this is how it is.

And way back I remember, there was a professor from Harvard, so he was doing a course about two decades back, and he was doing some profiling. And he told me that, you are like an eagle but the worry is that you will scare the doves off. People would not speak when you are there.

That hit me really hard. I was thinking, why should it be, why should people not speak up? They said, no. You are too aggressive, and you do not listen. But my style was doing very good. I was very successful in what I was doing.

But that conversation really made me think. And I started making efforts to see that I listened to everybody. And then, when I would be on the ground, looking at, people, their difficulties connecting with them, it brought a lot of different perspectives, to leadership, to business, to the way I see people, to what business should do for society, how organizations and corporates can make a huge difference to the country, to society, to individuals.

Those thoughts came to my mind, and I saw a shift in the way I was behaving, with people, with customers, with stakeholders. The level of empathy moved up significantly and much higher. And this also gets very good results. It was not that my early alpha male perspective was the only one giving results. When I moved forward, I realized that it is not only about the materialistic way when we look at things.

How do you connect to the higher plane? How do you see a much bigger purpose in building organization and company and team, which actually is sustainable and last for hundreds of years? So over time, this has been a huge shift in the way I have looked at things.

But if you ask me, the current phase, I think this is better than the phase that I was in the past years.

Satwik Mishra

Eagles, doves, engagement and empathy. That is fascinating. But one of the things which always stands out with your leadership style is your continuous emphasis on engagement. We have seen you engage with probably your employees and more importantly, even customers.

I have seen you travel all across the country and even abroad, making sure that you meet people outside and someone at your position meeting people regularly, what is the value of that engagement with people, customers and your regular employees? What drives you to do that and what is the benefit of something like that in today’s age?

Tapan Singhel

So, Satwik, somebody asked me a question, how do you innovate so much?

So, as a company, we have done a lot of stuff which was first in the world for the industry. Not only first in the country. And I was thinking, how does it happen? And the answer which I gave was – To be able to innovate you should have, two things I feel.

One is you should be eternally curious. Eternally curious means we should be like a sponge, pick up every bit of information and knowledge and be eternally curious. Ask people, be with people. Figure things out. Because your mind is a data lake. The more you fill it up with knowledge, it may be random knowledge, but it is perfectly good.

And second, you should have a high-level of empathy to be able to feel the pain and the concern of an individual, of the society, of the country, of the world at a very high level. When you combine these two together, your mind will figure out the solution and that is what we call innovation. And then you should obviously have the understanding to implement it.

See the transformation and push it through to the last mile to be able to get to where you want. Now to get the understanding, be it business, be it what is happening or what do you feel is going to happen in the immediate near future.

You don’t connect to people at an amazing level in which you are just being how should I say, at your humblest form, where you are able to listen and understand every sentence, every word the person is saying. Be it your customers, your stakeholders, your own team members, be it a person on the street, be it a person in Parliament, or anywhere.

You will not be able to take your organization to the next level. So, meeting people is very essential. Essential to understand business, to understand what’s the requirement, to understand what you do. Also understand from people’s perspective, what they are missing and what you as a leader should be doing.

Because leadership is not the way you think, because it is not about, leading or something like a very powerful marshal on the top. It is the lowest rung in the hierarchy. Where you actually have to be at every place to figure out how you can ensure that people are more comfortable and happier from the time they met you, to the time they meet you next.

Satwik Mishra

It is not just your engagement, if I may say so. Like, not only are you meeting people, but also, you are meeting people with a lens of curiosity.

And I say this because I have been following your work for over a decade, like I said. And I spoke about curiosity upfront because I cover your blog, I regularly read it, and your blog speaks to not only corporate strategy, but also philosophy or personal development, things which are very wide ranging and help individuals at different stages of their lives. How does your curiosity drive this engagement and what, how has that changed over the years?

Tapan Singhel

So, when, if I go back again when I was younger, I would feel that I am God’s gift to mankind. Whatever I say and do is the right way in the world and nothing else. Today, I feel that it’s unbelievable.

Every individual that I meet, everybody that I see, even if I look at a tree. They have so much to give, and I have so little time to understand and absorb. It is a huge shift. And that is why I said is it age? Is it understanding? Is it life? Is it the economic condition?

But this transformation I have seen. And I am very happy with this equation of mine in which I am eternally excited to meet people and to know and be awed by their experiences, the knowledge, and try and figure out, wow, this is something which is so amazing. I think it just hits me. Some people say it happens when you let go of your ego or you become more humble.

I don’t think I really worked on any of this. But maybe the equation that you have when you think that you are God’s gift to mankind, to a place where you feel that every individual is so amazing, I think that shift in your thinking actually opens you up much more to a bigger inner experience and better understanding of life and things in a much better perspective.

So, this has been the shift, Satwik.

Satwik Mishra

It’s absolutely fascinating. And let me go back to another thing that you mentioned earlier about short cycles of product development and innovation, where you need to be at the forefront every 20 days, every quarter, if I may say so, going ahead. And one of the things that Bajaj Allianz is especially known for is innovation and staying ahead of the curve as a leader.

How did you imbibe this within the work force, and how does that drive the company going forward? And what value would you give to this for leaders today?

Tapan Singhel

So, let us understand this. If you are too ahead of the curve, then it doesn’t really benefit your business a lot.

So how do you do that and how does it happen? I will give you two examples and then I will answer my first question I left you in the beginning saying I will answer later. One example is as a company, we are doing very well. You know, with the mid phase.

But the challenge was that we had the old government companies which have been there for close to 100 years. They had bigger sizes in terms of volumes and number of customers. And then I was thinking the difference is that they have more offices in a huge country like India compared to what I have. So, if I had to be ahead of them, I must have more offices.

That was first. Now to open more offices means I have to invest a lot of money in the infrastructure, which would be a huge capital requirement, which means I have to go to the shareholders and my return on the ROI would be much less. That was one thought which was there. The next thought was that I was at that time the chairman of the Indo German Chambers.

And then when I go to Europe and people would talk of demographic dividend. And that India has demographic dividend, and you would not have it. So that was the first time in early 2000, I was thinking, what are the speed brakes? I know about the dividend. But if we are able to provide employment to the youth here, it could actually become negative.

And then, you normally would say the government should do that, but as a professional, how do I contribute to that is the next question.

So, can I generate employment for a million youth? That was the second burning question. Now, if I am able to open offices across the country and I don’t have to invest in capital in terms of the infrastructure, then I can have a model.

Now you have two statements. I can provide employment as a professional to youngsters in my country. I can generate more business and take on competition. Now to do that, it looks very easy nowadays as you have, internet, good cameras and mobile, and tablets. But in those times, the internet was not so good.

But then you have the prime minister speaking about getting internet access to the villages. And maybe NITI Aayog you would also be not working on those stuff in that era. And that conversation is going on. So, when I started seeing the conversations happening, I realized that this is going to happen well and the speed at which technology is progressing, the cameras and the internet speed, and the tablets are going to get better.

So, I told my team to start working on virtual offices. Where you do not have physical presence, everything can be on the tablet. Today, it looks like everybody is doing it. But that time, people told me it was a foolish idea. My board members told me it is a foolish idea. It does not work. That there is no connection, nothing.

My bet was, by the time we develop this, the connections and other things will happen. And that is what happened Satwik. I think by the time we developed the technology, the infrastructure, did recruitment, you had internet in all places, the camera got better. So, we were just ahead of the curve.

We were the first to do that, and we launched one thousand locations, provided employment, generated business. And now we are ahead of most of the government companies in the Indian market. So, this is an example that you start preparing before the technology has arrived at the stage where it is. Because it takes time to prepare.

After it arrives you may be at the curve or just a bit behind it. But after everyone else has done it, then you are too far behind. But the one who just goes ahead of the curve, and you know you have to prepare in a way that by the time the technology hits, you already are ready to deploy it. That is where the money and the reach and the confidence come.

The second example I want to give you is as I told you is the distrust on the insurance industry, even though they are paying so many claims. And they are really paying through the nose a really high amounts of claims.

It is because the process of paying claims is so cumbersome. You lodge a claim, and you say they ask so many questions. They ask so many questions simply because they have public money and they really cannot be, giving it away to fraudsters and all. And that friction has to go. So, you have to make claims, friction free.

And that is when it happened, with machine learning. Now you have advanced AI, generative AI, and blockchain. We used it initially to ensure that as a customer, when you click pictures, upload and I’m talking about like 15 years back and we transfer money to your account. We tried to make it as friction free as possible.

So, when you put all these thoughts together, you realize that, for innovation, you have to put thinking which would take into account the developments happening. And you are actually at the time it happens.

Satwik Mishra

It’s almost about developing the foresight to see the curve not being far too ahead, but definitely not being behind.

It is absolutely fascinating. But also, this is about developing foresight and in the insurance sector you would also navigate you would have navigated through various crises which have happened in the economy. As a leader, is there a crisis that stands out which brought about some lessons for you?

And how would you suggest is the best framework to think about a crisis as a leader today?

Tapan Singhel

So, in India right now, there is a price war which has become very intense. So, I had to address a team some days back and I could sense that they were saying that the time is bad for business.

Then I got on stage. The first question I asked them was that was let’s say in the 23 years since we have built this company, which year that any one of you came and told me that this is an amazing time, whatever target you give, we’ll give you 500 times of that. That means, every year, there has been some crisis or other.

Every year is a difficult year. That is how business will be. You cannot have everything falling into place the way you want it. And you know, then you sit at home, and business starts growing. It doesn’t happen. Which means that crisis is not something to be afraid of. It is something to be loved. And the worse it is, the better it is for you.

Because in a crisis, the good companies become stronger, and better. And the weak and shallow companies, disappear.

Now, are you a good company? Do you have that kind of conviction and standard? This is what will determine how you move forward.

Okay, let’s say these many years I have been, when was something where I felt everything was perfectly fine and I am just happy? It does not happen. There would be some pressure. It’s how do you handle that? How do you see it and what is the point at which you make a company stronger? What are your competencies? In your own life, how do you handle that and how do you make it stronger than what it was, takes you to the next level.

Crises are good. Bad times are amazing. You should be excited about it. The worse it is, the more excited you should be because that gives an opportunity for you to really know, transform and push things. That is the first thought process that you should have.

The second is that you start looking at it. Be realistic about where you stand and how things are.  Have a dispassionate look at it and start figuring out from the end consumer perspective. End consumer does not mean only the client. It’s also your own employees. I gave you an example of Covid also, and how we handled that. And what exactly is hitting them hard?

And how, with what constraints you have, are you going to stand by them and make a huge difference to them? And how do you transform yourself to deliver that? That is what should be the obsession for the entire unit or organization, that you will lead. And that is when you put these two together, you start enjoying it. And then as time progresses, you start emerging stronger.

So, in our case, we are in 2007, completely free pricing. Or when we started coming in 2001, we only had government companies. So why would somebody trust a private company at that time. People would say that Tapan we know you, but how can we trust you? Government agencies not accepting our quotes, to the price going to 90% fall, to an environment not for the profits of the market going to disappear. In 23 years of building this organization, I have seen at least ten huge crises.

But fast-forward today, if I look at the past ten years, 70% of the industry’s profit was made by our company alone. We have 150 million customers organically grown brick by brick. We have the largest distribution of bank insurance in the world. We have 240 banks, and a hundred thousand agents. If we look at it, it is all built organic. Every crisis made us stronger than what we were before that. It is just the thought process that we just have to do.

So, in Covid, I remember when hospitals didn’t have beds, we actually set out 18 ambulances in different cities with six hours of oxygen competence. So, if something goes wrong with the customer or employees, we can actually keep them alive on oxygen and take them to a different city where the beds are available.

So, in every type of crisis, you keep thinking on your feet and see what you can do to be able to handle that. Not just sitting back. I think that is what keeps the excitement alive.

Satwik Mishra

So, this is one aspect of not becoming overwhelmed by the crisis at hand, the crisis will always be there. The second aspect that you spoke about was a little bit of building resilience within your company and within your organization such that you are ready for any crisis which comes ahead.

But there’s also the reality today that there are too many lenses of uncertainty around us. And you spoke about 150 million customers. That’s 150 million different uncertainties, that you have to drive through and navigate through. How do leaders think about uncertainty in today’s time? Like what do you see? And specifically with the lens of emerging technology.

I don’t think emerging technology has ever been as pervasive as it is right now in society. How do leaders think about driving through the uncertainties which have come about from these emerging technologies?

Tapan Singhel

So, somebody asked me this question, I think that is early on when generative AI, when it was just starting. And they said that, how do you look at technology?

I said, to me technology is a tool. I solve problems. And that problem, whatever technology has to be used, I use it to solve the problem. I think that is how we start. It is not that technology becomes all for what I am doing. Now, anything which is new, we are curious and excited about it. And keep on thinking how and what problem that you have where this can become a huge fit.

And how do you do that? It will have risks. There will be disruptions. But the biggest risk is if you are not able to use it well for solving issues, problems or customer experiences. If you are afraid of it and you want to be away from it, that is a bigger risk.

Risks evolve over time. I think when I started working, you would get, salary in terms of a cheque which we had to go collect in cash from the bank teller and carry cash home. And the biggest risk was pickpocketing then, and you try to see how to handle that. Today, the biggest risk is phishing and people taking money away from your account using mobile hacking or whatever it is.

Risks have changed. But obviously convenience has also moved in. In my experience as a customer, it has moved in. Things have become much better. So never be afraid of new technology. It will have risks.

The second thing is how do you build a mechanism where risk correction keeps on happening? Let us look at generative AI. Now there are a lot of hallucinations which would happen. It can go to unethical methodology. So, people are thinking of guard rails. But there should be continuous feedback because there are more good people in this world compared to not so good people. Let us put it like that.

The majority or overwhelming majority are of good people. That is why the world still exists. If the majority was not of good people, the world would not have existed. So, if you have a feedback mechanism of self-correction, where people start contributing and more and more people contribute in the right way, the Generative AI actually gets more powerful in the right ethical manner compared to what it is.

Now if you try to remove all that mechanism of feedback and you are so scared and you put so many guardrails, you don’t allow development to happen. You can’t stop it. Somewhere else. Somewhere, far off, maybe a rogue unit, organization may actually start building, using more feedback from negative. And make it more negatively powerful than possibly it can be.

So, my view on technology and the way I look at risk is, first and foremost, be super excited about all the development. Second, what problem does it solve? Use it well. Third, just keep it very open and allow society and the world to keep on adding there. Don’t make it very restricted because there are more good people than not so good people. Remember this.

That is how it works. And that would actually ensure that it actually gets better and goes in the right way. So that is the way I look at it, Satwik.

Satwik Mishra

So the way I’m thinking, one of it is, from your answer, it is like a philosophical lens in terms of not being awed by the uncertainty brought about by the technology, but flipping the paradigm and thinking about how the technology can be used to address the uncertainty and strategically, developing a framework such that you have continuous feedback channels, which is again, extremely interesting to think about while you are thinking about emerging technology.

But you also, like in your answer, spoke about phishing and the challenges and risks that stem from tech, like in the world today. And I remember Bajaj Allianz was probably one of the first firms to come up with cyber insurance. Now, walk us through that journey of coming out with cyber insurance. And what have been the lessons since then?

Tapan Singhel

Yeah, Satwik. It is again, a story. So, in one of my interactions, one elderly person lost some money because of phishing, and I met him, and he was so sad. And I was talking to him and thinking, as an insurer we are missing out on solving the worry of this person and it is going to happen again.

This was much earlier. Today, every second person is getting this phishing message and people have become smarter on both sides. But this was the early era. And then we said, we have to solve it. So, we came up with a retail cyber cover, the industrial cyber cover was there. But now any individual can buy that and be protected against that. And then we thought, there is also stalking happening on the cyber.

So, people normally do not know how to react with somebody stalking them. How can an insurance company provide assistance in taking care of the legal fees? How do you stop that? What are the issues that are happening in cyber? How are the insurance companies going to protect that? And we came out with the first retail cyber cover in India.

My daughter’s buying it. She was one of the first customers to do so, and we’re super excited. But those were early times. At that time the cyber risk was not so perceived as it is today. So, the policy did not take off as well or as much as I thought it should be. Because normally I am more excited than the time is.

But it started picking up, as the risks are moving in. But again, it came back from same thought process, the risks have changed. You know, nobody’s pocket is getting picked now, we don’t have pickpockets anymore. So, the insurance companies still providing cover against pickpocketing has no relevance, it has shifted.

So, how do you keep seeing the shift happening, the risk and how do you keep moving towards it? And how do you solve issues? And what all risks can you think of even putting out, so the product is also involved, coverage is also involved.

Also, a lot of companies also came forward and the risk also has moved up a lot now. But obviously we have the pleasure of the first to be launching it. And it started from that conversation with the elderly person on them losing some money to phishing. And that again, comes back to the question. You asked me, why do I keep on connecting with people? Because that’s when you realize what is the mess that you’re doing and how do you make yourself better.

Satwik Mishra

Wonderful. So, one particular question that I’ve always wanted to know, and I’m sure that a lot of young leaders today would be very interested in understanding is, you had this distinct pleasure of seeing your firm grow over the last three decades.

And scale to where it is right now. How do leaders think about their business models as their businesses scale, as the industries scale, as they get more consumers, as they probably navigate through the need of a larger workforce in times like this, as companies, think about scaling as we move from startups to probably a resilient company becoming a more standout organization in the world.

What are some of the lessons that you thought have worked? Why Bajaj Allianz scaled over the last three decades?

Tapan Singhel

So, when you are a startup, every decision moves very fast. When you become a large organization, the biggest worry I have is, it becomes bureaucratic, it becomes silos. You have, as I call it in my word, you have emperors with their own kingdoms.

And they do not want to know anybody else’s move. And you have a lot of arrogance which comes in. And that is what destroys organization. Organizations and individuals get destroyed when you reach this place. So, you have to be on a continuous, one, startup pace. Second, always be excited that you have to do much more.

And there’s so much to be done. And know what can you do. Be aware of the use of capital and cash. Be aware that it has to be leading to the benefit of society, the country and the world. And what you do, it has to have an impact on that. Be aware that organizations exist for society. And for that, they have to run it with an ethical, good governance perspective.

And be aware that the individuals in your organizations define the organization, not you. The culture that you have, the people that you have. Be very sensitive towards that. You can’t have a very toxic culture inside the organization and expect them to be taking good care of us when that happens. Be very aware of that.

Be obsessed with the thought process that every day that you get up, what difference do you make. So, when you keep this on and you don’t allow arrogance, the bureaucracy, the layering to come in and politics to come in, you have to keep on nipping at the bud.

Continuously. Be it in communication. It is not that one culture, you change, and it happens. It is the continuous work that you have to keep on working on and build that sensitivity towards that. Then you can have organizations which outlast the average age of organizations which normally is 20-25 years. Because look at the stock market and pick up companies at the top, 20 years back.

Most of them are not there today. So, to outlive that you have to be very, very sensitive on this part. And this is fun and challenging also because the natural tendency of large organizations is to become bureaucratic and to create silos, and to become arrogant.

Satwik Mishra

Tipping the arrogance that may develop as you scale, being very careful about not ensuring that bureaucracy comes in the way of you scaling and obviously being very mindful about the culture within the organization and so that it isn’t leading to any toxic work environments that you are able to grow sustainably.

One of my favorite questions to ask leaders today is – you spoke about the distrust which exists possibly amongst people, with the changing times with so much uncertainty, how do you personally think about trust, both within the firm while you are dealing with your employees and also, more importantly, while dealing with external stakeholders with partnerships, which Bajaj Allianz essentially is a partnership.

What is your take on the importance of trust today?

Tapan Singhel

Trust is everything, Satwik. You know. And trust does not mean that you are judging anybody, trust means that you accept everybody as how they are and who they are, and you believe them. Start from the point that you trust everybody. Never start from the point that you will trust people after they have proven themselves to be trustworthy.

I think that is the first point.

Let’s say if I meet even a stranger, I will start with complete trust and faith in the person and awe of what the person has done there. So, the way you have to look at trust is when you give a person that comfort, that trust, that feeling, they also trust you back. Human beings actually mimic. It’s a very interesting thing.

And, to understand this better, try and do this. So, next time you meet somebody just look a bit cranky. And observe, after some time if you watch them, they will also become stiff. You have done nothing; you have not spoken a single word. And next time you meet somebody, just keep on smiling for no reason, and you will see the results.

So human beings mimic. If you want trust. You decide to trust. It is not a game that somebody has to prove before you trust them. I think that is the first fundamental. You start with trust. What will happen? 100, you will trust. One will betray. Fair enough.

99 did not. So, you’re still winning more than losing. As I said, the world is full of amazing people.

There are a few who are not so amazing. It’s fine. Majority are. So why would you start with the negative?

So, in my view, trust doesn’t come from the expectation of somebody else showing it. It comes from us and demonstrates that we really trust everybody that we meet and talk to and go all out in all that we can do. Even if somebody is not kind to you, you be kind, and you watch the change happen.

I’ve never seen the change not happen. You be more kind. Somebody doesn’t trust you; you trust that person more, and watch the change. So, I think the basic rule is you start strong as we have to trust more and blindly. Even the cost of getting betrayed a couple of times, is perfectly fine.

Satwik Mishra

And, so it’s almost reciprocity, like building a reciprocity from first, from your lens, and then you expect. Once you start building, engaging people with trust, you will get trust back.

Tapan Singhel

Don’t expect. It will happen. You expect, you will have disappointment, Satwik. So, don’t expect.

Satwik Mishra

And my final question is – What advice would you give to young leaders today who are trying to navigate this intersection of technology and starting off in leadership positions, maybe in the government, maybe in the corporate world, maybe in terms of society, what are some essential steps that you think that young leaders should keep in mind in today’s times?

Tapan Singhel

The first thing is – ‘It is not about you.’ Always keep that in mind. It is about everybody around you, first.

Second, what do you feel for the most? Figure that out because that is what you have to get into to try and help make it better. Third, the more you try to make the world better, the happier and more prosperous you will be.

Now remember that. So, if you look at any business, becoming a unicorn or has billions of dollars, at the root of it, they made the world better, than what it was. So as a young leader, try to think how you would make a difference to the world? Then think how can you make it better? Or, forget the world, if not the world, the country, if not the country, the society, not the society, the family. And if nobody else, you. What difference can you make?

I think that is the first thing I always think about is to have a high level of empathy. And it is not about you.

The rest will follow. It is something that if you are so obsessed about position, if you are so obsessed about money and you chase that, then that may not come so soon. If you are obsessed with trying to solve things, to make a difference in society, these will follow very easily.

It is a very funny equation, like the one I told you about how people mimic others. So, do not be obsessed. Designations, power, money, fame will come to you if you do not obsess about it. If you obsess about it, it will take a lot of time. Be obsessed about what difference do you make.

‘It’s not about you.’ Just remember that. And in the end, never forget, be happy, be joyful. Keep smiling. Come what may, no crises will last forever. Even when it comes, smile it through. This too will pass.

Satwik Mishra

Those are wonderful words to end this podcast. I wanted to understand leadership in today’s times and there was nobody better that I could have discussed it with.

Thank you so much for being here.

Tapan Singhel

My honor, Satwik. Always great talking to you.