It seems every day brings new developments for the AI industry. At the forefront of it all is ensuring the ethical and responsible use of AI-powered systems and models.
So, what does it look like when governments, countries, and even continents adopt ethical AI frameworks? What happens when AI reveals harmful human biases? What can we do, today, to pave the way for trustworthy AI?
In this edition of Voices of Trusted AI, we answer these questions and explore different approaches to implementing responsible AI.
P.S. Take a look at our newly added section, "Team Panda Picks" down below for a list of top articles recommended by our team of data scientists.
What Do We Mean by Trusted AI?
Trusted AI is the discipline of designing AI-powered solutions that maximize the value of humans while being more fair, transparent, and privacy-preserving.
What it's about: Over the past year, defense and military organizations have affirmed their commitments to act as responsible and ethical AI-enabled users. In June 2022, U.S. Deputy Secretary of Defense Kathleen Hicks signed the Responsible Artificial Intelligence Strategy and Implementation Pathway. This document is intended to ingrain and operationalize AI ethical principles, adopted by the DoD in February 2020, which address common misconceptions and provide strategies for applying these principles based on accepted ethical, legal, and policy commitments.
The DoD AI Ethical Principles are not new, and they will not hinder AI adoption and use. The purpose of the principles is to provide guidance to DoD personnel and relevant stakeholders on how to apply ethics in their roles. Six implementation tenets to ensure RAI activities are executed include RAI Governance, Warfighter Trust, AI Product and Acquisition Lifecycle, Requirements Validation, Responsible AI Ecosystem, and AI Workforce.
Why it matters: Articulating AI ethical principles in defense is a crucial step in the overall effort to produce a responsible artificial intelligence environment. However, it is even more important to implement these ethical principles across the wider organizational culture, operating structures, and any AI-enabled technologies. We need to provide more guidance and tools to achieve operationalization across the general workforce.
The six responsible AI implementation tenets are vital in ensuring that activities are implemented appropriately. The first tenet, RAI Governance, modernizes the structures and processes that allow for continuous oversight of DoD use of AI. The second tenet, Warfighter Trust, aims to reach a standard level of familiarity and proficiency for system operators to be confident in AI capabilities and enabled systems. The third tenet, AI Product and Acquisition Lifecycle, works toward appropriate care in the lifecycle to make sure certain risks are considered from the start of an AI project. Requirements Validation, the fourth tenet, focuses on using the requirements validation process so capabilities that leverage AI match operational needs. The fifth tenet, Responsible AI Ecosystem, promotes a shared understanding of responsible AI design and use through domestic and international engagements. Finally, the AI Workforce tenet ensures that DoD AI workers have a solid understanding of AI technology, development, and applicable operational methods.
While this approach provides clarification on how AI ethical principles should be applied to better serve defense organizations, it can easily be adapted to guide the needs of your organization as well.
What it's about: In a recent article, Cassie Kozyrkov, Chief Data Scientist at Google, dives into the unfortunate “mishype” our society has placed on data and AI. Rather than celebrating how technological advancements could improve business, work, and personal lives, many focus instead on the perceived magic of data.
Because of these misperceptions, Kozyrkov continues, we must also accept that data design is not intuitive for most people. To make data useful, data scientists and forward-thinking leaders should consider primary vs. inherited data and real-world data collection. Getting the right data starts with hiring people who are skilled in making good data and translating existing data into useful applications.
Why it matters (from Niki Agrawal): In data science, it is commonly known that biased and poor quality data will threaten the validity of your model. Today, data scientists spend the majority of their time on data preparation tasks with data cleaning taking up to 26% of their average day, but is this preventable?
In her article, Cassie explains that extensive data cleaning “should be the work of last resort” and is in fact avoidable when good design is applied to the data collection process.
Rather than addressing the symptoms, this design-first approach flips the issue of poor data quality on its head by incorporating guardrails to collect useful and accurate data from the get-go.
Since orderly design of the data collection process is not intuitive or trivial, Cassie suggests that organizations a) seek out designers and value people who are“skilled in making good data” as well as b) consult resources like the Google Data Cards playbook which is a toolkit for transparent data design and documentation.
What it's about: Research teams from Stanford University, Columbia University, Bocconi University, and the University of Washington uncovered a number of ways in which image generation models (think: Stable Diffusion and DALL-E) spread dangerous and complicated stereotypes. For example, simple user prompts in one model of “a photo of the face of a terrorist” or “a thug” generated images that perpetuated disturbing stereotypes. Even more concerning, the harmful stereotypes are hard to predict and not easily controlled and stopped by users or the owners of the learning models.
The paper produced by these researchers reveals three key findings:
Beyond just reflecting disparities in society, cases were found of near-total stereotype amplification.
Prompts that mentioned social groups produced images with complex stereotypes that cannot be easily alleviated.
Why it matters: Machine learning models are getting increasingly better at producing naturalistic images. And as these models become widely available and ever more powerful, we must proactively prevent the acceleration of stereotypes and harmful bias.
But as these findings demonstrate, there is no easy solution to tackling bias in generative models. While some steps can be taken to filter out obvious harmful results, it’s much more complicated to remove the underlying negative bias models inherit from real-world training data.
Ultimately, this presents a sociotechnical problem more than a technical one since the models try to replicate biases in their datasets, and these biases are offensive to certain groups of people. This issue goes beyond AI and requires an overall commitment to analyzing biases and power relations—more thoughtful prompts promoting diversification and more responsible users are not enough.
In the future, we can anticipate debates about what the “correct” biases are for different models to display or not display. The number of approaches could be as diverse as the different ideologies in society.
What it's about: As AI changes the world we live in and increases its presence in our lives, it is more vital than ever to make sure that AI is ethical, lawful, and sturdy. The EU-funded TAILOR (Foundations of Trustworthy AI - Integrating Reasoning, Learning and Optimization) project has worked on developing powerful instruments for AI research and collaboration to advance the industry of human-centered, trusted AI.
One of these tools is the Strategic Research and Innovation Roadmap. This document aims to set the foundation of trustworthy AI in the European continent from this year until 2030. To enhance research on trusted AI, the roadmap identifies major scientific research challenges.
Three outlined objectives include providing guidelines for strengthening and growing the European research network on trustworthy AI, defining ways to advance scientific foundations and convert them into requirements that can be adopted by other industries, and pinpointing opportunities for fostering collaborations between various stakeholders spanning academic, industrial, governmental, and community sectors.
For non-experts interested in gaining a broad understanding of the obstacles concerning developing ethical and trustworthy AI systems, the project printed a Handbook of Trustworthy AI. The document includes an overview of the significant scientific and technical terms relating to trustworthy AI.
Why it matters: The TAILOR project offers important contributions that will aid in reducing AI-associated risks and optimize related opportunities for Europe and beyond. If we want to design AI that not only benefits society but is also trusted, we need to make sure it is centered around humans.
While the Strategic Research and Innovation Roadmap lays a clear plan for the future of trustworthy AI in Europe, it can also be applied to U.S. organizations. By focusing on the major research challenges facing the AI industry, the roadmap positions organizations to face these problems head-on and emerge with stronger guidelines and strategies to advance trustworthy AI design.
The Handbook of Trustworthy AI is an invaluable resource for forward-thinking leaders because it gives non-experts an accessible opportunity to understand vocabulary relating to AI—ultimately improving their AI literacy. If the future of responsible AI is for everyone, we must ensure everyone has the right tools for comprehension.
How can negative bias manifest in natural language generation?
While the proliferation of artificial intelligence use cases have brought about a number of amazing outcomes, there are serious and difficult questions to be answered about bias and discrimination.
According to the Brookings Institute, biased AI/ML models can negatively impact society by discriminating against certain social groups and shaping the biased associations of individuals through the media they are exposed to.
The companies developing and publishing access to state-of-the-art natural language generation models acquire training data through the massive collection of end user language and behavior on the internet. Unless society, humans, and technology become perfectly unbiased, that data will have some bias.
However, there are a couple of practical steps that can be taken to help mitigate the risk of negative bias:
Companies can employ diverse AI talent practicing value sensitive design to help curate higher quality training sets representative of social groups and their needs.
Have humans in the loop at every critical stage of the AI project lifecycle to test and audit that bias is not unduly propagating to decisions about individuals and society.
