Don’t Skip Data Management for Building Ethical AI Innovations

We’re days away from launching our second AI hackathon. Just a couple of months after standing up our AI Center of Excellence (AI COE) at Compassion International —our 70-year-young ministry to release children from poverty in Jesus’ name—the 48-hour “Ethical AI Hackathon” later this week will explore exciting benefits and learn-by-doing across our data science and business intelligence teams.

Project-Based Learning Builds Ethical AI Muscles

The focus for our hackathon is on project-based learning: Yes, for sure, we’re looking to expand the AI COE’s capability to help others solve problems and advance ministry outcomes with AI. But we’re also, purposefully, learning along the way and understanding what the analytics can help tell us about the best ways to accelerate our impacts, eliminate lossy bottlenecks in our operations, and transform lives as the Lord leads us along this path.

We’re advancing in AI from our ethics, not letting AI advances drive our ethics. This is consistent with Jesus’ “golden rule” to do to others what you would have them do to you. That’s doing in everything—no caveats—that we do (see Matthew 7:12). We’ve, therefore, landed on a specific set of ethical principles (listed below) aligned with Jesus‘ commands to act in ways that are righteous, merciful, and pure of heart toward others (see the beatitudes in Matthew 5:6-9) and that we think every organization and government would do well to implement regarding ethical boundaries for AI and ML:

Better Data for Better Learning & Ethical AI

This upcoming hackathon is a step forward in our efforts to mature our abilities for managing data and advancing analytics well. We can admit that “managing data” doesn’t sound very exciting... That’s okay: It is an arduous, geeky, but completely necessary step to get on to future possibilities ethically. And it’s a series of steps we’ve been leading that will continue into the future. (I recently wrote an article about steps to get started on this path for the Christian Leadership Alliance magazine special issue on institutional risks facing not-for-profits.)

Sometimes people get caught up in the cool possibilities of big data, analytics, machine learning, generative AI, quantum computing, and other possibilities. Getting caught up in the future vision doesn’t overcome the needed investments in workforce training and the data management steps to get there.

?A famous graphic in a 2015 proceedings paper by Google employees makes this point clearly (below). The small black rectangle in the graphic represents the ML code for the operations algorithms that yield great answers and insightful perspectives on data. Those perspectives may lead to benefits like discernment of the right actions for the moment.

But the gray and white rectangles around that black box indicate data management processes that had best work well to operate quickly and effectively for the black box to work ethically. That’s the hidden “technical debt” in machine learning the 2015 graphic references.

For example, consider deciding which direction and how sharply to turn a steering wheel or hit the brakes in an autonomous car. What would be the result, for example, if a hacker were able to introduce a fake virtual stop sign into the car’s video processor in the middle of an interstate highway? Bad for the vehicle occupants. What kind of trusted data management processes, infrastructure, and cybersecurity are needed to prevent that from happening?

Investing Wisely in Managing and Moving Data

Today, I wanted to share a story with you from IEEE Xplore ‘s Spectrum magazine that sums up some important benefits of investing in data management as precursor to future AI benefits. The language learning application, Duolingo , grew out of work at 美国卡内基梅隆大学 . The goal was to try to improve human language learning to match what could be achieved by a student working with a human tutor, presenting the student appropriate exercises at the moment to try to maximize learning.

In a March 2023 article, the developers write, “From the beginning we were challenged by the sheer scale of the data we needed to process. Dealing with around a billion exercises every day required a lot of inventive engineering.” They write about what they learned in employing natural-language-processing tools to accelerate content developers’ improvements and how they learned to gamify the experiences using the app and then learned from the user data which interventions were effective along the way.

What I like most about their story is the pivot they describe when they realized they needed to adopt a new data strategy. Rather than throw away all they’d learned and the data from their early users, they changed the way they managed each day’s lesson data so it could feed the model and be stored and moved in chunks.

They went from a 24-hour cycle of updates to being able to update the app’s predicted model of the learner “within minutes,” more like human tutors might adapt to a student sitting in front of them.

That change allowed the algorithms to take even better advantage of the streams of data, giving them “critical information about which concepts or exercise types were problematic” and letting them increase the speed of new updates.

It’s a story of data engineering geeks working together with those with domain expertise to make the most of every interaction to support the students. The AI wouldn’t work without the data engineering pipelines making the chunks of data available. The data streams by themselves couldn’t have led to the understanding of interventions that would be effective for helping transform learning for the individual students.?

Conclusion: Teamwork Makes (AI) Dreams Work

It’s been a wonderful journey. As we hit 2024, data science teams across Compassion are set to gain more data-driven insights for improving and speeding up the life transforming work on which we are engaged. Not only are we seeking promising impacts on children and their families, but also wonderful new ways to support churches; to connect with, understand, and love our supporters, potential philanthropists, and cultural leaders like Christian artists and sports stars; to motivate and train our workforce and wonderful volunteers; to make the optimal uses of precious ministry resources; and many other possibilities...

Our aim is to put DnA into our DNA: This upcoming second AI hackathon is but one approach we are taking to bring groups together to learn-while-doing, delivering improved capabilities and sharpening Compassion’s own talents for using improving Data and Analytics (DnA) methods, like ethical AI. We’re striving to empower our workforce with trusted DnA at the moment of need to make the best, data-informed decisions possible for releasing children from poverty in Jesus’ name.

References

Bicknell, K., Brust, C., & Settles, B. (2023, March). How Duolingo’s AI Learns What You Need to Learn. IEEE Spectrum Magazine, 28-33. Retrieved from https://spectrum.ieee.org/duolingo

Buytendijk, F. and Svetlana S. (2020, December). AI Ethics. Gartner. Retrieved from https://gtnr.it/33G6M94 (paywall)

?Sculley, D., et. al. (2015). Hidden Technical Debt in Machine Learning Systems. Proceedings from the Advances in Neural Information Processing Systems 28. Retrieved from https://bit.ly/3vgB0Ns

Jeff Collins (PhD, Carnegie Mellon University) writes from the Colorado front range. He loves working to release children from poverty as the Senior Director of Data & Analytics at Compassion International. He is a former United States Air Force commander, Pentagon strategist, Chief Technoloy Innovation Officer at NORAD & US Northern Command , cybersecurity analyst, and the founding director of Air Force CyberWorx. You can follow Jeff on LinkedIn. The views here are those of the author and do not necessarily reflect the position of Compassion International, the U.S. Govenment, Department of Defense or the United States Air Force.