AI for Earth and Sustainability Science | AI for Good Discovery - Do AI models know what they know? How explainable AI can help us trust what machines learn

AI models can make powerful predictions, from diagnosing diseases to forecasting climate extremes, but understanding why they make those predictions remains one of the biggest challenges in deploying them responsibly. In this talk, the speaker explores what it really means for an AI system to “know” something. Using a synthetic benchmark dataset inspired by climate prediction tasks, where the true drivers of the target variable are known, the assessment focuses on explainable AI (XAI) tools and their ability to recover these drivers under varying levels of noise and data availability – conditions that mirror many real-world scientific and societal applications. Two clear insights emerge: first, explanations become reliable only when models truly learn signal rather than noise; and second, agreement among different explanation methods or models can serve as a practical indicator of whether an AI system has learned something meaningful or is still operating in a state of “epistemic ignorance”. These findings offer guidance for building AI systems that are not only powerful but also aware of their own limits, a key step toward AI that supports science and society with confidence and transparency. Speakers: Antonios Mamalakis Assistant Professor, University of Virginia Moderators: Gustau Camps-Valls Professor in Electrical Engineering, Universitat de València AI for Good is identifying innovative AI applications, building skills and standards, and advancing partnerships to solve global challenges. AI for Good is organized by ITU in partnership with over 50 UN partners and co-convened with the Government of Switzerland. Join the Neural Network! 👉https://aiforgood.itu.int/neural-network/ The AI for Good networking community platform powered by AI. Designed to help users build connections with innovators and experts, link innovative ideas with social impact opportunities, and bring the community together to solve global challenges using AI. 🔴 Watch the latest #AIforGood videos! https://www.youtube.com/c/AIforGood/videos 📩 Stay updated and join our weekly AI for Good newsletter: http://eepurl.com/gI2kJ5 🗞Check out the latest AI for Good news: https://aiforgood.itu.int/newsroom/ 📱Explore the AI for Good blog: https://aiforgood.itu.int/ai-for-good-blog/ 🌎 Connect on our social media: Website: https://aiforgood.itu.int/ X: https://twitter.com/AIforGood LinkedIn Page: https://www.linkedin.com/company/26511907 LinkedIn Group: https://www.linkedin.com/groups/8567748 Instagram: https://www.instagram.com/aiforgood Facebook: https://www.facebook.com/AIforGood Disclaimer: The views and opinions expressed are those of the panelists and do not reflect the official policy of the ITU.