Evaluating AI Abilities

In this episode, Kozzy discusses his endeavors to compare the cognitive abilities of humans, animals, and AI programs. Specifically, we discussed object permanence, the ability to understand an object still exists in space even when you can’t see it. Our conversation traverses both philosophical and practical questions surrounding AI evaluation. We also learned about Animal AI 3, a gaming environment developed in Unity where AI programs and humans can go head-to-head to solve different problems in a gaming environment.

Guest

Konstantinos (Kozzy) Voudouris: Dr Konstantinos Voudouris is the cognitive scientist on the Alignment Team at the UK AI Security Institute. He holds a PhD in psychology (2024) from the University of Cambridge. His research focuses on advancing the sciences of AI alignment, scalable oversight, and AI evaluation, using tools from the cognitive sciences. Combining these diverse fields allows us to build better, safer, and more human-like AI systems, as well as informed and sensible AI policy. Konstantinos conducts empirical research in these fields as well as leveraging the funding of UK AISI to mobilise the wider cognitive science community towards these important problems. Konstantinos is a committed interdisciplinarian, with publications and active research projects in machine learning, AI safety & governance, cognitive psychology, psychometrics, the philosophy of science, evolutionary biology, and linguistics.