There is a clear ethical conflict between autonomy of choice and doing what is beneficial for the individual
Y-Lan Boureau, research scientist at Facebook’s Artificial Intelligence Research, was the guest speaker of the TUM IEAI Speaker Series session on Tuesday, 24 November 2020. In this session, she discussed the balance of the issues of trust, helpfulness and responsibility specifically for wellness and conversational AI with IEAI Director Prof. Christoph Lütge and our audience.
In her presentation, Dr. Boureau explored the importance of privacy and informed consent of the users engaging with conversational AI, as these tools are increasingly becoming a part of everyday life. Her analysis emphasized that identifying measures that are predictive of good outcomes, such as engagement, is essential in building systems that can actually achieve helpful conversations that promote wellness.
Moreover, she discussed the importance of choice autonomy. Users often prefer when AI tools are personalized to their features. Nevertheless, the level of user autonomy and choice raises fundamental ethical questions. Determining guidelines concerning the extent to which users should be able to shape these systems to their preference is a fundamental step that needs to be taken. For instance, is it permissible for an AI tool to manipulate users in order to get them to do something that would be beneficial for them (such as exercising)? These questions take front stage when dealing with building ethical AI systems that have well-being of humans as their ultimate goal.
We also had the pleasure of speaking with Dr. Boureau in an IEAI Q&A: Reflections on AI. We thanks her again for her time and the interesting discussion and hope to have her back to the IEAI soon.