29 May 2026, by Lauscher/Red.

Photo: Sarah Buth

The University of Hamburg is the scientific home to more than 6,200 researchers. Every 2 weeks, we offer a glimpse into their work as part of the “Research and Understanding” series in the Hamburger Abendblatt. In the latest issue, Prof. Dr. Anne Lauscher explains the implications of the fact that AI training data also reflects our stereotypes.

ChatGPT, DeepL, Dall-E: AI applications generate text, images, or videos—and in doing so, they always reflect our social reality. Even if users aren’t aware of it, these systems are trained on vast amounts of data that reflect not only our values but also our stereotypes. As a result, while ChatGPT and similar tools make our daily lives easier, they can also have problematic and unethical consequences when they reflect racist, anti-Semitic, or anti-LGBTQ+ attitudes.

Since AI is developing at a rapid pace and now profoundly influences nearly every aspect of our lives, we must counteract this. I want to contribute to designing and deploying AI in a way that ensures it is developed responsibly and yields positive societal benefits.

Making Discrimination by AI Measurable

One area of concentration in my research is to identify and make discrimination by AI measurable, because the first step is to understand who is disadvantaged by these applications and how. We primarily use methods from empirical AI research, which means we conduct large-scale experiments to investigate specific effects and quantify them. For example, we enter voice commands we’ve developed into the AI and compare the results we get with different input languages. In addition, we rely on so-called mixed-methods approaches and combine experiments with surveys or interviews to better understand social dimensions.

It has now been widely demonstrated that there are significant differences in the reliability of AI models depending on the input language. When I ask a question in English, there is significantly less incorrect output than, for example, with inputs in Korean. In one of our current projects, we also observe that discrimination occurs even within a single language: Large language models like ChatGPT attribute significantly lower competence to spokespersons with a dialect such as Low German when asked about potential career paths, and—compared to spokespersons of Standard German—more frequently associate them with terms like “uneducated” or “narrow-minded.” These prejudices have long prevailed in society and are now being revived by chatbots.

Focus on Language-Based Applications

In addition to our work on text-based AI applications, we are currently finalizing a study in which we demonstrate, for the first time, a connection between voice pitch and discrimination by audio-language models. These types of AI models can be operated directly via speech—and our results suggest that the discrimination is, in some cases, even more pronounced than in text-based systems. People with higher, supposedly female-sounding voices receive different results than those with lower-pitched voices, and the AI attributes stereotypical traits such as “gentle” or “community-oriented” to them.

These findings are particularly relevant because we expect audio-language models to become more widely accessible in the future than purely text-based systems. This means that users will not need to be able to write. Until now, AI applications have been only partially accessible to people with limited reading and writing skills, and the fact that this is changing is very positive. However, those affected often also have lower digital literacy and are thus more exposed to these biases.

Part of our research therefore also involves developing training methods for AI systems to reduce discrimination or prevent it in new applications. Especially in sensitive areas of application such as human resources, these systems must operate fairly. Our work demonstrates that this is possible: With the right data foundation, these systems can be designed—but only if bias is robustly measured, affected groups and contexts are explicitly considered, they are incorporated into the research, and countermeasures are consistently evaluated.

(This content has been translated automatically.)