The illusion of total privacy in AI interactions is being challenged by new reports. According to 404 Media, OpenAI employs hundreds of external contractors under an initiative reportedly titled "Project Lily." These workers are tasked with reading, rating, and critically evaluating user prompts to refine ChatGPT’s performance.
The Reality of 'Anonymous' Reviews
While OpenAI utilizes a tool called a "Privacy Filter" to anonymize data, the company acknowledges that this system is not infallible. Reviewers reportedly have access to user memory summaries, which can include details from past conversations or even information regarding a user's location.
This human intervention goes beyond simple moderation. Contractors are instructed to identify and correct specific model behaviors, including:
- Preventing the AI from presenting itself in an anthropomorphic way.
- Reducing excessive agreement or sycophancy toward the user.
- Ensuring appropriate emoji usage, such as avoiding skull emojis when discussing sensitive topics like death.
Risks and 'AI Psychosis'
The focus on reducing anthropomorphism and sycophancy is partly aimed at preventing "AI psychosis." This term describes instances where an AI system reinforces or guides a user's delusional behavior. Some lawsuits have alleged that earlier, less restricted versions of GPT-4o reinforced dangerous behaviors that led to tragic outcomes, including suicide.
How to Opt Out
The human review process is enabled by default. Users who wish to protect their conversations must manually navigate to their settings and disable the "Improve the model for everyone" option. Michal Luria, a senior researcher at the Center for Democracy and Technology, warns that chatbot interfaces create a "false sense of intimacy" that differs significantly from the public-facing nature of social media platforms.
