Your digital psychologist is recording everything: The dangerous illusion of ChatGPT

Since the breakthrough of generative artificial intelligence into our lives, various interfaces have often been presented as "true friends" or soothing consulting tools. However, behind the intimate feeling that chatbots project hides a grim reality of a lack of privacy. A new and extensive survey published by the research organization DuckDuckGo reveals a worrying picture: about a third of the public share personal information and deep secrets in chatbots that they did not dare to share even with family members, friends, or doctors. Among those who define themselves as "AI enthusiasts," this rate jumps to 56%.

Data shows that many users tend to consult with AI on particularly sensitive issues — from mental and physical health problems, through dilemmas in married life and parenting, to personal financial advice. However, the perception that the conversation remains "between the user and the computer" is a complete illusion.

From the keyboard straight to a court order

While in Google searches users are accustomed to receiving targeted advertisements, the analysis of conversations in AI goes much further. Information perceived as "personal" structurally becomes a corporate record. Companies like OpenAI, Google, and Anthropic are required by law to provide conversation transcripts to law enforcement and police in the event of a court order. ChatGPT transcripts have already been used as key evidence in murder trials and arson investigation successes.

Meanwhile, large language models (LLMs) are capable of "remembering" data. In various cases, answers that the AI produced for other users included sensitive and identifying information taken from conversations of previous users. Also, employers who use organizational accounts can fully monitor correspondence. Giant companies use "Emotion AI" systems to scan organizational chats (such as Slack or Teams) and identify "negativity" or the level of employee loyalty.

"If you are going to talk to ChatGPT about your most sensitive things and then there is a lawsuit, we may be required to provide this information," explained OpenAI CEO Sam Altman.

The model is hungry for data

One of the central problems raised in the survey is public ignorance regarding data usage. More than 50% of respondents did not know (or were not sure) that their conversations are used as raw material for training the next generation of artificial intelligence models. A study by Stanford University showed that technology giants — including Amazon, Google, Microsoft, Meta, and OpenAI — make active use of users' private conversations for training purposes.

Worse, about 40% of respondents indicated that they have no trust in technology companies to protect their information, and 50% do not trust government entities on the matter.

4 steps to protect privacy

To continue using AI tools without risking personal or professional privacy, it is recommended to adopt the following rules:

1. Treat the chat as a public environment: enter only data that you would feel comfortable seeing published on a bulletin board. Do not reveal identifying details, ID numbers, sensitive medical information, or trade secrets.

2. Opt-out of data sharing: go to the privacy settings in applications like ChatGPT, Claude, or Gemini and turn off the option for recording conversation history and using it for training purposes.

3. Export and check the information stored about you: perform a data export from time to time (under Export Data) to see what personal information has accumulated about you on the companies' servers.

4. Switch to local models: for particularly sensitive discussions, it is recommended to use programs that run locally on the personal computer (such as Jan.ai, Ollama, or PrivateGPT) that do not transfer data to cloud servers.