The so-called “hallucinations” of Artificial Intelligence — that is, incorrect, fabricated, or misleading statements — continue to pose a major challenge. Completely eliminating these distortions has proven difficult, and the debate remains ongoing. To address them, increasingly sophisticated training and analysis techniques have been developed, often with evocative names like Deep Search or Deep Reasoning. While these solutions aim to enhance the reliability of AI systems, they also risk fueling another phenomenon: automation bias — the tendency to blindly trust AI outputs simply because they appear more “intelligent.”
The illusion of a Cognitive Process
Solutions like Deep Search and Deep Reasoning have been developed with the goal of reducing improvisation in language models, guiding them toward more reasoned and evidence-based responses. But what do these terms actually mean? And, more importantly, how much have they truly contributed to improving AI reliability?
“Deep Search” refers to the ability of some advanced AI models to conduct autonomous real-time searches, accessing the web or updated databases to retrieve relevant and verifiable information. These systems are expected to go beyond simple data lookup, analyzing sources to integrate them into responses and providing citations.
“Deep Reasoning” on the other hand, refers to a model’s ability to process information through a structured procedure that simulates logical reasoning, broken down into sequential steps. Unlike traditional generative models, which produce responses primarily based on statistical patterns, models equipped with Deep Reasoning break down complex problems. This step-by-step decomposition is intended to allow the model to analyze information more critically, identify flaws in its own reasoning, and self-correct.
In principle, the synergy between these two capabilities represents a significant step forward in AI. Models that can not only access updated and verifiable information but also process it through a structured reasoning method could, in theory, move away from relying solely on word combination probabilities.
In practice, however, these new solutions can still produce errors, hallucinations haven’t disappeared. In fact, according to a study cited by The Week, some recent models exhibit error rates of 33% and 48% in controlled scenarios (it’s worth noting that investigations into the causes are still ongoing, and we cannot yet draw a direct link between these new methods and an increase in hallucinations).
In light of these numbers, one thing is clear: hallucinations in AI models remain a current and unresolved issue. Even advanced techniques like Deep Search and Deep Reasoning, designed to make responses more accurate and trustworthy, do not consistently deliver reliable results. In some cases, they may even introduce new types of errors. As of now, there is no truly effective or stable solution to significantly reduce these AI slip-ups.
The Legal risks of implementing AI
As artificial intelligence becomes increasingly integrated into professional and business environments, the phenomenon of so-called “hallucinations” represents a potential risk factor that should not be overlooked. Tools based on Deep Search and Deep Reasoning, although designed to enhance accuracy, can occasionally produce fabricated content with such confidence that they mislead users who fail to exercise due caution. Approaching these tools with a critical mindset, avoiding automation bias and questioning their outputs, is essential.
For a PMI that uses AI to prepare reports, market analyses or technical documentation, relying on inaccurate data can lead to reputational damage and legal liability, particularly in terms of contractual responsibilities. Consider, for example, the unintentional inclusion of unfounded information in a financial statement, an investor presentation, or an official communication. Mistakes of this nature could result in financial losses or legal disputes for negligence.
The core issue with hallucinations lies in their deceptive nature. Language models often present them with persuasive language and apparent authority, making them especially difficult to detect without careful verification. As we’ve seen, these misleading outputs can distort facts or present contradictory information, potentially affecting user trust and decision-making processes. For this reason, human oversight remains indispensable.
However, it’s also important not to fall into the opposite extreme: complete distrust of AI. AI is a valuable tool capable of streamlining and accelerating many processes, provided it is used with awareness. Especially in professional settings, it is crucial to remember that even the most advanced models require attentive human supervision. AI performs at its best when paired with the critical judgment of its users.
Article in collaboration with AW LEGAL
AW LEGAL is a law firm specializing in Intellectual Property, Privacy, and Legal Tech.
