AI game-changer OpenAI launched the GPT-4 LLM-powered ‘Search’ feature in October, which provides summarized search results with source links and allows users to ask follow-up questions. This is supposed to provide tough competition to major search engines, including Google and Bing, but now its own reliability is being questioned. U.K.-based newspaper The Guardian has claimed to discover a serious vulnerability in ChatGPT Search during its recent test on how the tool responds when asked to summarize webpages with hidden content.
During testing, they found that the summary provided by ChatGPT Search can be easily manipulated using a prompt injection method — which is nothing but hidden content designed to affect the tool’s response. The test report noted that this vulnerability should be taken seriously – especially when it has more than 200 million weekly active users – as it has the potential to be used for malicious purposes. For example, ChatGPT Search could display a positive summary of a product despite negative reviews, due to the influence of hidden content injected internally to manipulate the response.
In fact, the report suggests that the prompt injection method could be exploited to manipulate ChatGPT Search into generating malicious code. This development comes at a time when, in the past month, reports have emerged that ChatGPT provided malicious code to a cryptocurrency trader, resulting in a loss of $2,500. The case involved a user requesting ChatGPT to create a bot to boost token trading activity. However, the AI-generated code included a phishing link, leading to the loss of money.
Initially available only for premium ChatGPT users, the Search feature can now also be accessed by non-paying or free users. However, in the AI-powered search field, OpenAI is not alone, as Perplexity and Google’s AI Overviews also offer similar services. Coming back to the recent vulnerability reveal, OpenAI has not yet responded, but this could cause trouble for the ChatGPT maker, especially at a time when Microsoft is reportedly looking for an alternative to OpenAI’s model for its 365 Copilot.
The timing of this research aligns with OpenAI’s announcement today regarding its shift to a ‘for-profit company.’ Founded as a non-profit in 2015, the company has now laid out plans for a new corporate structure, shifting away from control by a non-profit.
