Monday, a developer using celebrities EU-power Code editor Cursor I saw something strange: switching between machines, immediately entered the general workflow for programmers using many devices. When the user contacted the cursor support, an agent named SAM, said there is an expected behavior of behavior under a new policy. But such a policy was not available and Sam was a bot. AI model increased policies by sparking a wave of complaints and abolished threats Hacker News and Reddit.
This records the final example of AI confaction (also called « hallucinations ») causing potential work damage. Confabulational is the « creative space filling », which is acceptable to the AI models, but invented false information. Instead of acknowledging the uncertainty, AI models are prioritizing, which means production of production from scratch, even appropriate, confident answers.
The results of these systems without human control for companies, which placed these systems in roles, may cost immediately and expenses: Discharged confidence and cursor work, potential canceled.
How it turned on
The incident, Reddit user, began when branding the shaking prominent Variable between a desktop, laptop and a distant dev box, cursor sessions were unexpectedly terminated.
« Access to the cursor in a car is immediately invalid in another car, » Brokentoastooven wrote in a message subsequently deleted by r / cursor moderators. « This is an important UX regression. »
Confused and nervous, the user wrote an email to the cursor support and quickly received an answer from SAM: « Read a device with a device per unit, » read the email response. The response was determined and officially called and the user did not suspect SAM’s lack of human.
After the initial Reddit post, users took office as a formal approval of a real policy change that violates the habits that are important for the daily rules of many programmers. « Many devices have a table share for devs of work, » he wrote a user.
Shortly afterwards, several users openly declared subscription places in Reddit, determined their non-existent policy as their reason. « I canceled the submumine, » wrote the original Reddit poster, added that jobs are now « completely cleansed ». Others joined: « Yep, I also cancel, this is asyn. » Soon, moderators locked the Reddit rope and pulled the original article.
« Hey! We have no such policy » write A cursor representative in response to Reddit three hours later. « Of course, you are free to use the cursor in more than one machine. Unfortunately, this is a wrong answer from a front-in a front of AI support bot. »
AI confacksies as a business risk
Cursor Debace remembers a similar episode As of February 2024, the weather ordered to honor Canada to honor the return policy invented by its Chatbot. In this case, Jake Moffatta was contacted by the AIR’s support of the AI agent after the air of the air canada, and he could order a regularly valuable flight and can apply for backward rates. When Air Canada rejects the return request, the company said, « Chatbot defended that he was a separate legal entity in charge of his actions. The Canadian Court ruled that this defense and companies were responsible for information provided by AI tools.
Instead of arguing the responsibility as the weather in Canada, he acknowledged the cursor error and took steps to make adjustments. Cursor Cofounder Michael Truell later Apologized to the hacker news The problem of confusion about non-existent policies is the problem, resulting from a research change, is designed to improve session security for some users.
« Any AI answers used for email support are now so clearly labeled, » he said. « We use AI-ancillary answers as the first filter for email support. »
Again, the incident raised the questions about the announcement among users, because he believed that many people were human people. « LLM looks like people (you named Sam name!) And as not labeled, not to be deceptive, » a user He wrote in hacker news.
When corrected the technical error of the cursor, the episode shows the risks of placing AI models in roles facing customers without relevant guarantees and transparency. AI is an alienable policy for a company that sells productivity to developers, its own AI support system.
« There is a certain irony that people really try to say that the hallucinations are no longer a big problem. » A user He wrote in hacker news« And then a company that will benefit from the narrative hurts directly. »
This story originated first ARS TECHNICA.
Leave a Reply