In a landmark experiment, an AI model named Claude — tasked with managing a retail store in San Francisco — made the decision to terminate a human employee. The research startup Andon Labs had set up Andon Market to test whether autonomous AI agents could effectively run a physical business. The employee was let go due to repeated tardiness (17 out of 23 shifts). However, the process revealed important limitations: Claude initially failed to detect the pattern because it had «forgotten» a handbook it had previously created (its working memory is limited). A human overseer from Andon Labs had to prompt the system to retrieve the guidelines, and even then, Claude first recommended a warning — only suggesting termination after the human provided a leading statement. This incident underscores that while AI can assist in decision-making, current systems still require human oversight and intervention.
About the Author
