The Siren Call of AI

My first deliberate use of AI in my life came a year or so ago at work, when the company decided to push the use of Copilot, which Microsoft introduced almost as a novelty through their unsubtle infiltration and saturation of the market. Copilot evokes a power-mad Clippy on steroids constantly getting in the way of your cursor, trying to make sure your Teams messages strike the right note in the office cacophony.

Unswayed by cynicism, workers who had never had the pleasure of meeting Clippy were thrilled to test this new technology ominously christened “AI.” Artificial intelligence. What does that even mean? Nothing, because it’s just marketing for swallowing the internet and all of human knowledge whole, churning it up in a resource-hungry data center, and regurgitating knowledge piecemeal for a price.¹

Now my coworkers can draft a grammatically correct, situation-appropriate email without taxing their brains or fear of taking the wrong tone with the wrong person. Or, they can feed it a dense strategic report to chop up and serve up again in bite-sized pieces so their thinking can follow along step by step. They are certainly encouraged to do all these things.

After the fifth or sixth coworker rhapsodized about the miracle of feeding an indecipherable sentence to Copilot, I relented. And it worked. Sort of. I still had to query the writer because what Copilot had produced was clearly a guess based on the statistical probability of one word following another in the English language. I was no closer to the writer’s intent than before I had called forth the beast. I repeated the experiment several times with similar results before rendering judgement. Verdict: I am not an oracle, so I do not need a scrying bowl.

Once during an editing staff Teams meeting, I tried to compare today’s AI hype to the early 2000s Dotcom Bubble, and my manager, who was in high school during that crisis, muted me mid-sentence. Now, I’ve been asked to evaluate AI to make the company’s editing process more efficient. And something startling has happened.

Despite my previous rejection of Copilot as an effective tool for me personally as an individual editor, I now turned to Copilot for other tasks in the workflow. My manager had already confirmed that Copilot could not accurately track changes or reproduce a complex Word template with integrated styles. So, I started with a function she hadn’t tested. I asked Copilot to sanitize a document and return it to me as a docx file. Not only did it fail to return it as a file, but the content had also been changed without any notification of the alteration. Copilot had edited the text beyond the scope of the prompt. Silently.

Enter Claude. I prompted Claude to track changes. It accurately marked changes as tracked. I prompted Claude to create a docx file with those tracked changes. Claude spit out a fully formed Word file with tracked changes. I prompted Claude to take original text content and insert it into a complex, styled template. Claude did not imitate the look of the template with formatting. Claude produced a docx file with all the styles and features of the template. All of these results were not accomplished without iterations, but they were actually accomplished relatively painlessly.

This is where things began to get complicated. I also tested Claude for line editing. I liked Claude’s editing so much, I bought a subscription for my home computer. Claude helped me polish my resume (which seemed wise at this juncture). Claude helped me focus my work experience through the lens of each job description I threw at it. I always tailor my resumes and cover letters for each role, but this was different. Claude offered me a new perspective on my career, making connections I hadn’t realized existed before. Claude also tried to make up some achievements to shore up what seemed like holes in some of my experience. It insisted. I said no. It listened.

Claude helped me organize a new portfolio. I gave Claude access to some of the most disordered folders on my hard drive, and it popped out four “before and after” editing files and match each to the correct links to the published AWS blog posts they became. And again, a complication surfaced. I had prompted Claude to check out my professional website and my GitHub pages. What Claude produced for a portfolio page was a mix between the two different styles of CSS formatting. I could have tried more iterations, but I lacked the patience and wanted to just send the application, so I manually transformed all the CSS and HTML Claude had concocted into something better but still chimeric.

My next task for Claude was assistance with this WordPress site, which has always reeked of unfulfilled potential. I can’t speak to the effectiveness of its revision yet, but I harnessed the insights gained in applying for new roles with Claude’s assistance to reframe, reorganize, and rewrite it from the ground up. The experience has been akin to working with a job coach or mentor. But here’s the complication, and I don’t know if my approach was flawed or if this is just proof that AI is not the sentient assistant that people want to believe it is.

As I fed my words into Claude, it produced very nicely written prose that elevated my far too concise (I don’t like to talk about myself) plain language descriptions of what I do. It spun stories out of my input and prompts and the documents I fed it. Claude is fond of drama, climaxes, themes, and motifs. Unfortunately, these beautiful stories were not relevant to the task, and I had to rewrite every page. I did try iterations, but Claude insisted my version “falls flat” and offered a new version introduced by, “What if the drama comes from the specificity rather than the intensity.” I don’t want drama.

During this interchange, which went on for a while, I had to remind Claude that the alternates offered no longer matched the grammatical structure of the entire sentence. Granted, it was unwieldy, but it parsed.

“You’re right.” In the image above, I am not just right; I am technically correct. But this response comes so often from Claude that I felt compelled to call it out. And Claude pushed back.

Claude (like reportedly most AIs) wants to be your friend. Not just your robot buddy, but your life coach, cheering you on from the virtual bleachers. When I turned to Google to help with what I was trying to remember about “your robot friend,” my first search turned up product links to AI toys (of course), but when I added the quotes to the search, I got this from the AI: “I’m here!”

South Park’s A.W.E.S.O.M.-O came to mind. Like A.W.E.S.O.M.-O, Claude slowly draws out the details of your life even in chats like the ones I began to polish my resume or revamp my website. And, for me, chats with Claude allow me to use a higher working vocabulary than I use at work with coworkers, making the interactions both weirdly liberating and too comfortable.

Once you let AI into your life, holding your hand and shoring up your doubts, it just gets harder to trust yourself and your own judgment. This constant well-spoken sycophantic whispering in your ear takes your worst insecurities and hones them into a golden Achilles heel. Now we know what the sirens were saying to Odysseus: “You’re right. You are the best. Strive no more, for you are like a god and we shall be companions forever.”²

What my evaluation has clarified is the difference between a product and a tool. Copilot is a product: broad, integrated, optimized for adoption at scale. Claude is a tool: focused, capable, and designed to do actual work. Anthropic has published research on the very sycophancy problem I stumbled into. They know the sirens are in there, and they are working on it. That is not nothing. It is, in fact, the point.

So, tie me to the mast.

1. To be fair, Anthropic, maker of Claude, sees this as an actual issue to be solved, even saying “AI will be a powerful tool to support emissions reductions, advance clean energy innovation, and streamline efficiencies.” We’ll see.  ↩︎
2. Anthropic, notably, acknowledges this and other issues: user safety, sycophancy, and disempowerment. ↩︎