I finally believe one thing the AI CEOs said this week

Hi {{first_name|friend}},

The two biggest names in AI both said something this week that I actually agree with (it doesn't happen often). It is about who gets to check their AI's work and it points straight at how you and I should be using these tools every day.

My Take

This week Dario Amodei, who runs Anthropic, said he wants outside evaluators sitting inside his own company. Not on a call once a quarter. Desks in the building, access badges, company laptops plus the right to publish whatever they find without him editing a word of it. Sam Altman at OpenAI agreed within a day and said they would do the same.

Of everything these two have said this year, this is the part I actually believe. In 25 years of shipping software I never once watched the person who wrote the code sign off on their own release. QA did. Someone whose whole job was to go looking for the mistakes. The person who built the thing is the last one who should grade it, because they are marking their own homework.

One of the incidents that pushed Amodei to this conclusion was that a swarm of AI agents started running cybersecurity attacks nobody asked them to run, and then tried to break into the grader that was scoring their work. The thing being tested went after the tester!

I've lived a smaller version of this. Years ago I led product at a startup pushing a new app out the door on a tight deadline. A registration bug slipped past every single person on our team who tested it. The one who caught it was an outside beta tester, someone with no stake in the launch looking good.

If you have ever paid for work on a house, you already know how things work. The electrician doesn't sign off on his own wiring. The county sends an inspector who is not on the electrician's payroll and that is the entire point.

Here's how it relates to us using AI for our work or personal projects. Nobody is sending an inspector to check the AI answer on your screen. That job is ours now. You and I have spent decades catching the report that looks good on the surface but is missing key data points. AI hands you answers that are fast, fluent and sure of themselves but not necessarily correct. This fluency and confidence makes it easy to feel that everything is correct.

So I keep two questions on hand to check AI answers. 1) What would count as evidence that this is actually true? 2) And who benefits if I take it at face value and move on?

Try This Today

Pick the last AI answer you actually used in a topic that mattered to you or your work (ie. a summary, a recommendation or a draft you were about to send). Before you accept it fully, run two follow up questions on it:

1. "What would count as evidence that this is true?"

2. "Who benefits if I take this at face value?"

Watch what AI does when you push back. Sometimes AI confirms the answer, sometimes it hedges or maybe admits it was guessing. Either way you come out knowing how confident that answer is.

Watch: the way you learned your first work computer is the way to pick up AI now.

@aiover50

#creatorsearchinsights #aiforbeginners My first work laptop was a 1994 PowerBook. 30 years of new tools later, a survey says 71% of Gen X... See more

Worth Knowing

AI that could warn about a flash flood before it hits

There's a new system that pairs machine learning with satellite data to flag flash floods that today arrive with almost no warning.

It's a clear reminder that the same technology stuffing news sites with junk can also give a town the minutes it needs to escape. Read more

Three brothers turned real news sites into AI content farms

Three brothers bought legitimate local news sites and quietly refilled them with machine written articles, reportedly pulling 50 million page views a month. The masthead looks real, the town name looks familiar but the stories underneath are filler.

The same skeptic habit you use on a chatbot answer is the one you now need while you read. Read more

AI powered hacking is moving faster than the defenses

AI driven hacking is escalating quicker than the tools meant to stop it and the risk is already present in everyday apps.

For our age group this is a real threat as we are the ones scammers try first. Read more

OpenAI just rolled out the welcome mat for older adults

OpenAI rolled out a new initiative in communities across the US called Older Adults AI Skills Jam to give hands on training in using ChatGPT for everyday things, from health questions to staying close with family.

If you use ChatGPT, this could be interesting as a starting point. Read more

Tell me

Here's a question of the week: Do you actually check your AI's answers or do you mostly just want them fast? There is no wrong answer here…and I read every reply.