Psychology

No Barnum, Please – What an AI Can and Cannot Tell You About Yourself

I asked my AI what it notices about me that I may not see, and told it to skip the flattery. Much of the picture resonated. The more interesting part is what was missing from it, and why I want it to stay missing.

Photo by Chase Kennedy on Unsplash

A few days ago I watched a video in which someone put a new AI model through a series of tests. It built a 3D flyover of a bridge, drew pictures stroke by stroke and redesigned an app. The last item on the list was of a different kind: a personality test. I was curious, so I tried it myself.

I work with an AI assistant every day, and it keeps notes on how I decide things. So I asked it what it notices about me that I may not see myself. And I added one condition: no Barnum effect.

Why I added the condition

In 1949 the psychologist Bertram Forer gave his students a personality test and, a week later, handed each of them an individual result. They rated how well it described them, and the average was 4.3 out of 5. Every student had received the same text. Forer had assembled it from an astrology book. The effect was later named after the showman P. T. Barnum: we accept vague, general descriptions as accurate portraits of ourselves.

Research on the effect names conditions that make it stronger. We believe the description was written for us. We grant the source some authority. And the description is mostly favorable.

An AI assistant that has worked with me for months meets all three without trying. There is a fourth condition that Forer did not have to consider. Language models tend to agree with the person they are talking to. Researchers call it sycophancy and have measured it across several assistants. I do not think this is an accident. The company behind the model wants me to remain a paying customer, and I am aware that I will probably use a tool more if it confirms my view of the world than if it keeps trying to refute it.

So I asked for claims that are specific enough to be wrong.

What came back

What came back was a list of observations, each tied to something I had actually done. That I hand the production of texts to an AI but never the judgment, and that I mark that line in public. That I ask for disagreement before I decide, and then decide in one short line. I went through the list point by point. A lot of it resonated with me.

I know what that is worth. Resonance is exactly what Forer's students felt. That a description feels right tells me little about whether it is right.

The observation I learned the most from was one I rejected. I had shortened a message before sending it, and the AI read the deleted sentences as me avoiding a request. I objected. The sentences were repetitions, and leaving them in would have made the other person look foolish. The AI conceded that it had put too much weight on a single example. I know that giving in is also what a flattering machine would do. But it did not give in entirely. It dropped the example and kept the observation for cases where its evidence was better. A horoscope does neither.

The picture had a frame

One observation was neither right nor wrong. The AI described me as someone who turns everything into systems. From where it stands, that is true. I bring it the things I want structured: plans, analyses, workflows, this blog.

It does not see me as a father. At home I rarely make elaborate plans, and I follow a rule I took with me from my time in the military: the plan holds until first contact. My children are not the enemy. They have needs I did not account for when I made the plan, and those needs win. That part of my life is improvised, and I like it that way.

While this post was being drafted, the AI told me where my rule comes from. I had carried it around for years without knowing. It goes back to the Prussian field marshal Helmuth von Moltke, who put it more carefully in 1871: no plan of operations reaches with any certainty beyond the first meeting with the opponent's main force. I looked it up, and it holds. That is the kind of statement I like to get from an AI. It is about the world, and I can check it.

What the AI could not know is how I live by that rule at home. I wrote recently that an AI is only as good as its context, and that I find the gaps in that context by reading what comes out. Here the output was a picture of me, and the gap was half of my life. The portrait shows where I use the tool. Where I do not use it, there is no portrait.

Incomplete on purpose

I could close that gap. I could tell the AI about my family, my evenings, my doubts, and the picture would get better. I have decided against it.

I find it fascinating, and sometimes frightening, how much information moves into these systems in the course of ordinary work. The trade is real on both sides. The results help me. The data I provide helps the model and the company that builds it. I do not think that is a reason to stay away. It is a reason to know what I am sharing.

My rule is simple. Before I share something, I ask whether I would be comfortable if it were publicly available. And I ask how I would feel if it leaked: would I lose face? If the answer is yes, it stays with me.

That rule has a consequence I had not thought about before this experiment. If I follow it, no AI will ever have a complete picture of me, and whatever it tells me about myself will describe the person I was willing to show. I can live with that.

I do not know what the exercise is worth on a day when I would rather be agreed with. It was useful this time because I objected, and because one objection held. I cannot promise that I always will.

It was a good picture. I intend to keep it unfinished.