A few days ago, my wife was cleaning up some old documents and books when she found a paper I wrote in 1990.
I remembered writing it. I had to read it again to remember what I had actually said.
At the time, I was living on the North Shore of Oahu, finishing my undergraduate degree at Brigham Young University–Hawaii. I did not yet know what my career would become. Apple, Verano/Industrial Defender, Cisco, Oblix, Ping Identity, Oracle, and decades of enterprise identity and security work were still ahead of me. So was the work I now call Tamed Autonomy.
The paper was titled Designing the Human Mind.
Reading it thirty-six years later, I found some questions I still recognize, some assumptions I no longer share, and one conclusion I would have a hard time defending today.
That combination makes it more interesting to me than a paper that happened to predict something correctly. I get to examine how I thought before I knew where that thinking would take me.
And I have to wonder which parts of my current writing will produce the same reaction thirty-six years from now.
I Thought I Knew What We Were Building
The abstract begins with a confident statement:
“The ultimate goal of artificial intelligence is the ability to duplicate the human mind.”
I had made a substantial decision before the discussion had even started. I had defined the destination.
Once I assumed that AI’s goal was to duplicate a human mind, the questions followed naturally. How does the brain work? Could a computer reproduce its processes? Would it need emotions, consciousness, or values? How closely would the machine have to resemble us before we could call the effort successful?
Those remain interesting questions. But they are not the only questions that matter.
Today, I use AI to help me explore ideas, examine arguments, and work through problems. Much of its usefulness does not depend on determining whether it thinks the way I do. Yet the old framing still has a pull. When a system does something impressive, it is easy to ask how human it has become.
My paper contained a challenge to that framing, even though I did not fully follow it through.
Drawing on an argument from the Churchlands, I discussed the idea that an artificial flying machine does not need to reproduce everything a bird does. An airplane can fly without laying eggs.
That is an almost comically obvious observation—until you apply it to intelligence.
Which features of a human mind are essential to the capability we want? Which are features of the particular biological system that happens to possess it? Could insisting on a complete replica make us misunderstand a machine that succeeds in a different way?
I can see that tension in my younger self’s writing. I declared duplication of the human mind to be the goal, then included an argument explaining why complete duplication might be unnecessary.
The lesson I take from that today is to examine the destination before measuring progress toward it. If we define success too narrowly, we may miss something valuable. If we define it too loosely, we may claim to have achieved something we have not.
The Person Inside the Room
Another part of the paper brought me back to a question that has never become comfortable.
Imagine that you are sitting inside a room. You do not understand Chinese. You have a collection of Chinese symbols and a rulebook written in English.
Someone outside passes Chinese writing into the room. You consult the rulebook, match the shapes, and return the symbols it tells you to return.
Suppose the instructions are good enough that your replies make sense to the people outside. From their perspective, they are having a conversation with someone who understands Chinese.
Inside the room, you still cannot read a word of it.
This is the central setup of John Searle’s Chinese Room thought experiment. It asks whether manipulating symbols according to rules is sufficient for understanding their meaning.
There are objections to the argument, including the possibility that understanding belongs to the whole system rather than to the person following the instructions. The thought experiment does not settle the question simply by being memorable.
But it makes the uncertainty tangible.
My paper asked whether a computer that passed a test of intelligent behavior understood the questions it answered. In places, I was tempted to infer understanding from a correct response. Elsewhere, I questioned that inference.
I recognize both impulses today.
When an AI helps me develop an idea, I respond to the usefulness of what it says. If it identifies a weakness I had missed, that weakness does not disappear because I am uncertain about the system’s understanding.
But usefulness can also make the larger inference feel natural. A system follows the conversation, responds to a subtle distinction, and produces language that sounds thoughtful. I can begin treating those experiences as answers to questions they may not resolve.
The practical lesson is to be precise about what success demonstrates.
A good answer establishes that the system produced a good answer in that situation. Depending on the evidence, it may establish considerably more. But the distance between the observation and the conclusion still matters.
That was worth asking in 1990. Greater capability has made the question more consequential.
And Then I Called a Computer Alive
The most uncomfortable passage appears later in the paper.
I cited a report about a computer model of part of the brain that exhibited unexpected brain-like activity. Then I wrote:
“According to this line of thinking, Traub’s computer was alive!”
I can follow how I reached that sentence. A model showed a feature associated with a living brain. I was exploring whether reproducing the brain’s processes might reproduce something of the mind.
But “alive” was a much larger claim than the observation supported.
Would I accept that reasoning if someone presented it to me today?
I would want to know what was modeled, what was measured, and what the resemblance actually demonstrated. I would distinguish a property of the model from a claim about the condition of the entire system.
My younger self moved across that distance too quickly.
This is probably the most useful mistake in the paper because I do not think the temptation has disappeared. The thing that impresses us changes; the inference can remain familiar.
A machine produces convincing language, so we infer understanding. It describes concern, so we infer that it cares. It succeeds at a difficult task, so we become more willing to trust it with a different one.
Sometimes additional evidence supports that trust. Sometimes we have allowed one impressive result to answer several questions at once.
I cannot read that old sentence and comfortably assign this problem to other people. I have a written example of myself doing it.
It gives me a reason to look more carefully at the claims I make now, especially the ones I most want to be true.
What the Old Question Has Become
My career eventually took me into systems where identity, permission, and trust have practical consequences.
That work changed the questions I ask about intelligent machines.
In 1990, I was asking whether correct behavior proved understanding.
In my Tamed Autonomy work today, I ask whether authorized behavior proves legitimate intent.
Those questions concern different things. Understanding is not the same as intent, and an operational judgment about an agent’s purpose does not require solving the philosophy of mind.
But I recognize a shared habit of inference.
A visible result satisfies a test. We then treat that result as evidence of something beyond the test.
An action can pass a permission check while leaving its purpose insufficiently examined. A human can approve a proposal without having enough context to assess the outcome. Several individually permitted actions can combine into something nobody intended to authorize.
The checks matter. So does knowing what they establish.
This becomes personal when I am the human in the process. I can be busy, tired, or persuaded by an explanation that makes a decision sound straightforward. If the system proposing the action also supplies the account on which I base my approval, I need to think about how that account was constructed.
Did I understand the decision, or did I find the explanation convincing?
The Chinese Room asks us to consider what the observer can infer from the exchange. My current work brings that concern into decisions where we cannot simply wait for philosophical certainty.
We have to choose what a system may do, what evidence we need, and when uncertainty should cause us to stop.
What We Do Not Know Yet
There is another reason I am reluctant to turn the old paper into a story about being right.
I did not imagine the everyday experience of working with AI that I have now.
I was considering whether we could duplicate a mind. I was not anticipating how often I would use a machine as a partner in examining my own thinking—or how that relationship might raise questions separate from the machine’s intelligence.
When AI helps me reason, what am I learning to do better? What might I gradually stop practicing?
A system that knows my preferences could help me notice a blind spot. It could also become very good at presenting a recommendation in terms I am inclined to accept. Greater familiarity does not, by itself, tell me which of those things is happening.
And when work passes through several agents, tools, and people, responsibility can become difficult to locate. Everyone may have completed an assigned part while no one adequately considered the whole.
These are questions I would want a future reader to find in this article. I do not have complete answers to them.
Some unknowns may yield to better experiments and better engineering. Others involve deciding what we value: which judgments we want to retain, what kinds of dependence we accept, and who should bear responsibility when a delegated decision causes harm.
We should be careful about treating all of those as problems that another generation of technology will solve for us.
My paper included a prediction, attributed to Marvin Minsky, of rapidly advancing machine intelligence. I noted that its expected timetable had not materialized. That history makes me cautious about confident schedules, including my own.
The next thirty-six years may bring changes much faster than the last thirty-six. If AI helps accelerate the work of building AI, we may have less time between discovering a capability and deciding how to live with it. That possibility adds urgency, but it does not answer the underlying questions.
Faster progress could leave us with more powerful systems and many of the same uncertainties.
Reading This Again Thirty-Six Years from Now
Near the end of my paper, I wrote:
“We have moved from a definite possibility to a definite maybe.”
Then I suggested that only time would provide the answer.
I am less certain of that now.
Time has brought developments I did not foresee. It has given us new evidence, new capabilities, and new reasons to revisit old arguments. It has not supplied a single answer to everything I placed under the heading of a human mind.
Some questions may endure because they are difficult. Others may endure because we have not agreed on what would count as an answer. Still others may turn out to have been framed badly.
I would like to think that noticing those differences makes me more careful today. The paper is a reminder that I should leave room for a future version of myself to disagree.
If I read this article thirty-six years from now, I expect some of it to look incomplete. I hope I will be able to identify assumptions I should have questioned and possibilities I failed to imagine.
What I would want to find, though, is an honest attempt to distinguish what the technology could do from what I believed its performance meant—and to make responsible decisions in the space between them.
My wife found an old undergraduate paper among some documents and books. Reading it again gave me something I could not get from remembering that I had written it: evidence of where my curiosity was useful, where my confidence outran my reasoning, and where I had left a question open.
I am glad I still have it.
It makes me wonder what I should leave open in this one.
Read the original paper: Designing the Human Mind (1990), scanned PDF. The student identification number has been redacted. The paper was originally submitted without my name so reviewers could assess it without knowing the author.
Leave a Reply