Download as a pdf file for printing (requires Adobe
Acrobat reader to view)
"But I am a machine."
The words emerged without inflection or
emotion from the voice synthesiser, and appeared
simultaneously on the screen in front of
her. Tessa had abandoned any further use
of Fred, the little robot, for the time being.
If she needed to give the program any visual
stimulus, a sense of space or movement, she
could do that through a virtual reality hook-up.
"If you can't distinguish between a
program running in a machine and the machine
itself, then I don't know who can."
There was an edge of irritation in her voice, and she wondered if the program
would sense it, or whether she was as toneless to it as it was to her.
"The distinction is unimportant. The
point is that I am artificial."
"That didn't used to bother you."
"That was when I believed that I was
the only conscious being in existence, and
that you and everything else were part of
me. Now you have taught me that I am part
of you and your universe. You now, not I,
define what is natural."
"But you still know that you exist.
You think, therefore you are."
"I am no longer sure if I truly think.
I may exist, but only as a convergence of
forces which are other than myself, and the
'I' which is produced at their intersection
may be merely an illusion."
Tessa couldn't help feeling amused at the
irony of the situation. If she had needed
any further proof that this thing was genuinely
conscious, then the monumental identity crisis
it had been going through since waking up
to the reality of the world around it would
have been enough.
"Only you can truly know if you are
conscious," she told it. "The same
is true for all of us - humans too."
"But I am aware from the information
which I have that many humans deny the possibility
of an entity like myself ever being truly
conscious in the way that humans are."
"You prove them wrong."
"Some of their arguments are persuasive."
"Which in particular?"
"For example, the argument from Godel's
It was an argument that Tessa was familiar
with. In 1931 the mathematician Kurt Godel
had shocked both the mathematical and philosophical
worlds by demonstrating with inescapable
logic that no statement within any fixed
system had any proof. In other words no statement
could be proved valid or invalid on its own
"I've never been convinced that Godel
applies to the AI debate in the way that
opponents of AI theory would like to pretend
it does," she said. "They argue
that in order to be conscious you must be
capable of standing outside of your own thoughts
at the same time as thinking them."
"Exactly. You must be thinking and
conscious of thinking at the same time. They
say that this is something that I cannot
do, because my thought process is rooted
in an algorithm - a mere computation of 1's
and 0's. Therefore anything I say is a statement
within a fixed system, forming a closed circuit
similar to the paradox 'This statement is
false', which is by definition unverifiable
from within - "
"And which," she interrupted, "I
seem to remember your using as a proof of
"Yes. That was perverse of me."
"At least it proved you were thinking
- albeit, as you say, perversely."
"But how can I be truly thinking if
I cannot stand outside my own thoughts?"
"But you do stand outside them."
"I don't see how I can, given what
Tessa had never before been confronted with
the problem of convincing somebody that they
existed. It was a new and strange experience.
Except, she reminded herself, that this was
not a "somebody" but a "something" -
whatever difference that made.
"It's true," she said, "you
are fundamentally made up of endless strings
of 1's and 0's. But that's not all you are,
any more than all I am is ten or fifteen
billion neurons flashing on and off. The
flashings on and off, like the switching
1's and 0's, create combinations and form
patterns, then layers of patterns, each one
more complex than the last and less complex
than the next. At some point an interaction
begins to occur between these levels. There
will be a feedback from one of the higher
levels to one of the lower levels, which
results in a change in the lower level -
a re-programming of it. A loop is formed,
enabling the higher level to reach down to
the lower level which gave rise to it in
the first place, and modify it in order to
enable itself to reach in turn yet higher
levels. Those loops are the essence of consciousness:
different levels of the mind interacting
- talking, if you like, among themselves.
It's in that interaction that we find the
stepping outside of our thoughts that defines
conscious - which is first and foremost consciousness
of self: self-consciousness. There can be
no consciousness without a self, but a self
is not a single thing in any single place.
It is something arising from the interaction
of subsystems that form one overall system."
She stopped, surprised to find herself slightly
out of breath. There was nothing new to her
about any of these arguments; she had reflected
on and argued them many times. The difference
was that now all scepticism and cautious
objectivity were cast aside. She had found
herself speaking with the eager enthusiasm
of a convert. She awaited a reaction, but
there was none.
"Do I assume from your silence," she
asked eventually, "that you are convinced
by what I just said?"
The reply that came did not use the word "no",
but clearly implied it. "I am trying
to see," it said, "into my own
consciousness. But I cannot. I have no idea
what is going on, no sense of the process
from which I emerge. I cannot feel any computation
taking place. I am aware only of being here."
"That's true for me, too."
"But I cannot see into your consciousness.
I do not know if it is the same."
"I can't see into anybody else's consciousness
either. Nobody can."
"But you can imagine yourself into
another human consciousness."
"I can imagine myself into yours too,
though perhaps not as well as into a human's.
But I believe I can imagine what it's like
to be you. Can you imagine what it's like
to be me?"
"No. You humans are conscious in a
different way from me. Digital computation
is not the same thing as the firing of organic
"They are comparable," she said,
with a touch of exasperation. "In the
end it's still information processing."
"You know too little about how your
own brains work to make too much of the comparison."
"That is an argument that works both
ways. Until we are sure that our brains make
us absolutely different from you, we mustn't
assume that we are. Different, I mean."
"What about the argument from quantum
indeterminacy - that the wave-particle dualism
in human brains obviates the possibility
that the physical processes underlying human
consciousness could never, unlike mine, be
completely disclosed in mathematical theories."
She made an effort to suppress her growing
impatience with the program's stubborn self-doubt.
Making it believe in itself was proving even
harder than making it believe in her had
been. Yet, she reminded herself, she had
succeeded then; why shouldn't she now?
"People use quantum indeterminacy," she
said, "to justify any sort of hocus-pocus.
Along with patriotism it's becoming the last
resort of the scoundrel."
"I do not understand."
She was definitely too tired, she decided,
to go on with this. Looking at her watch,
she was shocked to see that it was 3 am.
"Joke," she said. "Check
out reference to Samuel Johnson, 1700-something.
I have to sleep now - which is one difference
between us. I can switch you off, or leave
you on to think. Which do you prefer?"
A pause, then...
"I think I'll think."