e math society group

There is something very significant about what has happened here. It shows one difference between people and machines. It is possible to program a computer to do a routine task in such a way that the computer will never notice even the most obvious facts about what it is doing; but it is inherent in human consciousness to notice some facts about the things one is doing. If you punch "1" into an adding machine, and then add 1 to it, and then add 1 again, and again, and again, and continue doing so for hours and hours, the machine will never learn to anticipate you, and do it itself, although any person would pick up the repetitive behavior very quickly. Or, to take a silly example, a car will never pick up the idea, no matter how much or how well it is driven, that it is supposed to avoid other cars and obstacles on the road; and it will never learn even the most frequently traveled routes of its owner.



The difference, then, is that it is possible for a machine to act unobservant; it is impossible for a human to act unobservant. Notice I am not saying that all machines are necessarily incapable of making sophisticated observations; just that some machines are. Nor am I saying that all people are always making sophisticated observations; people, in fact, are often very unobservant. But machines can be made to be totally unobservant; and people cannot.



It is an inherent property of intelligence that it can jump out of the task which it is performing, and survey what it has done; it is always looking for, and often finding, patterns. The jumping out can be very simple, as for example getting bored watching a television program, therefor zapping to another channel; or realizing that you are wasting your time watching television in general, therefor turning it off. The jumping out can be very complex, as for example breaking a pattern of life, getting divorced, quitting your work or just stop short for some moments every day meditating on your existence and the meaning of your life.



How well have computers been taught to jump out of the system? Here is one example which surprised some observers. In a computer chess tournament not long ago in Canada, one program - the weakest of all the competing ones - had the unusual feature of quitting long before the game was over. It was not a very good chess player, but it at least had the redeeming quality of being able to spot a hopeless position, and to resign then and there, instead of waiting for the other program to go through the boring ritual of checkmating. Although it lost every game it played, it did it in style. A lot of local chess experts where impressed. Thus, if you define "the system" as "making moves in a chess game", it is clear that this program had a sophistcated, preprogrammed ability to exit from the system. On the other hand, if you think of "the system" as being "whatever the computer had been programmed to do", then there is no doubt that the computer had no ability whatsoever to exit from that system.