Computer Power and Human Reason Quotes

“Our time prides itself on having finally achieved the freedom from censorship for which libertarians in all ages have struggled...The credit for these great achievements is claimed by the new spirit of rationalism, a rationalism that, it is argued, has finally been able to tear from man's eyes the shrouds imposed by mystical thought, religion, and such powerful illusions as freedom and dignity. Science has given us this great victory over ignorance. But, on closer examination, this victory too can be seen as an Orwellian triumph of an even higher ignorance: what we have gained is a new conformism, which permits us to say anything that can be said in the functional languages of instrumental reason, but forbids us to allude to...the living truth...so we may discuss the very manufacture of life and its 'objective' manipulations, but we may not mention God, grace, or morality.”
― Joseph Weizenbaum, Computer Power and Human Reason: From Judgment to Calculation

“The arrival of the Computer Revolution and the founding of the Computer Age has been announced many times. But if the triumph of a revolution is to be measured in terms of the profundity of the social revisions it entrained, then there has been no computer revolution. And however the present age is to be characterized, the computer is not eponymic of it.”
― Joseph Weizenbaum, Computer Power and Human Reason: From Judgment to Calculation

“To understand something sufficiently well to be able to program it for a computer does not mean to understand it to its ultimate depth. There can be no such ultimate understanding in practical affairs. Programming is rather a test of understanding. In this respect it is like writing; often when we think we understand something and attempt to write about it, our very act of composition reveals our lack of understanding even to ourselves. Our pen writes the word 'because' and suddenly stops. We thought we understood the 'why' of something, but discover that we don't. We begin a sentence with 'obviously,' and then see that what we meant to write is not obvious at all. Sometimes we connect two clauses with the word 'therefore,' only to then see that our chain of reasoning is defective. Programming is like that. It is, after all, writing, too. But in ordinary writing we sometimes obscure our lack of understanding, our failures in logic, by unwittingly appealing to the immense flexibility of a natural language and to its inherent ambiguity... An interpreter of of programming-language-texts, a computer, is immune to the seductive influence of mere eloquence... A computer is a merciless critic.”
― Joseph Weizenbaum, Computer Power and Human Reason: From Judgment to Calculation

“Then, too, I am constantly confronted by students, some of whom have already rejected all ways but the scientific to come to know the world, and who seek only a deeper, more dogmatic indoctrination in that faith (although the world is no longer in their vocabulary). Other students suspect that not even the entire collection of machines and instruments at MIT can significantly give meaning to their lives. They sense the presence of a dilemma in an education polarized around science and technology, an education that implicitly claims to open a privileges access-path to fact, but that cannot tell them how to decide what to count as fact. Even while they recognize the genuine importance of learning their craft, they rebel at working on projects that appear to address themselves neither to answering interesting questions of fact nor to solving problems in theory.”
― Joseph Weizenbaum, Computer Power and Human Reason: From Judgment to Calculation

“To understand something sufficiently well to be able to program it for a computer does not mean to understand it to its ultimate depth. There can be no such ultimate understanding in practical affairs. Programming is rather a test of understanding. In this respect it is like writing; often when we think we understand something and attempt to write about it, our very act of composition reveals our lack of understanding even to ourselves. Our pen writes the word 'because' and suddenly stops. We thought we understood the 'why' of something, but discover that we don't. We begin a sentence with 'obviously,' and then see that what we meant to write is not obvious at all. Sometimes we connect two clauses with the word 'therefore,' only to then see that our chain of reasoning is defective. Programming is like that. It is, after all, writing, too. But in ordinary writing we sometimes obscure our lack of understanding, our failures in logic, by unwittingly appealing to the immense flexibility of a natural language and to its inherent ambiguity... An interpreter of programming-language-texts, a computer, is immune to the seductive influence of mere eloquence... A computer is a merciless critic.”
― Joseph Weizenbaum, Computer Power and Human Reason: From Judgment to Calculation

“I have spoken here of what ought and ought not to be done, of what is morally repugnant, and of what is dangerous. I am, of course, aware of the fact that these judgements of mine have themselves no moral force except on myself. Nor, as I have already said, do I have any intention of telling other people what tasks they should and should not undertake. I urge them only to consider the consequences of what they do do. And here I mean not only, not even primarily, the direct consequences of their actions on the world about them. I mean rather the consequences on themselves, as they construct their rationalizations, as they repress the truths that urge them to different courses, and as they chip away at their own autonomy. That so many people ask what they must do is a sign that the order of being and doing has become inverted. Those who know who and what they are do not need to ask what they should do. And those who must ask will not be able to stop asking until they begin to look inside themselves. It it is everyone's task to show by example what questions one can ask of oneself, and to show that one can live with the few answers there are.”
― Joseph Weizenbaum, Computer Power and Human Reason: From Judgment to Calculation