Notes on Society/Debate
Benjamin D. Singer
Abstract. In recent years, we have witnessed widescale quality problems in industries, professions and institutions, accompanied by arguments over causes of decline, and confusion over systems of standards assessment. Because there has been no general systematic attempt to comprehend standards -- their meaning, operation, decline -- across different institutions, I have suggested models for understanding the dynamics of standards, focusing upon the growth and problems generated by bureaucratized criterial systems in institutions such as medicine, law, education, science, mass culture, which clash with values and standards. Based on these issues, a proposal is made for the development of a social science of standards.
Résumé.
Dans les dernières années des problèmes de
qualité à grande échelle ont surgi dans les
industries, professions et institutions. En constatant ces problèmes,
on a invoqué des causes de déclin et la confusion
causée par les systèmes d'évaluation selon
des normes. Parce qu'il n'y a pas eu d'essai systématique
général de comprendre les normes -- leur signification,
leur mode d'opération et leur déclin -- dans les
différentes institutions, j'ai suggéré des
modèles pour comprendre la dynamique des normes. Je me
suis concentré sur la croissance et les problèmes
générés par les systèmes de bureaucratisation
dans les institutions de médecine, de droit, d'éducation,
de science et de culture de masse qui sont en conflit avec les
valeurs et les normes. De ce fait, on propose de développer
une science sociale de normes.
Social critics argue that societies pass through periods in which the standards of major institutions crumble. During the past decade, for example, medicine, law, education, industry, accounting and science have been criticized for declining standards (Brinkley, 1985; Drill, 1985; Katz, 1984; Sullivan, 1987; Van Harrison, 1986). Government agencies and concerned physicians have estimated that as many as 45,000 incompetent physicians were practicing in the US (Brinkley, 1986); malpractice claims have risen sharply (Van Harrison, 1986: 3), medical schools have expanded too rapidly, recruiting too many students and inadequately testing them (Bergen, 1988). In science, repeated allegations of shoddy research and fraud have filled the pages of newspapers and journals (Singer, 1989; Goldstein, 1991). In education, the decline in standards, from elementary schools to universities, has been documented through extensive research on student and teacher proficiency testing (De Witt, 1991; Henry, 1994). Judges and lawyers have complained of the lack of competence of contemporary lawyers (Strehlow, 1981) and have criticized the excessive numbers turned out by law schools. The accounting industry -- watch guard of fiscal matters in the largest US companies -- has come under unprecedented attack, spearheaded by Congressional hearings, for failure to "meet professional standards" in the massive collapse of savings and loan associations during the late 1980s (Wayne, 1989; Feinberg, 1987; Buchanan, 1991; Robinson, 1988). American industry has suffered substantial losses because of the decline in quality (the largest auto maker, in spite of being defined as "excellent" lost nearly half its domestic market in the 1980s). The service industries have followed other industries and institutions in decline according to academic and business analysts (Quinn and Gagnon, 1986).
During this period, many books and articles dealing with the problems in quality have appeared, offering diverse and often disparate diagnoses and nostrums for what has appeared to be a widespread decline in standards (Peters and Waterman, 1982; Gabor, 1992; Dobyns and Crawford-Mason, 1991). Major professions and institutions have been rethinking and debating the way in which standards are developed, assessed and applied. In education, for example, Albert Shanker, spokesperson for American teachers, has publicized the debate over educational standards in a series of editorial advertisements in leading newspapers and magazines (Shanker, 1988, 1990); there has been spirited debate concerning the examinations produced by Educational Testing Service, and upon which much of the US educational establishment depends; and the way in which standards are applied in decisions on graduate fellowships in Canada has been questioned (Singer, 1994). Events such as these lead some theorists and analysts to assert a general decline in standards. The most prominent theorist of decline, Ortega y Gasset, surveying what he perceived to be the low state of standards of European civilization in the third decade of this century, blamed the decline on the ascendance of barbarian Mass Man over the cognoscenti Men of Excellence who historically served as the standard setters of civilization (Ortega y Gasset, 1957). In more recent years, the historian Barbara Tuchman drew attention to both a "decline in competence in public life" (Tuchman, 1987) and the prevailing "anti-excellence spirit of our times" (1982), and psychologist M. Brewster Smith, following such critics as Alan Bloom, described a prevailing "uncertainty about all standards, whether they concern knowledge, art, or morals -- or the utter rejection of standards" (Smith, 1994: 406).
This uncertainty about standards and the critiques that accompany them raises questions about the very nature and origin of standards as well as about their dynamics -- their rise, their fall, and their maintenance. It speaks to our lack of resources for assessing claims and counterclaims concerning standards, and suggests the need for a new sociological paradigm, a sociology of standards, to address these issues as a sociological problem. For while at some time in the past our experience with the essential things of our lives -- the services and products of crafts people and professionals -- was most often individual and empirical, today the process of making individual judgments is being progressively replaced by collective systems and formal institutions such as professional societies, testing and rating organizations, government bureaus and international associations. This has resulted in the externalization of standards (Marcuse, 1982: 142). The diminishment of personal experience relative to external forces of a collective nature means that standard-setting has become increasingly sociological in nature.
A Sociology of Standards would help to establish the identity of a body of standards for important institutions and professions and would aid in developing an understanding of the different drivers of standards. Moreover, it would make possible comparisons across institutions and professions, and facilitate the analysis of gaps between the core values of institutions and the operational standards or criteria actually in place. It is the latter problem which this paper primarily addresses. I shall examine a key element of the problem of standards: the disparity or clash between the values and standards of a social structure and the criterial systems ostensibly generated to guard them.
The social nature of standards is demonstrated by the fact that they are so often driven by the very systems employed for their assessment and measurement, the criterial systems of bureaucracies, professions, and other institutions and social organizations. Some criterial systems have become independent institutions in their own right, e.g., television ratings, product and services ranking polls, and educational testing organizations. Such systems -- all of which are products of social organizations -- become the arbiters and manipulators of standards, their mechanisms often leading to self-fulfilling prophecies.
Thus, in this paper, it will be important to analyze the bases for the ascendance of criterial systems and, toward that end, to review critiques of our dependence on criteria; to understand processes behind the production and adoption of criteria; to examine the hidden biases in their selection and employment as well as the types of criteria that emerge from this and the consequences of their increasing power to influence behavior. This will be followed by a discussion of possible approaches to a sociology of standards. However, the first question to be addressed deals with the different ways in which the concept "standards" is defined.
The Standards Complex
Because the term standards is used in many different ways, a critical first step toward a sociology of standards is to attempt to clarify the concept. The term is often treated as a synonym for values and for criteria (and occasionally for standardization and for taste) (Bloch, 1977). Each is a component in what might be called a "standards complex," but each should be differentiated. We begin by exploring the role of values in relation to standards and criteria.
Values
Values express our general goals or ends, our preferences, our desired outcomes, our ideals, and even the means to achieve goals. Values are the foundation of all standards. As Anderson notes: "The close association between standards and values is not accidental because standards arise from and are dictated by values and objectives" (Anderson, 1976: 213). Beuk agrees: "To a certain extent, every standard, every judgment, every appreciation is an expression of a value" (Beuk, 1984: 1). Yet the value is not always apparent as the driver of standards for, as Robin Williams points out, "Even when not explicitly stated, values can often be inferred directly from verbal materials"(Williams, 1956: 378). We assume they lie behind statements of standards or of criteria but that assumption ought to be tested.
Standards
The term standards is commonly employed in four different ways:
1)
As quality and/or as its minima
2)
As consistency or standardization
3)
As a "high ideal" to be achieved or exceeded
4)
As interchangeable with criteria.
"Standards" is used most frequently in reference to quality, or as a model or template for purposes of comparison, often implying some minimal qualities or characteristics, consistency or ideal state. It also has frequently been employed as a synonym for explicit criteria or specifications for evaluating some thing or process.
Table 1 below illustrates the four major usages of the term.
While Table 1 indicates the typical ways in which the term is used, for purposes of analytic clarity, standards can be distinguished from values as substantively focused evaluative statements about a thing, process or outcome, "located" between values and criteria. Values are statements of general goals, objectives, and purpose; standards provide models for achievement at the individual, group or institutional level. In a well-ordered world, standards would be based on significant values -- values that are not always explicit or visible -- and that arise from the attempt to both preserve and to achieve value objectives.
Criteria
Criteria function as the dynamic aspect of standards, i.e., standards are held, criteria are applied. As Fincher has said, "standards should express the expectations we have carefully and systematically formed for competence and achievement, and criteria should provide us with objective indicants of how well standards have been met" (Fincher, 1984: 501). Thus the ideal-typical model advanced by authorities such as Anderson, and Fincher, asserts that criteria operationalize the standards for which values are the foundation. When values become substantive and specific, they emerge as standards -- normative ideas about quality to be expected. The application of measures, of differentia of quality or consistency then becomes the function of criteria -- at least in theory.
However, criteria, increasingly a product of formal organizations and other social forces, often reflect pragmatics -- the objectives and constraints of professional, institutional and personal interests, bureaucratic procedures and expediency (Altheide and Johnson, 1980). While standards are "closer" to values, criteria can be described in terms of Mannheim's concept of "functional rationality": they may be mechanically imposed and carried out without requiring any understanding of their purpose or meaning (Mannheim, 1951).
Criteria typically are explicit and formal; they also are much more varied and arbitrary than the values they presumably serve. Thus the criteria in place are the varied derivatives not only of obvious pragmatic and objective considerations but also of social, political and market processes or drivers.
An example of criteria applied to the evaluation of academic staff throws some light on the distinction between criteria, standards, and values. A university department's policy guide on promotion states: "For promotion to associate professor with tenure, a student rating of 3.5 and a minimum of six publications are required." This statement has been described as one of "standards." However, in terms of the model presented here, standards should be substantive, should describe the qualities sought -- such qualities as depth of scholarship, original research of an important nature, creativity. The university values represented might be those describing the quest for and dissemination of knowledge as a general statement of the university's purpose or central value. Thus, it is not values or standards that are being represented here, but rather, criteria -- surrogates for standards, treated as if they were standards.
The Ascendance of Criteria
Ours can be described as an "Age of Criteria," in which the desire for certainty is satisfied through measurement. Products, services, institutions and people are measured, evaluated, rated and ranked. Books of lists that rank and rate varied things from cities to airlines have become bestsellers. Millions of dollars are spent reporting the results of commercial criterial organizations that survey consumers in order to rank preferences in automobiles, computers or soft drinks. Travel books rate resorts, restaurants, and hotels, using the ubiquitous "star" system. Television programs and records are ranked, as are books on The New York Times Book Review Best Sellers list. Good doctors and poor doctors are now evaluated in books and magazines rating medical services (Who's Best? 1990), as are hospitals (Wright, 1995; McLeod, 1994). Faculty members are rated using composite measures of teaching and publication (Singer, 1989). Universities are ranked (Johnston, 1994), as are hopeful applicants who take various admissions tests. Even judges have become subjects of rating and ranking by lawyers (Crawford, 1989). Criterial industries have become a billion dollar enterprise and a leading influence in setting "standards."
It is evident that the standards process, once in great part a product of individual experience, has now become externalized and depends upon formal systems and the criteria they employ -- institutionalized criterial systems -- visible (but not necessarily valid) indicators of standards and values.
To understand the role criteria play in important institutions in our society, we must examine the very nature of criteria as an expression of measurement. Then we must assess the process by which criteria are generated and rationalized, and we must be aware of the biases involved in their selection. Such questions, dealing with the nature of criteria, their origin, use and validation are the desiderata, the necessary condition for beginning a more general inquiry into the social bases of standards.
While in the discussion that follows, the examples are limited to professional and educational fields, the general principles may be extended and will be applicable to other sectors of society.
The Critique of Criteria as Measurement
The bias toward measurement generated by criteria per se has been criticized because it (paradoxically) increases ignorance and deflects from critical awareness of important issues. The measurement paradigm, as Feinstein asserts, once a tool of enlightenment, "has now led to major intellectual crises that remain unsolved by various ad hoc modifications of the basic paradigm, and is now being used to substitute for enlightened thought or to thwart it" (Sydenham, 1979: 4). Criteria become a critical issue when the standards they are presumed to operationalize and the values that are their foundation, are bypassed in the concern to generate and apply measures: when fewer questions are asked about what is being assessed than how to measure it (Mannheim, 1951). This tendency has been particularly noted within fields such as education. As Ernest Boyer, president of the Carnegie Foundation for the Advancement of Teaching, said in criticizing traditional educational measurement: "we emphasized testing without asking the crucial question of what we were trying to measure" (Leslie and Wingert, 1990).
Paralleling Boyer's criticism of educational measures, Shanker has agreed: "These are measures, but they don't measure what we want to know. They don't give us any concrete sense of our students' skills or limitations" (Shanker, 1990). In a similar vein, Rogers has pointed out that good medical practice is impeded by the emphasis on easy measurement at the expense of other values: "Excessive attention to measurable attributes (Medical College Admission Test scores, National Board scores, etc.) is driving out attention to less easily quantifiable items that are important in identifying the good future doctor" (Rogers, 1988).
Producing Criteria
Such apprehension over the application of measures leads us to examine the rationales by which criteria are generated, selected and validated in institutional settings. The term "rationales" is used to describe a process that functions as both a mechanism and legitimation.
Table 2 indicates the rationales for three of the most frequent patterns for the majority of institutions: the use of experts, the use of normative groups for comparison, and validation by external variables.
1. Experts as rationale
Initially, experts are employed to generate criterial instruments. In health care, for example, a panel of experts may set "standards" for quality of care (McAuliffe, 1978: 645). In education, experts in various disciplines are asked to create tests and examinations for admission to schools and to various professional fields.
2. The Normative Group as rationale
Either an internal ranking provides the rationale and thus the criterion for evaluation of individuals or collectivities, or an external group is employed normatively. Without substantive knowledge, a distribution is examined and varied cutoff points may be selected. Thus, the normative group becomes a "standard" to be met or exceeded, and the process is described by educators as "norm-referenced" measurement, in contrast with substantive criteria measures often referred to as "criterion-referenced" testing.
However, there is considerable debate that reflects confusion over such differentiation. Some education theorists and measurement researchers believe the distinction is not useful, since criteria are often established normatively, yet others, such as Glass and Smith, have argued that "In our opinion, `criterion levels' or standards cannot be determined other than arbitrarily" (Glass and Smith, 1978: 12).
3. External Criteria as rationale
Regardless of the origins of the criterial system, success in one measure is predictive of success in another. For example, the quality of a doctor's career history may be considered predictive of low future mortality rates, high marks as an undergraduate are predictive of acceptance to graduate or professional school or to a profession or income level after graduation.
While critics argue over the quality and validity of major modes of establishing criteria, what seems often to go unnoted is the role of biases in selection of criteria, which occurs at an early stage in the process. The biases explored below are not exhaustive but are suggestive of the universe of criterial selection biases previously ignored.
Criterial Biases
In all processes of evaluation, biases, rarely visible as such, influence selection of criteria. The criteria that become the keystone of the criterial system may emerge as the result of existential biases, rather than being driven by values, objectives or substantive standards.
The bias of availability occurs whenever criteria, regardless of their value or relevance, are available in contradistinction to potential criteria that are not visible. Availability bias is significant because it sets the agenda for what will be considered important. Media research is an instructive example: audience figures are available but potentially more important data on the quality or impact of programs are not; hence, the major criteria by which programs are evaluated are audience size and other audience attributes. In a highly computerized society, what has been coded and put into a database exists and may be selected as criteria; that which is not found during a data search does not exist, as Weizenbaum once warned, for "what counts as fact is determined by the system" (Weizenbaum, 1976).
The bias of classifiability. Things that have been classified already or which are easily classified lead to selectivity of perception. As Daniel Boorstin has pointed out, classifiability leads historians to include some facts and to exclude others in their data (Boorstin, 1987).
Related to (but not the same as) the bias of classifiability is the bias of measurability. What is easily measured may be the basis for inclusion as a criterion. A common problem in industrial society is the tension created by differential rewards for jobs where the most demanding tasks may be the least able to be assessed. The technical ease of measuring determines choice of criteria by bureaucracies since they favour criteria that are easily available and that lend themselves to unambiguous operationalization. Quantification of output fills the bill very nicely. As Eccles argued in calling for change in criterial systems in the business sector, "What gets measured, gets attention, particularly when rewards are tied to the measures" (Eccles, 1991: 131). Criteria gravitate toward the most highly visible manifestations of major institutions.
The bias of vividness of information. As Nisbett and Ross have pointed out, "vividness of information exerts a disproportionate impact on inferences" (Nisbett and Ross, 1980: 45). Thus, to make certain that a dimension is noted and included as a criterion, individuals subject to evaluation may seek to emphasize its characteristics (as in the communication-display behavior of many species of animals).
The bias of activity. As noted in some studies employing semantic differential scales, that which is active or which can be made to seem active appears superior to what is perceived as inactive (Osgood and Succi, 1969: 49). Thus we assume that a doctor who prescribes pills or hospitalization when facing a sick patient will appear to be doing something and will be perceived as more competent than one who doesn't take immediate action.
Types of Criteria
Recent controversies over standards in professions and institutions reflect confusion over the criteria to be employed in assessing service givers and in the methods of enforcing standards. While laying out an exhaustive "list" of criteria would be beyond the purview of this paper, I would like to draw attention to three primary criteria found in most professions and many institutions. I do not argue that these are the only criteria but rather that they are the most frequently employed, and the most important. The criteria, whether employed explicitly or implicitly, comprise outcome measures, credentials, and process/interaction.
In the case of outcome measures, the specific criterion might be how many suits a lawyer has won, or how many patients a physician has cured. In a different context, one could ask what was the measured knowledge of the students upon graduation, or how many were admitted to graduate or professional schools? The second criterial pattern emphasizes the role of credentials in evaluation: the diplomas, letters of reference, peer review reports, honors bestowed, past jobs, etc. In the case of process-interaction, it is important to evaluate the doctor's bedside manner, or her taking of the history, or the teacher's interpersonal communication, or perhaps the way in which a lawyer exhibits his knowledge of court procedure during a trial.
Table 3 indicates some examples of the application of the three criteria.
Outcomes
One might expect outcome measures -- evidence of success whether in product sales, patients cured or cases won -- to be the ultimate basis for criteria in any system of standards in industry, esthetics or professions. As Abbott has argued concerning professions, there is an objective bottom line involving measurable results that all professions must meet in order to maintain their power, status and very existence (Abbott, 1988: 140). However, there is sharp debate over the adoption of outcome criteria, reasonable and relevant as they might seem to be. Recent controversies in medicine, education and industry, concern the accuracy, fairness and workability of "outcome measures" as criteria (Gottlieb et al., 1990; Geigle and Jones, 1990; Kassebaum, 1990).
While suggested outcome criteria in education have included dropout rates, tests of knowledge such as E.D. Hirsch's cultural literacy scale (Hirsch, 1988), admission to university, scholarships, jobs held or income level after graduation, the most frequently employed and criticized outcome measure is grades, a controversial criterion (Singer, 1994). As well, there have been repeated calls for national standardized testing of high school students as a way of assessing the quality of education in the different jurisdictions (De Witt, 1991).
The outcome measures selected also will reflect the bias of groups who control the selection of criteria as well as the technical ease with which output can be measured. Moreover, the universal validity of outcome measures is affected by the fact that professionals who may pick and choose assignments can avoid those that are complex or risky. Doctors refer patients to specialists, overdo test procedures, or refuse to accept high-risk patients or risky procedures, thus presenting fewer failures. Lawyers take on cases in which they feel the odds favor their success. Professors write banal substantive papers with a high probability of acceptance by "refereed" journals that favour replicative work, or they insist on teaching classes in which their ratings by students are not in danger (Singer, 1989). Producers choose to do television programs of a type that attracts substantial ratings rather than producing complex, demanding programs that might garner a higher quality but smaller audience. Clearly, the criterial system keyed to outcomes is selectively employed to control the observed outcomes, rather than to measure differences in order to achieve higher standards. In this way, criterial systems exert control over standards, rather than standards driving criteria.
Credentials
Credentials are a well-established part of the criterial apparatus that have become increasingly important in a society of high complexity as a surrogate for firsthand experience with people and institutions. Credentials of professional persons applying for positions, institutional memberships or licensure (a credential in its own right) typically consist of written records of past performance and accomplishments such as school grades and standing, references by supervisors or peers, research accomplishments, special training.
Credentials and credentialism have come under increasing scrutiny in recent years. Questions have been raised about their accuracy and veracity, as well as their predictive performance. While traditionally they have been a reasonable method for making decisions, there is growing doubt about the wholesale use of credentials as major criteria. As Professor Donald Klein puts it, "Ideally, we are interested in measuring output; we have no way of measuring output so we measure input -- marks, diplomas, et al. All effort has been on credentialling, and that misses the mark. For credentialling determines minimal standards."
Process-Observation
The third common criterion, process-observation, may be employed at three different authority levels: peer, superordinate personnel (such as a senior physician or judge or a board of education inspector), and subordinates. The experienced doctor observes the practitioner, medical student or intern examining a patient or taking a case history. The senior teacher visits the classroom of the teacher who is to be evaluated. The judge is a firsthand (often critical) witness to a stream of lawyers presenting (or mispresenting) cases at court (Strehlow, 1981). Process-observation has been criticized because of its essential subjectivity (Gross, 1988).
Consequences of the Ascendance of Criterial Systems
We have witnessed the rise of bureaucratized criterial systems, which have become the predominant generators and maintainers of standards. This influence extends far beyond institutions and professions, to broad patterns of behaviour by the public; increasingly, people choose books, cars, doctors, lawyers, and television programs on the basis of the influence of ever more powerful public criterial systems -- ratings and rankings, both in the US and Canada (Singer, 1994).
Because of this growing pattern, another issue has emerged. The criterial systems have generated problems of an ironic nature as people and organizations work incessantly to find ways to overcome their effects. In education, where the effects have been most apparent, it was discovered a few years ago that school systems in every state in the US, attempting to cope with the fact of standardized tests, claimed their students' scores on standardized tests to be above average -- a statistical impossibility (Finn, 1992); teachers have learned to teach to the test and parents frantically undertake expensive programs to train children to overcome tests and their idiosyncrasies. Albert Shanker, commenting on the growing power of standardized testing asserts "the tests now rule the schools ... and with tests now driving curricula, rote memorization and recall of isolated facts rule the classroom" (Shanker, 1988). Understandably, parents enroll children in courses to teach them how to work around the system; and mass media reinforce the notion that it is the criterial system, not learning, that is important as feature stories urge parents to prepare their children through learning such test-taking skills as how to skip answers efficiently (Kiester and Valent, 1989). A further irony is that teachers' colleges that admit aspiring teachers based on their test scores help them prepare for the tests; thus "the tests and the school programs are mutually reinforcing" (Owen, 1984: 14).
The criterial systems encourage professionals, such as physicians, to take advantage of the frailties of the systems to falsify credentials; scientific researchers publish fraudulent findings to make it through criterial systems for tenure and promotion. Rating systems have become the norm in the assessment of university research output: quantity of published papers, number of citations, rank of the journal are now everyday statistics in most academic disciplines, and tenure depends upon the way in which such systems are constructed at different universities, and on the ability of professors to find ways around these systems (Singer, 1989). Among mass culture institutions, broadcasters manipulate the ratings systems by presenting special reports on sensational topics such as teenage prostitution or drugs in local schools during the "sweeps" period of audience surveys to boost ratings (Clark, 1989). And authors have learned that the quality of their books may be secondary to the boost that sales will be given by "making the list" (e.g., The New York Times bestseller list), and hence negotiate with publishers for high promotion budgets. Like others, they realize that criterial systems not only reflect reality but have the power to change it.
Toward a Sociology of Standards
The problems blamed on declining standards of so many institutions, professions and disciplines are not necessarily problems of standards, per se: our lack of analytic tools has obscured our vision. Criterial systems -- frequently functioning as surrogate standards -- have themselves all too often become the problem. Defects in the criterial systems -- error, bias, ambiguity -- have become structural features of systems. The struggles to overcome the effects of problematic systems, the clash between the criterial systems and the values and standards they should represent, lead to a further deterioration in quality.
To cope with the nature and power of these criterial systems, we need to develop transinstitutional approaches to the study of standards, as well as multidisciplinary perspectives that will help us to identify and to understand the values and standards that lie hidden beneath criterial systems. We need to break out of the confines of specific fields and to develop a general social science of standards.
While this essay has focused upon the role of criterial systems, a sociology of standards should also investigate in greater depth the other major forces that drive standards. Beyond the scope of this paper but requiring extended analysis are such recognized influences on standards as marketplace dynamics (heretofore addressed only by economists), social organizational forces such as bureaucracy and professional systems with their powerful special interest groups, and the increasingly influential specialized criterial institutions alluded to earlier. To begin to accomplish these things, we must be willing to actually confront the idea of standards as a phenomenon sui generis -- something the social sciences have all too often eschewed in the past for ideological reasons -- if we are to seriously address the issue meaningfully in social policy initiatives and if we hope to regain some sense of control over the standards that we believe ought to guide our society.
References
Abbott, Andrew1988 The System of Professions. Chicago: University of Chicago Press.
Altheide, David L. and John M. Johnson 1980 Bureaucratic Propaganda. Boston: Allyn & Bacon.
Anderson, Ronald1976 "Technical Standards and Performance Standards in the Pursuit of Quality," Behaviour Research Methods & Instrumentation 8: 2111-2217.
Berger, Joseph 1988 "Panel Recommends Dramatic Changes in Nation's Medical Schools," The New York Times (November 13).
Beuk, Ceesh 1984 "A Method for Reaching a Compromise Between Absolute and Relative Standards in Examinations," Journal of Educational Measurement 21: 147-52.
Bloch, Doris 1977 "Criteria, Standards, Norms-Crucial Terms in Quality Assurance," Journal of Nursing Administration 7: 20-30.
Boorstin, Daniel J. 1987 "The Historian: A Wrestler With the Angel," The New York Times (September 30).
Brill, Steven1985 "Curing Doctors," The American Lawyer 7: 1.
Brinkley, Joel 1985 "Should Doctors Be Given a More Thorough Examination?" The New York Times (November 10).
1986 "U.S. Seeks Tighter Control of Doctors," The New York Times (February 4).
Buchanan, William W. 1991 "Developing Standards (Letter)," Globe and Mail (August 8).
Cipolla, Carlo M. 1967 Clocks and Culture 1300-1700. London: Collins.
Clark, George 1989 "Trash Shows Limited By Their Own Luridness," London Free Press (January 14).
Crawford, Michael G. 1989 "Judging Judges," Canadian Lawyer 13: 18-33.
De Witt, Karen
1991 "The Push to Consider A Once Taboo Subject: National School Tests," The New York Times (February 3).
Dobyns, Lloyd and Clare Crawford-Mason 1991 Quality or Else. Boston: Houghton Mifflin.
Eccles, Robert G. 1991 "The Performance Measurement Manifesto," Harvard Business Review 69: 131-37.
Feinberg, Andrew 1987 "Accountants Try to Put a Little Kick in Their Image," The New York Times (September 27).
Fincher, Cameron 1984 "Norms, Standards, and Criteria," Research in Higher Education 20: 499-502.
Finn, Chester E., Jr. 1992 "Turn On the Lights (Editorial Advertisement)," The New York Times (January 12).
Flexner, Abraham 1910 Medical Education in the United States and Canada: A Report to the Carnegie Foundation for the Advancement of Education. New York.
Fromm, Erich 1941 Escape from Freedom. New York: Avon Books.
Gabor, Andrea 1992 The Man Who Discovered Quality. New York: Penguin Books.
Geigle, R. and S. B. Jones 1990 "Outcomes Measurement: A Report from the Front," Inquiry 27: 7-13.
Giner, Salvador 1976 Mass Society. London: Martin Robertson.
Glass, Gene V. and Mary Lee Smith 1978
The Technology and Politics of Standards," Educational Technology 18: 12-18.
Goodstein, David 1991 "Scientific Fraud," The American Scholar 60: 505-15.
Gottlieb, L.K., et al. 1990 "Clinical Practice Guidelines At an HMO: Development and Implementation in a Quality Improvement Model," ORB 16: 80-86.
Gross, James A. 1988 Teachers on Trial: Values, Standards and Equity in Judging Conduct and Competence. Ithaca, NY: ILR Press.
Henry, William A., III 1994 In Defense of Elitism. New York: Doubleday.
Hirsch, E.D. 1988 Cultural Literacy. New York: Random House.
Johnston, Ann Dowsett 1994 "Measuring Excellence," Maclean's (November 14).
Kassebaum, D.G. 1990 "The Measurement of Outcomes in the Assessment of Education Program Effectiveness," Academic Medicine 65: 293-96.
Katz, Dolly 1984 "Bad Doctors/Who Is Watching? 6/7," The Detroit Free Press (April 6).
Kiester, Edwin and Sally Valint 1989 "How to Improve Your Child's Test Scores," Reader's Digest.
Leslie, Connie and Pat Wingert 1990 "Not As Easy As A, B, or C," Time (January 8).
Mannheim, Karl 1951 Man and Society in an Age of Reconstruction. New York: Harcourt, Brace.
Marcuse, Herbert A. 1982 "Some Social Implications of Modern Technology." Pp. 138-82 in The Essential Frankfurt School. New York: Continuum.
McAuliffe, William E. 1978 "On the Statistical Validity of Standards Used in Profile Monitoring of Health Care," American Journal of Public Health 68: 645-51.
McLeod, Don 1994 Death Rate Data Stir Controversy," AARP Bulletin: 12.
Morgan-Webb, Charles 1934 The Rise and Fall of the Gold Standard. New York: MacMillan.
Mulholland, Sister Mary Ambrose (ed.) 1941 Early Guild Records of Toulouse. New York: Columbia University Press.
Nisbett, Richard and Lee Ross 1980 Human Inference: Strategies and Shortcomings of Social Judgement. Englewood Cliffs, NJ: Prentice-Hall.
Olson, Walter K. 1991 The Litigation Explosion. New York: Truman Talley Books/E.P.Dutton.
Ortega y Gasset, Jose 1957 The Revolt of the Masses. New York: W.W. Norton.
Osgood, Charles E. and George J. Suci 1969 "Factor Analysis of Meaning." P. 49 in Semantic Differential Technique -- A Sourcebook. Chicago: Aldine.
Owen, David 1984 "Testing, Testing," The New Republic (June 4): 12-14.
Peters, Thomas J. and Robert H. Waterman Jr. 1982 In Search of Excellence: Lessons from America's Best Run Companies. New York: Harper & Row.
Provine, Doris Marie 1986 Judging Credentials, Nonlawyer Judges and the Politics of Professionalism. Chicago: University of Chicago Press.
Quinn, James B. and Christopher E. Gagnon 1986 "Will Services Follow Manuifacturing into Decline?" Harvard Business Review 64: 95-103.
Robinson, Chris 1988 "Auditors Must Move Fast to Save Their Image," Financial Post (February 15).
Rogers, David E. 1988 "Clinical Education and the Doctor of Tomorrow: An Agenda for Action," Proceedings of the Josiah Macy, Jr. Foundation National Seminar on Medical Education.
Shanker, Albert 1988 "Teachers Grade `Reformers'" (Editoral Advertisement). The New York Times (May 22).
1990 "Cavazos Unveils the Wall Chart" (Editorial Advertisement). The New York Times (May 13).
Singer, Benjamin D. 1989 "The Criterial Crisis of the Academic World," Sociological Inquiry 59: 127-43.
1994 "Standards from a Distance." Pp. 253-62 in Sociology of Education in Canada. Toronto: Copp Clark Longman.
Smith, M. Brewster 1994 "Selfhood At Risk," American Psychologist 49: 405-16.
Staley, Edgcumbe 1967 The Guilds of Florence. New York: Benjamin Blom.
Strehlow, Nancy A. 1981 "Evaluating `Competency' Criteria': Toward a Uniform Standard of Lawyer Performance," Washington University Law Quarterly 59: 1019-46.
Sullivan, Ronald 1987 "Cuomo Panel Supports Plan for Doctor Competency Reviews," The New York Times (June 28).
Sydenham, P.H. 1979 Measuring Instruments: Tools of Knowledge and Control. Stevanage, UK: Peter Peregrinus.
Tollchin, Martin 1989 "Concern Over the Costs of Malpractice Liability," The New York Times (November 3).
Tuchman, Barbara 1982 Practicing History. New York: Ballantine Books.
1987 "A Nation in Decline?" The New York Times Magazine (September): 52-58+.
Van Harrison, R. 1986 "Medical Malpractice Claims," ISR Newsletter (Spring-Summer): 3.
Wayne, Leslie 1989 "Where Were the Accountants," The New York Times (March 12).
Webb, John J. 1970 The Guilds of Dublin. Port Washington, NY: Kennikat Press.
Weizenbaum, Joseph 1976 Computer Power and Human Reason. San Fransisco: W.H. Freeman.
"Who's Best ... A Short Bibliography" 1990 Newsletter of the Consumer and Patient.
Williams, Robin M. 1956 American Society. New York: Alfred A. Knopf.
Wright, John and Linda Sunshine 1995
The Best Hospitals in America. New York: Gale Research.
Editor's Note
Back
to the Recent
Issues Page
Back to CJS Online