Deming, W. Edwards

W. Edwards DEMING

b. 14 October 1900 - d. 20 December 1993

Summary. Long recognized as a leading proponent of statistical sampling and approaches to quality improvement, Deming led major efforts to bring statistical approaches and methods to bear on problems in government, science and industry.

William Edwards Deming was born in Sioux City, Iowa, USA, and died at the age of 93 in Washington, D.C. Deming was active to the end of his life; he spoke at the Joint Statistical Meetings in San Francisco in the summer of 1993, and he had presented a four-day seminar on quality management as recently as December 7-10 in the Los Angeles area. At the time of his death, Deming was arguably the most famous living statistician.

Deming studied at the Universities of Wyoming (B.S. 1921) and Colorado (M.S. in Mathematics and Physics, 1924) before going to Yale University, where he received his doctorate in Physics in 1928. From 1927 through 1939 he held an appointment as Mathematical Physicist in the Bureau of Chemistry and Soils of the U. S. Department of Agriculture (USDA). Deming was instrumental in arranging lecture series at the USDA. Fisher spoke in 1936, but the more influential series was one presented in April 1937 by Jerzy Neyman. Deming arranged the publication of Neyman's lectures as a mimeographed book, Lectures and Conferences on Mathematical Statistics (USDA, 1938; 2nd ed. 1952), one of the earliest and most widely read American textbooks on mathematical statistics. Deming also lectured on least squares, and he assisted Shewhart in the 1939 publication of his lectures as well.

Subsequently, from 1939 to 1945, he was Head Mathematician and Advisor in Sampling to the U.S. Bureau of Census, where among his innovations was the application of Shewhart's quality control ideas to the Census's coding and cardpunching operations, and the introduction (with F. Stephan) of the Deming-Stephan algorithm for the iterative proportional fitting of contingency tables. During 1942-45, at the instigation of W. Allen Wallis, Deming was actively involved with Holbrook Working in arranging a highly successful series of short courses on the use of statistics in support of the war effort. In 1945, Deming moved to a position as Advisor in Sampling in the Bureau of the Budget.

Deming's great international fame arose from his work on the re-establishment of Japanese industry after the Second World War. From 1947-50, Deming had been assigned to General Douglas MacArthur's Supreme Command of the Allied Powers in Tokyo, as Advisor in Sampling Techniques, and he remained to consult with the Union of Japanese Scientists and Engineers (JUSE) for two years after that. It was a most agreeable arrangement for both sides. Deming had come upon Shewhart's methods of quality control in 1928 and had written and spoken about them on many occasions; in Japan he found an eager audience. Post-war Japan by 1950 had a reputation for such poor quality products that it eagerly embraced Deming and statistical control methods, both for the substance of the methods and the symbolism of Deming as an icon of American values. In 1950 JUSE established the Deming Prize for excellence in quality control, and the prize has been awarded annually since 1951. Deming's contributions to quality improvement in Japanese industry became the "stuff" of legends and in the later decades of his life the story was retold in many ways by many different authors (e.g., see Mann, 1987).

Thereafter, although he held various adjunct teaching positions (including at New York University (NYU) and at Columbia University) he worked out of his Washington, D.C. home as a private consultant. He conducted frequent multi-day seminars in the U.S. and overseas on managerial methods. He insisted upon the involvement of top-level management in his programs, and he attacked the use of production quotas and performance criteria, emphasizing cooperative problem solving. His approaches to management were reduced to a 14-point philosophy (e.g., see Deming, 1986) and promulgated by many others in the United States and abroad.

Deming's great gifts were enormous energy and an early keen sense of the broad applicability of statistical methods in government, science, and industry. The ideas he is associated with came from many sources: quality control from Shewhart, statistical methods from Fisher and Neyman, sampling techniques from Stephan and Hansen. But the mixture of these and the forceful way he melded them to a philosophy of management were his own. He also had a great interest in the history of statistics, and in 1940 he arranged for the reprinting of two original papers by Thomas Bayes, adding his own commentary to Bayes's paper on divergent series.

Deming married Agnes Bell in 1922, and she died eight years later. In 1932 he married Lola E. Shupe, who had been a co-author on a number of physics articles with Deming and later helped him with calculations for his research. She died in 1986.

Deming was awarded the President's National Medal of Technology in 1987, but characteristically Deming was unwilling to cancel a previously arranged managerial seminar in Detroit; the medal was received from President Reagan on his behalf by his two daughters at a White House Rose Garden ceremony. Deming was the President of the Institute of Mathematical Statistics in 1945.

Deming on Least Squares and the Adjustment of Data

In 1934, Deming published, jointly with the UC Berkeley Physicist Raymond T. Birge, an influential paper "On the Statistical Theory of Errors." This review paper brought the methods of Fisher, Neyman and Pearson, and Jeffreys to a broad audience of American physical Scientists. It was reprinted with additional notes in 1937 by the Graduate School of the USDA, and it served effectively as a textbook for over a decade. In correspondence with Deming, Fisher was generally quite complimentary, although in a 1940 letter to Maurice Fréchet about Deming and Birge, Fisher complained that he found "these writers very often obscure." This material also made its way into Deming's mimeographed lecture notes on least squares which first appeared in 1938, which he later turned into the 1943 book on the adjustment of statistical data.

Deming worked with Frederick Stephan on what was an crucial problem for the Bureau of the Census, how to adjust a sample table of counts to have the same marginal totals that were available for the population. In 1940, the Bureau implemented the first sample long form and needed to have the sample data consistent with that from the complete census long form for selected marginals. Their answer, which they published in a 1940 paper, was to apply least squares to minimize the weighted sum of squared discrepancies between the adjusted frequencies and the sample frequencies, subject to the marginal contraints using Lagrange multipliers. They weighted the individual squared discrepancies by the inverse of the sample cell counts (thus using a chi-square-like metric). Their method could be viewed as a form of post-sratification which generalized ratio adjustment, an approach which handles one set of margins only. This method generalized to three and higher dimensions where adjustments were made for one way and even two-way totals, but the solution of the normal equations proved computationally daunting. Thus Deming and Stephan suggested an approximation which they described as the method of iterative proportions, which successively rescaled the sample data by a proportional adjustment to have the correct margins and iterated until convergence. This approximate method proved workable and soon became the method of choice at the Census Bureau under the label "raking." Deming included a chapter on this method in his 1943 book.

Stephan, after some further interaction with Deming, published a second paper on the topic in 1942, in which he proposed an alternative iterative method that actually produced the least squares solution for this problem and noted that the method of iterative proportions did not actually solve the least squares problem and that there was no proof of convergence for more than two dimensions. There the problem lay, until Yvonne Bishop, in work on the National Halothane Study (see Bishop, 1969), noted that the method of iterative proportions solved the loglinear model estimation problem for contingency tables. At about the same time several others found various ways to exploit the algorithm (e.g., see the uses and related references presented in Bishop, et al., 1975).

Deming on Sampling and its Applications

Deming's work on sampling went beyond that linked to the statistical adjustment of data and methods worked on while he was at the Bureau of the Census. His books on sampling (1950) and the design of business research (1960) both had considerable impact on the applications of sampling beyond government, as did his expository articles on sampling such as Deming (1968). He wrote about nonsampling errors, and he applied the notion of capture-recapture to human populations (Sekar and Deming, 1948), linking the approach to sampling and stratification. After the method had been promulgated by others at the Bureau of the Census and elsewhere, he worked with the demographer Nathan Keyfitz to extend capture-recapture ideas to multiple recaptures, but without making the link to loglinear models and his method of iterative proportions.

Beginning in the 1950s, Deming referred to himself professionally as a "Consultant in Statistical Surveys," and he consulted widely with those in government and in industry. He was among the first statisticians to testify as an expert witness in court about the results of sample surveys. He wrote on this topic as early as 1954, and he included an example drawing on this work in his 1960 book.

In 1958 Deming presented a Special Invited Address on "Principles of Professional Statistical Practice" at the Boston meeting of the Institute of Mathematical Statistics (see Deming, 1965). This was one of the first papers to appear in the statistical literature which emphasized the professional ethics associated with the practice of statistics, and much of what he wrote then and subsequently on the topic served as the foundation for codes of ethics developed both by the American Statistical Association and the International Statistical Institute.

References[edit]

Reprinted with permission from Christopher Charles Heyde and Eugene William Seneta (Editors), Statisticians of the Centuries, Springer-Verlag Inc., New York, USA.