Data For Research
beta

Maximum Likelihood from Incomplete Data via the EM Algorithm

  • A. P. Dempster, N. M. Laird, D. B. Rubin
  • Journal of the Royal Statistical Society. Series B (Methodological), Vol. 39, No. 1 (1977), pp. 1-38

  • Published by: Blackwell Publishing;Royal Statistical Society
  • Stable URL: http://www.jstor.org/stable/2984875
  • Abstract: A broadly applicable algorithm for computing maximum likelihood estimates from incomplete data is presented at various levels of generality. Theory showing the monotone behaviour of the likelihood and convergence of the algorithm is derived. Many examples are sketched, including missing value situations, applications to grouped, censored or truncated data, finite mixture models, variance component estimation, hyperparameter estimation, iteratively reweighted least squares and factor analysis.
  • Subjects: statistics
  • Keyterms: algorithm, likelihood, miss, maximum, incomplete, convergence, exponential, parameter, theorem, statistic, converge, eigenvalue, sufficient, sundberg, iteration, complete, estimation, observe, square, maximize, estimate, denote, bound, general, lemma
  • CiteRank: 10
  • Times cited by articles in JSTOR: 1047

Top Words
Word Count
the1561
of954
and543
in497
a460
to426
is385
for264
that259
i227
by181
are179
be178
algorithm177
we171
Top Bigrams
Bigram Count
### ###1496
### #396
# ###374
of the277
### and199
the ###189
### i141
and ###131
i ###126
# the126
### the123
of ###121
the em118
in the106
em algorithm103
Top Trigrams
Trigram Count
### ### ###787
### ### #153
### i ###105
# ### ###98
the em algorithm97
i ### ###78
### ### i64
# ### #61
### and ###60
### ## ###50
et al #47
of the em46
### ### ##42
### ### and40
dempster et al36
Top Quadgrams
Quadgram Count
### ### ### ###421
### ### ### #108
# ### ### ###69
### i ### ###65
### ### i ###53
i ### ### ###44
of the em algorithm40
### ### ### i38
dempster et al #36
### ### ## ###26
## ### ### ###22
the em algorithm #22
### ## ### ###21
### ### ### ##21
# maximum likelihoodfrom incomplete19
Top Keyterms
Keyterm Weight
algorithm1.0
likelihood0.369
miss0.296
maximum0.289
incomplete0.253
convergence0.246
exponential0.217
parameter0.173
theorem0.159
statistic0.154
converge0.15
eigenvalue0.15
sufficient0.132
sundberg0.131
iteration0.13

References

ANDREWS, D. F., BICKEL, P. J., HAMPEL, F., HUBER, P. J., ROGERS, W. H. and TuKEY, J. W. (1972). Robust Estimates of Location. Princeton, N.J.: Princeton University Press.

BAUM, L. E. (1971). An inequality and associated maximization technique in statistical estimation for probabilistic functions of Markov processes. In Inequalities, III: Proceedings of a Symposium. (Shisha, Qved ed.). New York: Academic Press.

BAUM, L. E. and EAGON, J. A. (1967). An inequality with applications to statistical estimation for proba- bilistic functions of Markov processes and to a model for ecology. Bull. Amer. Math. Soc., 73, 360-363.

BAUM, L. E., PETRIE, T., SOULES, G. and WEIss, N. (1970). A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Ann. Math. Statists.41, 164-171.

BEALE, E. M. L. and LITTLE, R. J. A. (1975). Missing values in multivariate analysis. J. R. Statist. Soc., B, 37, 129-145.

BLIGHT, B. J. N. (1970). Estimation from a censored sample for the exponential family. Biometrika, 57, 389-395.

BROWN, M. L. (1974). Identification of the sources of significance in two-way tables. Appl. Statist., 23, 405-413.

CARTER, W. H., JR and MYERS, R. H. (1973). Maximum likelihood estimation from linear combinations of discrete probability functions. J. Amer. Statist. Assoc., 68, 203-206.

CEPPELLINI, R., SINISCALCO, M. and SMITH, C. A. B. (1955). The estimation of gene frequencies in a random- , mating population. Ann. Hum. Genet., 20, 97-115.

CHEN, T. and FIENBERG, S. (1976). The analysis of contingency tables with incompletely classified data. Biometrics, 32, 133-144.

CORBEIL, R. R. and SEARLE, S. R. (1976). Restricted maximum likelihood (REML) estimation of variance components in the mixed model. Technometrics, 18, 31-38.

DAY, N. E. (1969). Estimating the components of a mixture of normal distributions. Biometrika, 56, 463-474.

DEMPSTER, A. P. (1972). Covariance selection. Biometrics, 28, 157-175.

EFRON, B. (1967). The two-sample problem with censored data. Proc. 5th Berkeley Symposium on Math. Statist. and Prob., 4, 831-853.

EFRON, B. and MORRIs, C. (1975). Data analysis using Stein's estimator and its generalizations. J. Amer. Statist. Assoc., 70, 311-319.

GOOD, I. J. (1965) The Estimation of Probabilities: An Essay on Modern Bayesian Methods. Cambridge, Mass.: M.I.T. Press.

- (1956). On the estimation of small frequencies in contingency tables. J. R. Statist. Soc., B, 18, 113-124.

GRUNDY, P. M. (1952). The fitting of grouped truncated and grouped censored normal distributions. Biometrika, 39, 252-259.

HABERMAN, S. J. (1976). Iterative scaling procedures for log-linear models for frequency tables derived by indirect observation. Proc. Amer. Statist. Assoc. (Statist. Comp. Sect. 1975), pp. 45-50.

HARTLEY, H. 0. (1958). Maximum likelihood estimation from incomplete data. Biometrics, 14, 174-194.

HARTLEY, H. 0. and HOCKING, R. R. (1971). The analysis of incomplete data. Biometrics, 27, 783-808.

HARTLEY, H. 0. and RAo, J. N. K. (1967). Maximum likelihood estimation for the mixed analysis of variance model. Biometrika, 54, 93-108.

HARVILLE, D. A. (1977). Maximum likelihood approaches to variance component estimation and to related problems. J. Amer. Statist. Assoc., 72, to appear.

HASSELBLAD, V. (1966). Estimation of parameters for a mixture of normal distributions. Technometrics, 8, 431-444.

(1969). Estimation of finite mixtures of distributions from the exponential family. J. Amer. Statist. Assoc., 64, 1459-1471.

HEALY, M. and WESTMACOTT, M. (1956). Missing values in experiments analysed on automatic computers. Appl. Statist.5, 203-206.

HOSMER, D. W. JR (1973). On the MLE of the parameters of a mixture of two normal distributions when the sample size is small. Comm. Statist., 1, 217-227.

(1973). A comparison of iterative maximum likelihood estimates of the parameters of a mixture of two normal distributions under three different types of sample. Biometrics, 29, 761-770.

HUBER, P. J. (1964). Robust estimation of a location parameter. Ann. Math. Statist., 35, 73-101.

IRWIN, J. 0. (1959). On the estimation of the mean of a Poisson distribution with the zero class missing. Biometrics, 15, 324-326.

(1963). The place of mathematics in medical and biological statistics. J. R. Statist. Soc., A, 126, 1-45.

JÖRESKOG, K. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34, 183-202.

MCKENDRICK, A. G. (1926). Applications of mathematics to medical problems. Proc. Edin. Math. Soc., 44, 98-130.

MANTEL, N. and GREENHOUSE, S. W. (1967). Note: Equivalence of maximum likelihood and the method of moments in probit analysis. Biometrics, 23, 154-157.

MARITZ, J. S. (1970). Empirical Bayes Methods. London: Methuen.

MARTIN, J. K. and McDONALD, R. P. (1975). Bayesian estimation in unrestricted factor analysis: a treatment for Heywood cases. Psychometrika, 40, 505-517.

MOSTELLER, F. and WALLACE, D. L. (1964). Inference and Disputed Authorship: The Federalist. Reading, Mass.: Addison-Wesley.

ORCHARD, T. and WOODBURY, M. A. (1972). A missing information principle: theory and applications. Proc. 6th Berkeley Symposium on Math. Statist. and Prob.1, 697-715.

PATTERSON, H. D. and THOMPSON, R. (1971). Recovery of inter-block information when block sizes are unequal. Biometrika, 58, 545-554.

RAIFFA, H. and SCHLAIFER, R. (1961). Applied Statistical Decision Theory. Cambridge, Mass.: Harvard Business School.

RAO, C. R. (1965). Linear Statistical Inference and its Applications. New York: Wiley.

RUBIN, D. B. (1974). Characterizing the estimation of parameters in incomplete-data problems. J. Amer. Statist. Assoc., 69, 467-474.

- (1976). Inference and missing data. Biometrika, 63, 581-592.

SUNDBERG, R. (1974). Maximum likelihood theory for incomplete data from an exponential family. Scand. J. Statist., 1, 49-58.

(1976). An iterative method for solution of the likelihood equations for incomplete data from exponential families. Comm. Statist.—Simula. Computa., B5(1), 55-64.

TURNBULL, B. W. (1974). Nonparametric estimation of a survivorship function with doubly censored data. J. Amer. Statist. Assoc., 69, 169-173.

(1976). The empirical distribution function with arbitrarily grouped, censored and truncated data. J. R. Statist. Soc., B, 38, 290-295.

WOLFE, J. H. (1970). Pattern clustering by multivariate mixture analysis. Multivariate Behavioral Research, 5, 329-350.

WOODBURY, M. A. (1971). Discussion of paper by Hartley and Hocking. Biometrics, 27, 808-817.

BARGMANN, R. (1957). A study of independence and dependence in multivariate normal analysis. Mimeo Series No. 186, University of North Carolina.

BATSCHELET, E. (1960). Uber eine Kontingenztagel mit fehlenden Daten. Biometr. Zeitschr., 2, 236-243.

BISHOP, Y. M. M., FIENBERG, S. E. and HOLLAND, P. W. (1975). Discrete Multivariate Analysis: Theory and Practice. Cambridge, Mass.: M.I.T. Press.

DEMPSTER, A. P. and MONAJEMI, A. (1976). An algorithmic approach to estimating variances. Research Report S-42, Dept of Statistics, Harvard University.

DRAPER, N. R. and GUTTMAN, I. (1968). Some Bayesian stratified two-phase sampling results. Biometrika, 55, 131-140 and 587-588.

EDWARDS, A. W. F. (1970). Estimation of the branch points of a branching diffusion process (with Discus- sion). J. R. Statist. Soc. B, 32, 155-174.

FEDOROV, V. V. (1972). Theory of Optimal Experiments (E. M. Klimko and W. J. Studden, eds and trans- lators). New York: Academic Press.

FELLMAN, J. (1974). On the allocation of linear observations. Commentationes Phys.-Math., 44, Nos. 2-3.

FISHER, R. A. (1925). Theory of statistical estimation. Proc. Camb. Phil. Soc., 22, 700-725.

GEPPERT, M. P. (1961). Erwartungstreue plausibelste Schutzen aus dreieckig gestutzen Kontingenstafeln. Biometr. Zeitschr., 3, 54-67.

GOODMAN, L. A. (1968). The analysis of cross-classified data. Independence, quasi-independence and interaction in contingency tables with or without missing entries. J. Amer. Statist. Ass., 63, 1091-1131.

*- (1974). Exploratory latent-structure analysis using both identifiable and unidentifiable models. Biometrika, 61, 215-231.

GUTTMAN, I. (1971). A remark on the optimal regression designs with previous observations of Covey- Crump and Silvey. Biometrika, 58, 683-685.

HABERMAN, S. J. (1971). Tables based on imperfect observation. Invited paper at the 1971 ENAR meeting, Pennsylvania State University.

- (1974). Loglinear models for frequency tables derived by indirect observation: maximum likelihood equations. Ann. Statist., 2, 911-924.

HEMMERLE, W. J. (1974). Nonorthogonal analysis of variance using iterative improvement and balanced residuals. J. Amer. Statist. Ass., 69, 772-778.

HEMMERLE, W. J. and HARTLEY, H. 0. (1973). Computing maximum likelihood estimates for the mixed A.O.V. model using the W transformation. Technometrics, 15, 819-831.

HEMMERLE, W. J. and LORENS, J. 0. (1976). Improved algorithm for the W-transform in variance component estimation. Technometrics, 18, 207-212.

HOWE, W. G. (1955). Some contributions to factor analysis. Report ORNL 1919, Oak Ridge National Laboratory.

LAIRD, N. M. (1975). Log-linear models with random parameters. Ph.D. Thesis, Harvard University.

-- (1976). Nonparametric maximum-likelihood estimation of a distribution function with mixtures of distributions. Technical Report S-47, NS-338, Dept of Statistics, Harvard University.

LAWLEY, D. N. and MAXWELL, A. E. (1971). Factor Analysis as a Statistical Method (2nd edn). London: Butterworth.

LITTLE, R. J. A. (1974). Missing values in multivariate statistical analysis. Ph.D. Thesis, University of London.

MCCLACHLAN, G. J. (1975). Iterative reclassification procedure for constructing an asymptotically optimal rule of allocation in discriminant analysis. J. Amer. Statist. Ass., 70, 365-369.

MORGAN, B. J. T. and TITTERINGTON, D. M. (1977). A comparison of iterative methods for obtaining maximum-likelihood estimates in contingency tables with a missing diagonal. Biometrika, 64, (in press).

PEARCE, S. C. (1965). Biological Statistics: an Introduction. New York: McGraw-Hill.

PEARCE, S. C. and JEFFERS, J. N. R. (1971). Block designs and missing data. J. R. Statist. Soc. B, 33, 131-136.

PETERSON, A. V. (1975). Nonparametric estimation in the competing risks problem. Ph.D. Thesis, Stanford University.

PREECE, D. A. (1971). Iterative procedures for missing values in experiments. Technometrics, 13, 743-753.

RAO, C. R. (1955). Estimation and tests of significance in factor analysis. Psychometrika, 20, 93.

RUBIN, D. R. (1972). A non-iterative algorithm for least squares estimation of missing values in any analysis of variance design. Appl. Statist., 21, 136-141.

SILVEY, S. D., TITTERINGTON, D. M. and TORSNEY, B. (1976). An algorithm for D-optimal designs on a finite space. Report available from the authors.

SMITH, C. A. B. (1969). Biomathematics, Vol. 2. London: Griffin.

SNEDECOR, G. W. and COCHRAN, W. G. (1967). Statistical Methods, 6th edn. Ames, Iowa: Iowa State University Press.

THOMPSON, E. A. (1975). Human Evolutionary Trees. Cambridge: Cambridge University Press.

TSIATIS, A. (1975). A nonidentifiability aspect of the problem of competing risks. Proc. Nat. Acad. Sci. USA, 71, 20-22.

TUKEY, J. W. (1962). The future of data analysis. Ann. Math. Statist., 33, 1-67.

TURNBULL, B. W. (1976). The empirical distribution function with arbitrarily grouped censored and trun- cated data. J. R. Statist. Soc. B, 38, 290-295.

TURNBULL, B. W. and WEISs, L. (1976). A likelihood ratio statistic for testing goodness of fit with randomly censored data. Technical Report No. 307, School of Operations Research, Cornell University.

Selected: 6,428,186

JSTOR is part of ITHAKA, a not-for-profit organization helping the academic community use digital technologies to preserve the scholarly record and to advance research and teaching in sustainable ways.
©2000-2010 ITHAKA. All Rights Reserved. JSTOR®, the JSTOR logo, and ITHAKA® are registered trademarks of ITHAKA.