One of the central, unresolved controversies in biology concerns the distribution of primitive versus advanced characters at different stages of vertebrate development. This controversy has major implications for evolutionary developmental biology and phylogenetics. Ernst Haeckel addressed the issue with his Biogenetic Law, and his embryo drawings functioned as supporting data. We re-examine Haeckel's work and its significance for modern efforts to develop a rigorous comparative framework for developmental studies. Haeckel's comparative embryology was evolutionary but non-quantitative. It was based on developmental sequences, and treated heterochrony as a sequence change. It is not always clear whether he believed in recapitulation of single characters or entire stages. The Biogenetic Law is supported by several recent studies -- if applied to single characters only. Haeckel's important but overlooked alphabetical analogy of evolution and development is an advance on von Baer. Haeckel recognized the evolutionary diversity in early embryonic stages, in line with modern thinking. He did not necessarily advocate the strict form of recapitulation and terminal addition commonly attributed to him. Haeckel's much-criticized embryo drawings are important as phylogenetic hypotheses, teaching aids, and evidence for evolution. While some criticisms of the drawings are legitimate, others are more tendentious. In opposition to Haeckel and his embryo drawings, Wilhelm His made major advances towards developing a quantitative comparative embryology based on morphometrics. Unfortunately His's work in this area is largely forgotten. Despite his obvious flaws, Haeckel can be seen as the father of a sequence-based phylogenetic embryology.