Dark energy was first introduced as a possibility as a result of the formulation of Einstein’s equations of general relativity. When he considered how the universe as a whole would behave under the general relativity description of gravity, he added a term to his equations, known as the cosmological constant. At the time the prevailing view was that the universe was static, and neither expanding nor contracting. The term was intended to balance the self-gravitational energy of the universe, and it thus acts as a repulsive force, rather than an attractive one. His basis for introduction of the cosmological constant was erroneous in two respects. The first problem is that the static solution was unstable, as if balanced on a knife edge. If you nudged it a little bit by increasing the matter density in some region slightly, that region would collapse, or if you lowered the density ever so slightly, that region would expand indefinitely. The second problem is that by 1929 Edwin Hubble had demonstrated the universe is actually expanding at a significant rate overall.
Subsequently, Einstein called the introduction of the cosmological constant his “greatest blunder”. After the realization that we live in an expanding universe, while the possibility of the cosmological constant having a non-zero value was sometimes entertained in cosmological theory, it was mostly ignored (set to zero). Over the next several decades, attention turned to better measuring the expansion rate of the universe and the inventory of matter, both ordinary matter and the dark matter, with the amount of the latter implied by long range gravitational effects seen both within galaxies and between galaxies. Was there enough matter of both types to halt the expansion? It seemed not, rather that there was only about 1/4 of the required density of matter, and that was mostly in the form of dark, not ordinary matter. Matter of either type would slow down the expansion of the universe due to its gravitational effects.
After 1980, the inflationary version of the Big Bang gained acceptance due to its ability to explain the flat topology of the universe and the homogeneity of the cosmic microwave background radiation, the relic light from the Big Bang itself. The inflationary model strongly indicated that the total energy density should be about 4 times greater than seen from the matter components alone. It is the total of energy and matter (the energy content of matter) which determines the universe’s fate, since E = mc^2.
In 1998 the astounding discovery was made that the universe’s expansion rate is accelerating! This was determined by two different teams, each of which were making measurements of distant supernovae (exploding stars). And it was confirmed by measurements of tiny fluctuations in the intensity of the microwave background radiation. The two techniques are consistent, and a third technique based on X-ray emission from clusters of galaxies, as well as a fourth technique based on very large scale measurements of relative galaxy positions, also give results consistent with the previous two techniques. The inflationary predictions are satisfied with dark energy presently three times more dominant than the rest mass energy equivalent from dark matter plus ordinary matter. Further measurements have refined our understanding of the relative strength of dark energy in comparison to dark matter and ordinary matter. The best estimates are that, today, dark energy is 74% of the universe’s total mass-energy balance.
In the cosmological constant formulation, dark energy is constant in time, while the matter density drops as the universe expands, in proportion to the cube of the scale factor. So if we consider the universe in its early days the energy contained in the dark matter would have dominated over dark energy, as the mass density would have been much greater than today. The crossover from matter dominated to dark energy dominated came after the universe was about 9 billion years old, or about 5 billion years ago. This emergence of dark energy as the dominant force, due to its nature as a repulsive property of “empty” space-time, results in an accelerating expansion of the universe, which has been called the “runaway universe”. Our universe is apparently slated to become hugely larger than its current enormous size.
Why is dark energy important then? Since five billion years ago, and on into the indefinite future, it has dominated the mass-energy content of the universe. It drives a re-acceleration of the universe. It inhibits the re-collapse (“Big Crunch”) of our entire universe or even substantial portions of the universe. Thus it naturally extends the life of the entire universe to trillions of years or much more – far beyond what would occur were the universe to be dominated by matter only and with density at the critical value or above. Dark energy thus works to maximize the available time and space for life to develop and to evolve on planets found throughout the universe.