I am seeing a lot of confusion around dark energy and the cosmological constant. What are they? Is gravity always attractive? Or is there such a thing as negative gravity or anti-gravity?

First, what is gravity? Einstein taught us that it is the curvature of space. Or as famous relativist John Wheeler wrote “Matter tells space how to curve, and curved space tells matter how to move”.

Dark Energy has been recognized with the Nobel Prize for Physics, so its reality is accepted. There were two teams racing against one another and they found the same result in 1998: the expansion of the universe is accelerating!

Normally one would have thought it would be slowing down due to the matter within; both ordinary and dark matter would work to slow the expansion. But this is not observed for distant galaxies. One looks at a certain type of supernova that always has a certain mass and thus the same absolute luminosity. So the apparent brightness can be used to determine the luminosity distance. This is compared with the redshift that provides the velocity of recession or velocity-determined distance in accordance with Hubble’s law.

A comparison of the two types of distance measures, particularly for large distances, shows the unexpected acceleration. The most natural explanation is a dark energy component equal to twice the matter component, and that matter component would include any dark matter. Now do not confuse dark energy with dark matter. The latter contributes to gravity in the normal way in proportion to its mass. Like ordinary matter it appears to be non-relativistic and without pressure.

Einstein presaged dark energy when he added the cosmological constant term to his equations of general relativity in 1917. He was trying to build a static universe. It turns out that such a model is unstable, and he later called his insertion of the cosmological constant a blunder. A glorious blunder it was, as we learned eight decades later!

Here is the equation:

The cosmological constant is represented by the Λ term, and interestingly it is usually written on the left hand side with the metric terms, not on the right hand side with the stress-energy (and pressure and mass) tensor T.

If we move it to the right hand side and express as an energy density, the term looks like this:

with  as the vacuum energy density or dark energy, and appearing on the right it also takes a negative sign. So this is a suggestion as to why it is repulsive.

The type of dark energy observed in our current universe can be fit with the simple cosmological constant model and it is found to be positive. So if you move to the other side of the equation, it enters negatively.

Now let us look at dark energy more generally. It satisfies an equation of state defined by the relationship of pressure to density, with P as pressure and ρ denoting density:

Matter, whether ordinary or dark, is to first order pressureless for our purposes, quantified by its rest mass, and thus takes w = 0. Radiation it turns out has w = 1/3. The dark energy has a negative w, which is why you have heard the phrase ‘negative pressure’. The simplest case is w = -1, which the cosmological constant, a uniform energy density independent of location and age of the universe. Alternative models of dark energy known as quintessence can have a larger w, but it must be less than -1/3.

Credit: http://www.scholarpedia.org/article/Cosmological_constant

Why less than -1/3? Well the equations of general relativity as a set of nonlinear differential equations are usually notoriously difficult to solve, and do not admit of analytical solutions. But our universe appears to be highly homogeneous and isotropic, so one can use a simple FLRW spherical metric, and in this case one end up with the two Friedmann equations (simplified by setting c =1).

This is for a (k = 0) flat on large scales universe as observed. Here is the acceleration (second time derivative) of the scale factor a. So if is positive, the expansion of the universe is speeding up.

The term can be rewritten using the dark energy density relation above. Now the equation needs to account for both matter (which is pressureless, whether it is ordinary or dark matter) and dark energy. Again the radiation term is negligible at present, by four orders of magnitude. So we end up with:

Now the magic here was in the 3 before the p. The pressure gets 3 times the weighting in the stress-energy tensor T. Why, because energy density is just there as a scalar, but pressure must be accounted for in each of the 3 spatial dimensions. And since p for dark energy is negative and equal to the dark energy density (times the square of the speed of light), then

is always negative for the dark energy terms, provided w < -1/3. That unusual behavior is why we call it ‘dark energy’.

Overall it is a battle between matter and dark energy density on the one side, and dark energy pressure (being negative and working oppositely to how we ordinarily think of gravity) on the other. The matter contribution gets weaker over time, since as the universe expands the matter becomes less dense by a relative factor of , that is the matter was on average denser in the past by the cube of one plus the redshift for that era.

Dark energy eventually wins out, because it, unlike matter does not thin out with the expansion. Every cubic centimeter of space, including newly created space with the expansion has its own dark energy, generally attributed to the vacuum. Due to the quantum uncertainty (Heisenberg) principle, even the vacuum has fields and non zero energy.

Now the actual observations at present for our universe show, in units of the critical density that

and thus

And the sum of them all is around -1, just coincidentally. Since there is a minus sign in front of the whole thing, the acceleration of the universe is positive. This is all gravity, it is just that some terms take the opposite side. The idea that gravity can only be attractive is not correct.

If we go back in time, say to the epoch when matter still dominated with  and  , then the total including pressure would be 2/3 +1/3 – 1, or 0.

That would be the epoch when the universe changed from decelerating to accelerating, as dark energy came to dominate. With our present cosmological parameters, it corresponds to a redshift of , and almost 6 billion years ago.

Image: NASA/STScI, public domain