The Short Version
The evidence strongly supports a real physiological link. Very low body fat is associated with lower estrogen because adipose tissue helps produce estrogen and because low fat often accompanies low energy availability, which can suppress ovarian hormone production. The main caveat is that there is no single body-fat percentage that defines "insufficient" for every woman.