What the telescope cannot tell you

For most of the twentieth century, astronomers believed planets orbiting other stars would look broadly like ours: small rocky worlds close in, gas giants far out, orbits roughly circular. It was a reasonable belief. It was also, in hindsight, mostly a description of the only planetary system anyone had ever seen. Then in 1995 we found 51 Pegasi b — a gas giant the size of Jupiter, orbiting its star closer than Mercury orbits ours, in four days. It should not have existed. Within a decade we had found dozens more like it. The instrument chooses the discovery Here is the part that gets skipped. Hot Jupiters were not the most common kind of planet. They were the kind our instruments could see. The radial velocity method detects a planet by the wobble it induces in its star, and a big planet close in produces a large, fast, obvious wobble. A small planet far out produces almost nothing. So the first exoplanet catalogue was not a census. It was a portrait of our own sensitivity curve. Astronomers knew this — the term is selection effect, and it is taught in the first week — and it still took years to fully unwind the intuitions that the early data had quietly installed. An instrument does not merely fail to see what it cannot detect. It makes the undetected thing feel rare. This is not only an astronomy problem A hospital that only records the patients who come in learns nothing about the ones who stay home. A survey conducted by telephone in the afternoon finds a population that is at home in the afternoon. A dataset of logged errors describes the errors your logging was designed to catch, which is a different thing from the errors you have. In each case the missing data is not neutral. It is absent in a patterned way, and the pattern usually points in the direction of whatever was hardest to reach — which is often whatever mattered most. What to do about it You cannot eliminate a selection effect by being careful. You can only characterise it. The useful question is never "is my data biased" — it always is — but "biased in which direction, and by roughly how much?" Astronomers answered it by modelling their own detection efficiency explicitly, then correcting the catalogue against it. When they did, the picture inverted: small planets turned out to be overwhelmingly the most common, and hot Jupiters the genuine rarity. The universe had not changed. The instrument had finally been included in the calculation. That is the discipline worth borrowing. Before asking what your measurements show, write down what they are constitutionally unable to show. Keep that list next to the results. It will be shorter than the results and more important than most of them.

Responses