Same reason why you can sit in a sauna with very hot air or pass your hand through a flame quickly without severe burns. Low density matter does not transfer heat very well. And space is especially devoid of matter.
Interesting to think that while it's not a concern to Voyager at its pokey 17km/second, a true interstellar ship traveling at some respectable fraction of C would compress the diffuse interstellar gasses enough to make them a potential hazard. You frequently see people saying stuff like "if we could accelerate to a high fraction of C you could get anywhere in the galaxy in a single lifetime", but it may not be so simple.
And that is the champion understatement of this thread!
Radiative heat transfer, roughly speaking, tries to bring the temperature of the probe to the average temperature of all the matter that it has line of sight to -- somewhere between the temperature of the sun and the temperature of the cosmic background radiation. Since the probes are far away from the sun, this average temperature is very low.
Both effects are present everywhere. On Earth, with our dense atmosphere, conductive transfer is usually the stronger effect. In space, with extremely low density, radiative heat transfer is stronger.
Imagine that there is one venomous and aggressive snake (in a cute little survival-suit) in some random spot in Antarctica. This means "the average snake in Antarctica" is ultra-dangerous.
But there's only one, and it's almost impossible for you ever to meet, so in practical terms it's still safer than Australia. :p
Temperature is a measure of the kinetic energy of a particle, so they can be both extremely hot and extremely diffuse.
Except where Voyager is, the "air" is so thin there are like a dozens zeroes on the percentage thinner it is, so the amount of heat it carries is also divided by a similar amount.
Each particle is carrying a huge amount of heat, but it gets hit by very few particles. Earth is the inverse; each particle carries a very moderate amount of heat, but you get hit by a lot of them.