Do you need a battery to test for voltage?
You do if you're using a standard digital multimeter,....
I don't think you need to restrict that to "standard" digital multimeters. I can't see how any sort of digital meter could function without a battery of some sort (or a mains supply, if it were a bench instrument) - something has to power the electronics and display!
... that's one of the reasons they're frowned upon as a means of testing for dead.
AFAIAA, the same is true of most/all of the "two pole testers" advocated for 'testing for dead', isn't it? - again, if they have any electronics or a display they must have a power source (other thgan the voltage being measured). Only truly passive (moving coil or moving iron etc.) analogue meters could measure voltage without a battery (or mains supply) - and only us oldies are likely to have one of those
Lamps (or LED equivalents) are another passive option, but they don't give a 'measurement' (LED ones may give a vague idea of voltage range).
Kind Regards, John