You don't say what type of cell, I remember my CSE physics exam, being asked about a Daniel Cell, and I had never heard of one, seems used to calibrate voltmeters.
One looks at a battery to see just Alkaline, Zinc, Zinc Chloride, Ni-Mh, Nickel-Cadmium or Li-ion and that is just the ones found on my desk. The declared voltage varies from 1.2 to 1.5 volt.
I have studied Lead Acid, and can say how to test these, but with other types it gets more complex, and of course state of charge matters.
So in the main test no load, and 1.3 for rechargeable and 1.5 for primary cells is seen as in the main the pass mark. Just tested a CR 2016 at 3.18 volts a Lithium cell. Today we find more and more lamps using the button cells.