It might be useful if people mention which ISP they are using.
If I PING this site I see that response times range from 38ms to 56ms, the Time To Live is 57ms. By way of a contrast, the TTL for Google is 114ms (with response times of 33-36ms).
A bit above my pay grade, but it could well be a server issue with some users, possibly, exceeding the TTL.