I'm not surely that I fully understand the logistics of what your plan. If the fan usually comes on with the light (and with a timed over-run), at night you would have to find a way of turning off the isolator before turning on the lights (otherwise your noisy fan would come on, at least briefly, and wake people up) - and you would then have to remember to turn the isolator back on again (after turning off the light) in order for it to return to light-switch-activated functionality. Sounds a bit complicated (impractical?).A switched fan (which otherwise operates on a timed overrun) that can be independently turned off will reduce noise, particularly at night (there is a toilet in the same room).
Kind Regards, John