Notch7 is abolutely correct.Active imagination there!
Yes, transgender people can be prevented from accessing single sex spaces, but not on some whim of any induivduals, nor on any general classification of spaces.
A specific space has to be specifically designated as a single sex space, with the provision of excluding transgenders entering or using the services, for genuine justifiable reasons.
Here's a guide for your perusal.