one of our children has a granny annexe came with the property and atm used as an office and this has a large bedsitter, kitchen shower and bathroom own access down the side and rear. They can move the office to to the dining room or spare bedroom but they too fear what I partially feared, IE you don't really know the person/s coming into your home and therefore no but my family is willing to volunteer and buy stuff for these groups if required locally.
I think these are legitimate concerns, I don’t have the space, if I did I’d be keen to do it but would be very concerned, perhaps unreasonably.