You are of course wrong, because of dilution of the air in the bathroom, moist air will be drawn out along with it. You seem to have an problem understanding that air does mix, and mix very easily.
I'm afraid I still don't understand your argument.
The issue is not about what air is moist and which is not but, rather, about what air is heated and what air is not heated.
Start with the situation in which the bathroom (in practice, probably the whole house) is full of 'heated' air. It doesn't matter whether you say that some is moist and some is not or that (because of mixing) it all is moist (with an 'average' level of moisture, after the mixing) - the important point is that, moist or not,
it is all heated air.
Now turn on the extractor fan. Since the only air in the room (probably in the house) is heated air (whether moist or not), all of the air extracted from the room is necessarily heated air - and that can only be replaced by' unheated' air entering the house from outside (whether entering into the bathroom or other places in the house).
Hence initially every X litres of air extracted will be 'heated air' and will necessarily be replaced by X litres of unheated air entering the house from outside.
Even subsequently, when that unheated 'replacement air' mixes with the heated air in the house (thereby reducing temp within the room/house) when that 'partially unheated' mixture is extracted, it will be replaced with 'totally unheated' air, thereby reducing the temp in the house even further. In the absence of a heating system (see below **) if the extractor continued to run that process would theoretically continue indefinitely until, eventually, the entirety of the house's heated air had been replaced with unheated air (i.e. inside temp became equal to outside temp).
[ ** I am, of course, ignoring the fact that there will usually be a heating system in operation, which will try (and probably succeed) prevent the fall in temp within the room/house - but that is the very 'wasted energy' which this whole discussion is about. In other words, what one would normally expect to see as a result of turning on an extractor would not be a fall in room/house temp but, rather, an increase in energy usage to maintain the desired temp ]
What is so difficult to understand about that?
Kind Regards, John