Because the controller has a built-in wireless receiver, it is difficult to know how to bypass it.
I haven't been able to find any technical information on the receiver itself, other than it being a:
WORCESTER 87161561170 PROGRAMMABLE RECEIVER
View attachment 325539
There appear to be only three wires on the back, so it's possible that it is simply +V, gnd and switch, but I wouldn't like to bet on that without testing.
@Badger1966 , can you remove the module, and pull out the plug and see if the Nest then has control?