I think the key to this will be establishing the curtilage of the adjoining listed building. As I said before it may not be where you think it is.
If you are sure you are not within the curtilage you should be OK. It doesn't affect your PD rights but it will determine if you need listed building consent which could still derail your proposals.
If you are sure you are not within the curtilage you should be OK. It doesn't affect your PD rights but it will determine if you need listed building consent which could still derail your proposals.