It sounds like you are on friendly terms with your neighbour and this is probably the best way to leave it.
IMHO - as they must have taken the chimney stack down at the time you mentioned when doing extension, what if you had had an open fire at that time. smoke would have come through then. It's all well and good taking down internal stacks etc, but no one thinks about the remaining structure either above or next door.
Hopefully the remaining chimney stack his side from the loft floor up, is adequately supported on an RSJ. Where supported I would look there on his side to see if there is a breach. It may be where the ends of the supports are punched into your party walll!
These old chimney's were never well built, they should have been 'parged' with render all the way up, but probably never were. I expect the two chimneys always leaked into one another, but it didn't matter when they were both being used, drawing air up and out the top.
They have removed a certain amount of fire protection between your party wall, from the loft floor down. You perhaps only have a half brick wall between your fire flue and his living space.
It's not worth saying it's his problem as they compromised the shared flue wall, so it may be time to bite the bullet and install a twin wall flue. You probably will have to take out the fire again, as you will have to form some sort of gather at the start of the flue above the fire.