As you have a right of way over the drive, I would argue that your "boundary" is the centre of the right of way....
That is the dumbest argument I have ever heard on the subject.
Right of use does not make you the owner; you can't terminate the flue there any more than you can build your shed on it.
It's along the same line as any right of use.
A lifetime interest in property bequeathed to you in a will means you can live there, but you don't get a penny from the proceeds of the sale, nor can you convert it into pub.