They are correct.However WDP say that they only took over responsibility from SWEB up to and including the meter, but no further
Long ago, the 'electricity board' for the area was responsible for everything including the meter and in some cases additional devices such as that VOELCB.
Today the electricity distributor only covers items before the meter. Your energy supplier is responsible for the meter.
Other devices are yours, even though they may not have been originally.
You might be able to convince the energy supplier to replace it with an isolator.
Or find an electrician who will remove it.