The important speed here is your ADSL sync. You can check that by your modem. Some ISP's report it (Zen do) as well as the BT BRAS profile speed (a little lower than the sync, and the maximum throughput you'll achieve)
Sync speeds are between you and your BT exchange - unless you're unlooped, it doesn't matter which ISP you're with. The sync is HUGELY dependant on your internal wiring.
Find your current sync speed. If it's an external modem it'll be on it's web configuration setting, and if internal or USB it'll be in the software somewhere. Google your modem details to find out how.
Now - visit samknows and punch in your number, you should get an estimate of your speed. These are usually fairly accurate and under-read if anything.
If your sync is FAR below your existing, from my experience the most likely cause IS your internal wiring. (Yes, the BT wiring is often at fault too, but rule out yours before calling them)
Normally you'd find your BT master socket and remove the front plate with two screws. This SHOULD also isolate all internal extensions which are wired into it. You'd then plug your modem directly into the socket in the backplate and reconnect. If your sync speed is magically better, your internal wiring is causing problems.
If you have a second master, or even an extension, coming off the grey box outside, something's awry. If it's just the master being fed off it, no problem. If that dodgy socket is fed seperate from the master, remove it or face a bill when BT discover it. You're right - it's done badly. ALL internal wiring should be after the single BT master socket and connected via it's front plate.
This is a standard BT NTE5 master socket. You'd remove the front plate and plug into the concealed socket behind to test;
If your sync speed is good, consistent, and way better than transfer speeds suggest, you can check your BRAS profile online via a BT site -
http://test.speedtester.bt.com/
to ensure it's good (it'll stay low after line noise, but automatically reset).
If that is good, then the most likely result is that your ISP is rubbish.
Talktalk /are/ rubbish, but cheap. If you want good, only two I would recommend are Zen and BEthere. Zen is more expensive but has excellent support. BE cheaper but good technically, but I haven't tested out their support personally.