Forking out an extra £500+vat is probably the pragmatic solution if you and the builder have parted company.
It gets you a boiler that is certificated and therefore your house insurance is intact.
Its especially worth it if you have a good plumber that you trust. Its good they have a price set for it and are prepared to do the certfication. It also means you have somebody that will come out and help with any problems during the guarantee period.
I've heard this (in bold) a few times - I suppose it is possible a home insurance policy may refuse to pay out if the Gas boiler was directly responsible for whatever damage is caused, I've yet to hear of a single instance where this has actually happened. This is for residential owners of their own property - a rental property has a legal requirement for a Gas Safety Check.
However, if the above is a concern, then a £50 - £100 gas safety certificate is all that is required. The building regulations compliance may well be a legal requirement, but unless you are selling your property, it is never asked for. And in the majority of cases, when selling, a useless indemnity policy satisfies the lender, and a Gas Safety Check by the vendor satisfies a potential buyer (or should do, as it is far more useful giving the current state of the installation, than a Building Regs sign off from months or years ago).
This is coming from somebody who had sleepless nights when my installer couldn't issue a certificate (despite being Gas Safe registered) and getting somebody else in because I was told scare stories such as a) your warranty will be invalid (not true - as long as the benchmark document is filled in correctly), b) your home insurance will be invalid. In hindsight, I should have just had a Gas Safety inspection done.
I do agree with the latter part of your post, though. Having somebody you trust on hand, in case of a problem, is what a homeowner should be looking for.