This is actuallyu quite simple.
If you paid his boss, or his bosses company, then there is absolutely no reason why he shopuld return to inspect and certfy for free. HAS the boss retired? Was it a Ltd Co, or self employed. If a Ltd Co, which has been dissolved, then you pretty much have run out of options. If he was a sole trader, or a partnership, then his responsibility does not end just because he has stopped working, and you could sue through the Small Claims Court.
The comment that it is the installers responsibility to register the installation, rather than the boss, I beleive is completely wrong. He may be expectected to fill in the forms or paperwork on site, but the boss or his offoce staff would do the physical registrsation, and pay any fee required. (assuming that you paid the boss)
If you paid the fitter IN HIS OWN RIGHT (i.e. you may have given him the payment for him to pass to his boss), then he can not charge you for completing the work, and you should consider the SCC route.
So, you need to establish with whom your contract is, and are you in a smoke controlled area. If you are in a SC area, is yopur appliance approved for such use? Unfortunately, your posts are heavy on misery and shot on actual facts.
If he hasn't fitted a register plate: If you paid his boss, he could argue that he was instructed to fit it in that manner, and remember he wasn't HETAS reg. then, so he could argue that he ws still "in training", and that hois waork was being supervised. In this case you would struggle to bet redress.
If it was him you were paying, then he clearly should do everything for no further charge.
Of course, if you just shoved £2K in someones hand, and got no receipt, then it is a hard lesson learned. I am a little surprised that you could research the 'net for a fitter, but not research some basic consumer protection.