Yes, it's your home and so affects everything you do. But, you will get it resolved with a new builder and in a year or so all will be done, completed, and you will look back at the challenges from your new improved space, and it will seem far better.
Trust me, I'm 2 1/2 years into a self build, and it feels like it will go on forever sometimes and there are issues, mistakes, specs missing, changes. You get to certain points where somethings been achieved, and you feel a lot better.
Best bet is get someone new on side to get the work progressing again for you.
Moving a steel and relaying some drains is really nothing much to a decent builder... it seems daunting for you but they will knock it all together within a few days, job done.