What I need to clarify is it the voltdrop which is causing the trip or an earthing issue?
There is no way for anyone here to know.
Coule be the cable is damaged
Could be the high capacitance of the ridiculously long and undersized cable
Could be a faulty RCD
Could be wired incorrectly
Could be faulty appliances
Could be high leakage currents in the house and the summerhouse mess just adds enough to trip the circuit
Could be any combination of the above, or other things not mentioned.
Having an RCD in the house for this circuit is wrong, should be in the shed/summerhouse.
2 RCDs on the same circuit is wrong.
Adding an earth rod won't fix the problem.
4mm SWA connected to a 40A MCB is wrong
The electrician you had in was right - to install this circuit properly using the correct size cable and other materials will be ridiculously expensive and completely OTT.
Even if using 10mm 2-core, the cable alone will cost hundreds of pounds, and when rolled on a spool will weigh over 100kg - far too heavy for a single person to lift.