Free trade agreements are usually extremely tight on which goods are included, and which are excluded, including quotas on such goods. They do not cover projects, grants, infrastructure and income generation schemes, etc. Free trade agreements are not concerned with the health and well-being of the citizens.
The EU, the Single Market and Customs Union is not so restrictive. The EU contributions does far more than compensate for the lack of tariffs.
As I said, it's a choice between a customs union (free trade with a membership contribution which does far more than covers trade tariffs) or trade tariffs (and non-tariff barriers).