Best done when the system has been off and cooled for a while. Yes the screw can be removed from the pump,water will come out ,more of a trickle,so have large towel on hand. The short stub should not have it's screw removed ,just loosen it . Air will come out if present ,followed by water ( rather like bleeding a radiator).
Reading your other thread ,you said that water in the cylinder is now heated ,but still no central heating ,is that so ? If it is ,there isn't any real point in venting the stub of pipe but won't do any harm.
I assume the boiler has heated the cylinders water ,not an electric immersion heater ?