Could you clarify this 'Would this amount of exposure really cause any harm?'
If you mean would removing artex in an uncontrolled manner cause harm, then yes it would. Although the asbestos contect of artex is low, using a power drill on it could cause high fibre levels in an area where your face is likely to be.
So for the sake of a small glob of wallpaper paste it make sense to do it.
As for the risk in comparison to other activities, well asbestos cause three main diseases, asbestosis, mesothelioma and lung cancer. Asbestosis and lung cancer are dose related, so a one off exposure is unlikely to cause a major problem, mesothelioma however is not, and one exposure could cause the onset of the disease, obviously prolonged and frequent exposure is more likely to cause problems.
Please don't base your health and safety around the fact you have other dangers in your life, thats not a sensible approach. Around 4000 people die each year from asbestos related exposure, its the UK's largest industrial killer.
My advice would add 5 minutes to the job and cost under a tenner, worth it for peace of mind in my book.