Some double sided Velcro or a loop of cord . A short length of (white) polythene pipe, this acts as the width restrictor.
Loop one end of the Velcro around the handle, then around the inner. Maybe figure a method of holding part of the Velcro (a slot?) so it stays together.
Or using the cord, two loops of cord, joined in the middle by a strong elastic band/ rubber. Pull to fit
You could replace the spacer with a squidgy ball
Sounds amazing, trying to get my head around how it would look.