Notwithstanding why you want to do it and what you feel it might achieve there are two obvious ways:
1. Ask for a motion of no confidence to be put on a future Town Council agenda - although it may be rejected for inclusion. let alone be voted on.
2. Organise a Parish Poll
Note that the Town Council would have no legal obligation to act on the results of a parish Poll whatever the voting outcome.