A member of the public cannot force an item on to the agenda, so a vote of this nature would require the cooperation of a councillor or the clerk. The council could refuse to take the vote and agree a resolution to move to the next item of business. A vote of no confidence has no binding consequences.
A member of the public can insist on a vote at a parish meeting, although again such a vote is not binding on the council.
Six members of the public can require a parish meeting to be held. And then insist on a vote on any matter.
Opinions can be voiced, but there is no way to remove a councillor (unless they become disqualified) except at elections. Having said that, substantial and continuing pressure from a sizeable number of the electors is likely to have an effect in the end.