If you have 6 wards, say for example: 5 wards of 3 councillors and 1 ward of 6 councillors, then if only 3 candidates put themselves up in the ward with 3 seats, there is no election in the ward, they are elected without contest.
If you have 4 candidates in a ward with 3 seats then you have an election. So in a village or town you could have one part of town with an election and one part without. (Depending on where your polling stations are, you could have some people in the village hall with one ballot paper (e.g. district) and the other side of the polling station with 2 (district and parish).
If others want that councillor out, put up at least as many candidates as there are seats in that ward (the clerk can confirm each ward), then if he re-stands, there will be an election in 'his' ward, and the other candidates can run an election campaign (and because the area is usually quite small, it can be quite cheap - but they have to follow the Electoral Commission rules). They can even run on a local 'slate' so 4 candidates can stand together with a description of "Make West Village Nicer" (max 6 words).
In the lead up to the election period, the Parish Council has powers to promote / advertise a forthcoming election (there are NALC resources for this https://www.nalc.gov.uk/elections) that be shared online or even promoted on Facebook, etc. (Obviously, they can't promote a candidate).
(edited as I got the numbers wrong at the top).