Democratic accountability and the ballot box are your main weapons here. Even the council cannot remove the chairman until the end of the council year, in May. At the start of each new council year, the chairman is elected, and if you are dissatisfied with the council chairman, you can press councillors to elect someone different next May.
Beyond that, councillors may well be swayed by vocal and extensive public protest and consider making changes, or even resigning. The latter is helpful provided you have alternative candidates lined up. 10 electors can insist on an election rather than a vacancy being filled by co-option.
Once every four years, the whole council must be elected - the next election will be in May 2015.