We've had this come up. Well, our question was whether councillors not on a committee can speak at the public questions agenda item for the committee. The Local Authority Deputy Monitoring Officer informed me that they can't, because they are councillors and not members of the public, but as such, they should be able to approach the committee in other ways (such as informing the committee chair that they wish to speak). This then led me to ask about whether such councillors should leave with members of the public during closed sessions and they maintained the line and said they shouldn't - because they are councillors, not members of the public.
However, I really don't think the Deputy Monitoring Officer at my LA is right (perhaps it were the case in years gone by). Imagine a confidential staffing issue that the clerk wished to raise with the staffing committee about one particular councillor, yet that councillor was present because they couldn't be removed. What John1706 says is what I believe is right and it's the current advice from NALC in Legal Topic Note 5E. Essentially, if you're sitting on the "business" side of the table, you stay, and everyone else should vacate.