The Local Government Act doesn't mention panels at all. There is no legal framework for them. You cannot act unlawfully in the use of panels, as there is no law. Panels, under many different names, are a means by which some councils work with the wider community and, as such, they have an important role to perform. The expectation is that panels offer a greater level of public access and transparency than the more formal structures. I have panels at present on children's play/adult gym and on communications. They're a forum for wider public consultation and they make recommendations to the Council.
However, whilst we cannot limit the powers of panels, their existence doesn't alter the requirement for all council decisions to be made lawfully in accordance with the LGA. Panel decisions have no greater legal status than a conversation in a pub. The council's standing orders and financial regulations, particularly if the model documents are used, provide a robust framework for legal, transparent and democratic government. It is a requirement that financial and staffing issues are made by a council or its committees, in public where possible, and this power doesn't extend to secret meetings of private groups.
In my original response, I referred to sub-committees, as this is a route used by some councils to avoid public scrutiny. Contrary to popular opinion, sub-committee meetings do not have to be open to the public.
So to sum up, there's no point in challenging the legality of the panels per se, but you should ask your Clerk to explain how the decisions that are made align with the Council's standing orders on decision making and the legal framework on which they are based.