The clerk can tell anyone to remove a fence if they wish, just as i (a mere angry resident) could tell you to remove your fence. You would probably tell me where to go. Without a resolution at a meeting of council they would not be acting on behalf of the council.
However, regardless of this, i am not sure that the Parish Council have any powers to order you to remove a fence anyway.