I think the reason they're not held to account is because everyone else can just point to that individual and say it's not their fault. But with anonymous voting, acts of Congress would be blamed on the entire Congress, and would incentivize Congressmen to police themselves lest they lose their jobs along with the at-fault party.
Also, there's a lot of other bad behavior that individual Congressman can engage in that would be firmly pinned on that individual. Their words in committee or on the floor, would still be available, and their voters could vote them out on that.
girlfreddy