We never hear of what happened to the ones who did challenged him.  Only a handful survived.  Behind the scenes, his held a tight reign of anyone who spoke out against him.  He himself created a security force so there was no opposing forces.