In theory police serve the mayors and FBI serves POTUS, all of whom are obviously elected.  So they aren't "illegitimate" per se.  They are, however, not directly accountable to the people.  Which in many cases isn't an issue, but I think we now have clear evidence on numerous situations in which it most definitely is an issue.