It's not just men. It isn't for me, anyway: I judge women based on the type of man they've chosen as well. If a woman dates somebody much less attractive than herself, usually meaning they're fat, I get the vibe that she has little self-respect and is likely to fuck up the genes that made her as attractive as she is by having hamlets with some planet.