Gender Bias in LLaMA-3 Embeddings: Implications for LinkedIn-Style Retrieval Systems

Large language models increasingly power professional network retrieval systems, with LinkedIn recently deploying LLaMA-3 for member and content embeddings. We investigate whether these embeddings exhibit gender bias by measuring semantic drift when we attribute identical professional content to names of different perceived genders. We constructed 406 paired LinkedIn-style posts with identical text content, headlines, and professional context, differing only in author name (male versus female variants), and extracted embeddings using LLaMA-3.2-3B's hidden states with mean pooling—replicating LinkedIn's published methodology. We find systematic bias: mean cosine similarity between paired embeddings is 0.994 (not the expected 1.0), with Cohen's d = -0.93 (large effect) and p < 0.0001 across both parametric and non-parametric statistical tests. This approximately 0.6% embedding deviation, while small per-pair, represents systematic differential treatment that compounds across retrieval, ranking, and recommendation systems—potentially affecting search visibility for millions of professionals. We release our dataset, code, and statistical framework to enable reproducible bias auditing of LLM-based retrieval systems.