Reproducibility of "A Light-Weight Strategy for Restraining Gender Biases in Neural Rankers"

The following paper is a reproducibility report for "A Light-Weight Strategy for Restraining Gender Biases in Neural Rankers"  published in ECIR 2022. The authors, Bigdeli et al., claim that their negative sampling strategy decreases the level of gender bias while maintaining a comparable level of retrieval effectiveness with the additional benefit that the strategy does not require any changes to the architecture of SOTA reranking models. The information provided with the paper and their linked github-repository proved to be enough to implement their proposed sampling strategy and get similar bias-reduction results, but we weren't able to validate their claim that the retrieval effectiveness doesn't change in a statistically significant way. This could be due to missing information regarding their experimental setup, and especially, chosen parameters. The results from our experiments lead us to different conclusions that challenge the findings that Bigdeli et al. published in their paper.