Mentioned Isn't Recommended: How ChatGPT and Gemini Recommend HR Software (CobaltKite Research 001)
Description
I asked ChatGPT (GPT-5.6 Sol) and Gemini (Gemini 3.1 Pro) which HR software a 50-person US startup should buy. Three prompts, ten runs each, on both assistants: 60 answers, collected in October 2026. For every vendor, I tracked whether it was named at all and whether it made the top three of the recommendation list.
Deel was named in 68% of answers (41 of 60) but made the top three in just 1 of 60. Rippling was ranked first in 52 of 60 answers (87%) and was in the top three of every one. Four vendors (Rippling, Gusto, BambooHR and Justworks) took 99% of all top-three slots. Workday was named in 13 of 60 answers and never recommended; every one of those mentions advised against it.
Being named by an AI assistant is not the same as being recommended. This dataset includes every answer word for word, the coding behind every number, and the script that reproduces the results. See README.md for the full method, definitions, and limitations.
Methods (English)
60 runs: 3 prompts x 2 assistants x 10 runs. Fresh temporary chat for every run, personalization off, no follow-up messages, prompts rotated. Web search left at each assistant's default. Runs made from outside the US without a VPN. Recommendation lists coded by hand; mentions detected automatically from the answer text. Rates reported with Wilson 95% intervals, differences with a stratified bootstrap (10,000 resamples), and headline findings checked by leaving out each prompt and each assistant.
Files
mentions.csv
Files
(356.5 kB)
| Name | Size | |
|---|---|---|
|
md5:8e98e479560c56ed69a607e223f9c61d
|
17.8 kB | Download |
|
md5:b7c34fa51a487a21f41bdf3011dd2959
|
15.1 kB | Preview Download |
|
md5:073de4e8c7577d4b5037e2ca8f946d02
|
440 Bytes | Preview Download |
|
md5:341f651f12ea1140e033dd4e1dabc1f2
|
14.0 kB | Preview Download |
|
md5:9f18b9eeaa28953cebb0d3bc69e9e9dd
|
273.3 kB | Preview Download |
|
md5:5ca029fc73a6b39a7eabc308e3dd7956
|
35.8 kB | Preview Download |
Additional details
Related works
- Is described by
- Other: https://cobaltkitecreative.com/research/mentioned-isnt-recommended/ (URL)
Dates
- Collected
-
2026-10Data collection