I highly suspect that the man has some bisexual or bicurious tendencies(they mention 'demisexuality' at one point which led to my suspicion), it's just that the only Mormon woman who was desperate enough to marry him despite his predominant attraction towards men was a lardbeast, so whatever sensual spark or fire was just quenched instantly anytime she took off her clothes.