I used to think that, but "baggy" is relative.  I can conceal something like a glock 19 appendix or at 4 o'clock and not look out of place at all.  Fat guys simply can't do appendix and most have to do 3 o'clock because they can't even reach 4 o'clock.