If you get that reaction from perfume, but not cologne, you are, like me, allergic to ambergris.  What is ambergris, you may be asking? Whale snot.  Not an exaggeration.  People act like I'm the weird one, because I can't tolerate the contents of a whale's sinuses.