The way to remember is to keep in mind that they don't want to name themselves as the "old" gender anymore, because it's what they're trying to get away from.  So a person saying "I am a trans man" won't be somebody who was born with male parts.  They name themselves after what they want, not after what they're fleeing from.