True story.  I rode my entire life until my early 20s (three-day eventer, here).  I sold my horses and took a break for 5 years to focus on triathlons, work and family. I started riding again after having been off for over 6 years.  Even though I was in great shape, I was so sore those first few weeks of riding lessons.  It is a very physically demanding sport.  You don't just sit there.