People usually lose muscle simply because they're not using it enough, and the body is like "fuck it, I don't need this. Let's get rid of it." It's the same as if you were at a caloric surplus and got lazier.