Yep, I do that all the time, not walking on armrests though, but by overstepping the legs of person in the aisle seat. They usually sleep so I don't want to be a bother and I just sort of walk/jump over their legs without touching them.