Tasha, we've removed stuff from a chair before when it hadn't moved for more than 2 hours and they didn't say a word to us. They seemed to take a longer look at us, then looked the other way and looked at those people and since they couldn't figure it out, just didn't say anything at all. We ignored them and went about our business of lounging, no problem mon!
I would suppose if the persons stuff was moved and they felt ignorant enough to say something they would, but so far, it hasn't happened to us. Plus, since I don't get out on ANY deck before 10am (unless I'm going on an excursion), I don't run into this problem as much because I've always been able to find a space that hasn't been saved. I'm pretty tolerant and will let some things slide but when its been over an hour, forget it, rules don't apply anymore and the space is mine. If asked, I do say that the spot was vacated before I got there and have no idea what they are talking about. That usually shuts them up and they move on in a huff. Not my problem.