I read once that a poster would sit pool-side for a while and scope out what chairs had been saved. Then they would remove the towel or book and place it in the adjacent saved chair (they felt that savers travel in packs). The poster then dragged the chair to another location on the deck. That way, there was no confrontation when the saver returned.
I don't know if I could be that bold.
When I leave my chair in the "saved" mode it's only to run to the WC or grab some iced tea.