My son is now 19, and my step daughter 18. We are proud to have them join us, as both were raised early on as "only children". They were not perfect kids, and I think every parent can share one or two hellish experiences with their kids acting like rug-rats.
Personally, if I had to do it all over again (or maybe when I'm a grandfather) Disney would be the best option, it seems more kid-friendly. The older I get, the more impatient I become and forget that just 15-years ago, I myself had a 5-year old.
I believe that children should be heard, and seen. For that matter, a 5-year old or a 105 year old has something to say, and it wouldn't hurt for us to exercise more tolerance (and love) and listen...maybe? I believe that as older adults we forget what it was like when our children were that age, or heaven forbid, we were too! :o Nothing worse than a crotchety, cranky soul scowling at a child who isn't yet touched by the crazy, harried and harsh world we live in (at times). No, children are not for everyone, but it's not their fault.
My vote: If you're unhappy with your table mates, move...rather than attempt being pleasant and exchaging kind words, ask the Maitre'D to move you.