Cozumel you certainly don't need an excursion. There are lots of beaches you can go to with the kids, and lots of taxis at the pier to take you there, and at the beach clubs to bring you back.
Jamaica is a bit trickier. You may want to look into getting a day rate at Sandal's Resort for the family. Spend the day at the beach and their pools.