I've been there seven times and never missed the port.
The problem with Grand Cayman is that it is surrounded by a reef and cruise ships cannot go over the reef and must "tender" passengers ashore. If the water is too rough then the ship cannot tender passengers and bypasses Grand Cayman for a sea day.
We have yet to bypass it. Actually a small percentage of cruises have to bypass Grand Cayman. If it was bypassed too many times they would remove it from the itineraries.