seas could be rougher than normal after Irene has passed, but the further you get away from her and her previous path, things should calm down quite a bit. As for the ports of call you will be visiting, that could be another story !
Better make sure your trip insurance covers you if you elect to cancel though. Some policies let you cancel for any reason, but not all. And to cancel now because you are afraid you may not have a good time due to the hurricane passing would fall under a cancel for any reason allowance in your policy.