Never wore my shorts and once I wore a polo shirt.
A short sleeve or long sleeve shirt with a light sweater works well also a short sleeve fleece and a light nylon parka or rain slicker did the trick.
We actually bought for 1/2 price, second day out the Celebrity nylon jacket which comes with a hood and is great quality,