The Florida Aquarium is right next to the port. It's small but nice to see.
You can also go to the Univeristy of Tampa and visit the Henry B. Plant Museum. It's only a short drive from the port. You could walk it but it is about 1 3/4 miles. It brings you through downtown Tampa.