I've been there and loved it. The thing to do though is leave the port. It's an industrial port. The island is divided like a butterfly into Grande Terre and Basse Terre I believe. The western, mountainous side is where you want to go. Soufriere is a semi-active steaming volcano you can climb, there is a really nice waterfall in the south, Pidgeon Island on the far west coast is where Cousteau gained fame, there's a nature preserve mid-island near La Mamilia (something like that - just think Mammory - that's what the twin mountains are named for :-)
My advices is to head to the western side of the island, either on your own or with an excursion. Hertz has an office very close to the dock, but be warned that many people there speak limited English. French is the language.
We stopped at Guadeloupe on a cruise a few years ago. We loved it. We spent the day at Club Med (day pass) on one of the prettiest beaches in the caribbean (I believe it was Plage St Anne). Gorgeous soft sand, many beautiful palm trees.