We were there a couple of years ago and the people are agressive, but we took a shore excursion thru Carnival and had no problem. We saw Cayoba Gardens, which was beautiful, they took us shopping in a restricted area, so the vendors weren't aggressive. Dunn River Falls was great.....the area where the food vendors and craft vendors are was the only place we ran into competitive peddlers. We had no problem, and we were two gals. I am not sure how comfortable I would be visiting the island on my own, though.
Unless it has changed a lot inthe last couple of years, I wouldn't worry about it as long as you are with a supervised tour.
Have a great time