It's a charge per person by destination - so both factors are considered into the cost.
This website may help you with taxi prices on St Thomas: http://vinow.com/stthomas/getting_around_stt/taxi_stt/
On St John from Cruz Bay (ferry landing):
Also, if you prefer to see the island from the water, you could always take the ferry from Charlotte Amalie to Cruz Bay and not have to taxi to Red Hook (I get car sick easily). The last cruise we took to St Thomas, we anchored in the bay and the tenders took us directly into Charlotte Amalie, about 10 steps from where the ferry to Cruz Bay leaves. However, many ships dock a couple of miles outside of town, so you would need to take a taxi either to Charlotte Amalie or Red Hook to catch a ferry.
Ferry schedules: http://vinow.com/general_usvi/interisland_ferry/
The ferry from Charlotte Amalie costs $7 (versus $3 from Red Hook) but if you don't have a taxi to Red Hook, the costs balance out. However, the ferry from Charlotte Amalie doesn't run as often.
When we vacation on St John, we very seldom use the Red Hook ferry due to proximity to the STT airport. Nothing like stepping off the plane and soon feeling the bits of ocean spray on the ferry (we sit on the outside part). : )
Trunk Bay has the $4 per person entrance. Another option you might consider is Cinnamon Bay - the next beach along the north shore. Has facilities, food, great snorkeling, etc - and no entrance fee. You will see Trunk Bay from the road (there is a pullout) on the way to both Trunk or Cinnamon. We like going to the far left end of Cinnamon Bay past the rocks - not many people there (called "Little Cinnamon") and still awesome snorkeling. There is a campground at Cinnamon.
St John is definitely worth visiting! Just always remember to leave nothing there but your footprints in the sand! : )