You will be fine on either ship. Any ship with longer cruises (14+), especially the exotic destinations (Tahiti, south seas ... ) will attract a more mature crowd, but in the Caribbean you will have a nice balance of fellow passengers.
If you do cruise at a time when kids or teens might be cruising, and you want to avoid them, it's best to go with the smaller ship. The larger ships with the most activities attract more families and young people.