I haven;t been there but I hear it is not chaotic at all. Only one ship in at a time. They say the process is very smooth.
Horizon 1990, 1992, 1999
Crown Dynasty 2000
Century 1996, 2001
Explorer of Sea 2003
Norwegian Dawn 2004,2005
Voyager of the Sea 05/05
Celebrity Zenith 1/23/06
Celebrity Century 11/13/06
Norwegian Dawn DD cruise 10/14/06
Celebrity Constellation TA 9/15/07
Norwegian Dawn 10/25-26 Nowhere
The Bayonne port is in the midst of the old abandoned Military Ocean Transport terminal. Many of the low old style (WWII) buildings are empty and decaying. Some are used for warehousing. It's a little hectic right in front of the terminal building while loading and unloading, but other than that, it's pretty wide open and empty.
At some point in the future, a cruise terminal will be built and the area will experience a rejuvenation. But, it is far less of a hassle than coming into New York itself. From Bayonne to mid-town is about a 15 to 20 minute ride. In typical NY rush hour - add another 2 hours for traffic.