I've never had any real problem with noise. Then again, my neck often gets sore from leaning down and listening to the glass against the wall.
The biggest problem I've had with neighbors are leaving lights on, on the balcony. The worst was a couple who strung strobe "Christmas Lights" on the balcony. Those drove me nuts even with the curtains drawn. It took just a small conversation for them to turn them off before they went to bed.