You want to know the bottom line...
Just get a notorized letter..Is it worth the chance that you don't get it notorized, and you are about to board when the agent at the pier say's "Just one more item before you board, I would like to make a copy of your notorized letter authorizing your son to travel with you from his father"...Ooooopppsss.