To Sheila who said that her ex was incarcerated: You can bypass the ex.
You can obtain such permission from the court of jurisdiction to allow travel. Call your family attorney who handled your custody case. A document allowing travel, issued by the court is even better than what is described as a "notarized" letter. Such a document, once applied for, would be issued by the Judge and is pretty much "good" indefinitely. Look into this because your particular situation is not one where you want to dredge all that up every time you want to go somewhere...call your lawyer. He/she should do this for you as a courtesy because it's a small matter and it can be taken care of rather quickly....you might have to pay a small filing fee for the document to be filed into the court record.