Best thing to do? Take the passports first & stick them in your purse or in the zipper area that we have in them as you start packing.
Passports are always the first on our list - everything else is secondary only becuase clothes & such can be bought anywhere. So if we forget something, big deal.
Passports though? Oh heck no. They are the first things packed.