The best way to get over an irrational fear is by confronting it....so, buy a fanny pack!
Seriously, even though they are out of style, I still use one in ports. Like an extra set of pockets. Great for things like extra camera battery, some cash, kleenex. I mainly use a very light nylon backpack for most stuff, but hate having to dig through it for something small.