Don't miss the Provincetown Portuguese Bakery for some delicious treats. Do you have pirate-loving kids in your family? Be sure to stop by the Whydah Pirate Museum in West Yarmouth, MA.