Dubbed the “Holiday House,” the Watch Hill mansion was bought by Swift in 2013, complete with an Olympic-sized swimming pool and a past that ties it to the Standard Oil fortune.

The holiday commemorates the full and complete enactment of the Emancipation Proclamation in the U.S. President Abraham Lincoln issued the proclamation to free enslaved African Americans in ...