Many retirees are surprised to learn that their Social Security benefits can be subject to federal income tax. The IRS determines the taxable portion based on what it calls your “combined income.” Understanding this formula can make a meaningful difference in how much income you keep during your post-career years.