High consumer price sensitivity leaves brands with little room to offset the costs of tariffs and labor market disruptions.