Doherty leaving Charmed after Season 3 was shrouded by controversy with speculation that she quit due to behind-the-scenes drama with her co-star Alyssa Milano, who played Phoebe Halliwell.