Sanjay Dutt used to whisper 'I love You' to Madhuri Dixit on sets, but their relationship ended after his arrest and divorce from his wife Richa Sharma.