Khan, now 72, went along to a charity fight night in India's Mumbai in January 1980 hoping merely to catch a glimpse of 'The Greatest' in action.
Instead, he got closer to his idol than he could ever have imagined, going blow for blow with ex-heavyweight champion Ali in the ring for three rounds of three minutes each.
"Beforehand my dad and I were just joking but he said, 'If you get the chance then box him,'" Khan told AFP, his eyes wide and grinning at the memory.
Muhammad Ali rewrote the rule book for athletes as celebrities and activists
"Then during the event Ali shouted out to the audience and said, 'Is there anybody who will box with me in India?'
"Nobody was standing up. So I stood up, put my hand in the air and said, 'I will accept the challenge, I will box you.'"
Khan, a national bantamweight champion in 1964, had always been a massive fan of Ali, who died on Friday aged 74 after health problems complicated by a long battle with Parkinson's disease.
In his tiny south Mumbai flat, Khan removes a photo album from a plastic bag that he says holds his "treasure" -- photographs and newspaper reports documenting that first bout with Ali.
"When I put my hand up Ali kept saying, 'You want to box with me? You want to box with me? Aren't you scared of me?, Aren't you scared of me?" Khan explains.
"I said, 'No, I'm scared of Allah but I'm not scared of you. I'm Haroon Khan, I'm from India. You're a boxer, I'm a boxer, so let's box."
"Ali was fooling with me. He asked, 'What do you box, oranges or bananas? I replied that I box boxers and I'm ready to box you."
Why Muhammad Ali's star is on the wall, not the Walk of Fame
Khan, who now coaches youngsters and keeps fit with a punchbag in his home, remembers thinking that because he was an amateur and Ali was a professional he must be on his game.
"I felt I couldn't just joke around so I fought a serious fight. Ali was surprised by how good I was but when I punched him it felt like I was hitting a mountain."
"Nothing happened. He didn't move at all. He just kept grinning. I wondered what sort of man is this? How much power does he have to take these punches?"
Ali, a three-time heavyweight world champion, was past his prime by 1980 but was still a year away from retirement.
He went easy on Khan during the first round of their exhibition bout, but the Indian boxer soon experienced the full force of Ali's powers.
"After the first round people started shouting at him saying, 'You're not boxing. We want to see a fight.'"
"In the second round he started to get serious. He was so fast. He started jabbing me and within a minute-and-a-half a whole side of my face was swelling," explains Khan, mimicking Ali's moves.
In the third round Ali displayed some of his famously quick footwork, but slipped and fell at one point. A photograph in Khan's album shows him smiling, standing over Ali after the fall.
"At the end he held my arm up and praised me to the crowd. 'I like this boy, I like his guts,' he said. He was a gentlemen boxer. I felt so lucky, happy and proud to have boxed him."
Two years later Khan saw a newspaper advert saying that Ali would be the star guest at an exhibition fight in the United Arab Emirates so he went along seeking a second bout.
Last name Ever, first name Greatest: 11 memorable quotes by Muhammad Ali
Khan said he approached Ali at a press conference and he agreed to another three-round fight.
"I was better prepared this time. I knew how he boxed. I kept guard, I covered myself," Khan proudly recalls, adding that the two of them went for dinner the next day.
Khan says he was heartbroken when his son called him on Saturday to say Ali had died.
"Tears just started to stream down my face."
COMMENTS (1)
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ