Richardson also said the withdrawal of so many top golfers from the Rio Olympics next month had made it harder for sports such as cricket, that were considering bids for Games status.
But with no prospect of T20 at the Olympics until the 2024 Games at the earliest, the question of how the ICC deals with fixers is likely to remain a more immediate concern.
Thursday's match at Lord's was Amir's first appearance in a Test since the infamous spot-fixing clash against England at 'the home of cricket' in 2010.
That match saw Amir and Pakistan new-ball partner Mohammad Asif deliberately bowl no-balls on the instructions of then captain Salman Butt as part of a newspaper 'sting' operation.
Pakistan rallies behind Amir as he returns to Test cricket
All three received five-year bans from cricket and jail terms.
England captain Alastair Cook has been among those calling for life bans instead, although ex-England skipper Mike Atherton has argued that players such as Amir, a teenager at the time of his ban, deserved more sympathetic treatment.
It was a view shared by Richardson.
"I am not uncomfortable with it at all," Richardson, speaking at Lord's, told BBC Radio's Test Match Special when asked about Amir's return.
But the former South Africa wicket-keeper, a qualified lawyer, was less enthusiastic about compulsory life-bans.
"Each case should be treated on its merits. You cannot hang everybody," he said.
"I think we have to stick with the principle that the punishment should fit the crime ... (and) players who influence others should be treated much more severely than those who are influenced."
I have no problem facing Amir: Cook
Richardson added: "We must ensure the deterrent is sufficient ... and we have bolstered our investigations so that they (can be conducted) more efficiently and quickly."
Meanwhile, the ICC continue to ponder whether an application to for T20 to become an Olympic sport from 2024 would be the best way of growing the game globally.
Several leading golfers have pulled out from their sport's return to the Olympics for the first time since 1904, citing health concerns related to the Zika virus in Brazil.
"It is not an easy decision, and golf is probably a lesson to be learnt," said Richardson.
"It raises a question for cricket: 'Is it really good for cricket?'"
There is a view that a sport where an Olympic gold medal is not seen as the highest honour has no place in the Games.
"Will cricketers see the Olympics as the pinnacle, or would they rather play (in a World T20, or an Ashes)?," said Richardson.
ECB forced PCB to ignore my case: Kaneria
However, he added: "I still think the majority of (ICC) members think that if cricket were in the Olympics, it would do wonders for the globalisation of the game.
"If that is what we want, then we probably have to be in the Olympics."
Before that happens, the ICC is set to introduce a two-division Test championship with promotion and relegation.
If an agreement is reached later this year, the new system could be installed by 2019.
At the moment a straight league table appears to be the favoured option but Richardson hasn't given up on the idea of a Lord's final to crown the new Test champions.
"The beauty of Lord's (for a Test final) is that, even if it was neutral teams, I'd imagine we'd still fill it out," said Richardson.
"There'd be the interest, there's the history."
But whatever decisions are made, Richardson was adamant that change had to happen if Test cricket was to have a viable future.
"Doing nothing is not an option any more," he said.
"In particular, that is because at the moment we don't have an ICC event for Test cricket.
"Bilateral cricket is under pressure from ICC events ... and domestic T20 leagues."
COMMENTS
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ