Massive job losses and farm distress have damaged the aura of invincibility Modi and his ruling Bharatiya Janata Party (BJP) had until recently as the world's biggest democracy approaches a general election due by May.
But the opposition suffers too from a perceived lack of leadership, so sought to project unity at Saturday's rally and bind forces against Modi's Hindu nationalist party.
Indian opposition stages giant joint rally to oust Modi
They have not, however, presented a candidate to replace him, saying their first priority was to defeat Modi. Finance Minister Arun Jaitley, a top BJP leader and government troubleshooter, said a "presidential"-style vote casting Modi against an opposition coalition would help him.
"The opposition is making Prime Minister Modi's continuation in office as a key political issue. We in the BJP welcome this agenda being set by the opposition," Jaitley said on Twitter.
"What has been presented by the opposition is a post-election leadership battle, no common programme, no policy and a disastrous memory of their administrative disabilities ... It is only the fear of (Modi's) popularity and comeback that is bringing (them) together."
The main opposition party Congress has already ruled out naming its leader Rahul Gandhi, scion of the Nehru-Gandhi dynasty that has dominated Indian politics for decades, as a prime ministerial candidate before the election.
The opposition front has various other leaders with prime ministerial ambitions, including the chief minister of west Bengal state, Mamata Banerjee.
COMMENTS
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ