Erdogan's candidate, Binali Yildirim, conceded defeat as initial results showed his opponent ahead with 53.69 per cent of the vote with more than 95 percent of ballots counted.
"According to the result, as of now my competitor Ekrem Imamoglu is leading the race. I congratulate him and wish him good luck," Yildirim said.
Imamoglu, a little-known district mayor at the start of the year, had portrayed the re-run as a test of Turkish democracy after he was controversially stripped of his first victory in March.
Election authorities annulled that result after Erdogan claimed irregularities in the counting -- but that galvanised Imamoglu, who vowed "a battle for democracy" to take back the city of 15 million.
Critics say Erdogan was simply reluctant to give up control of Istanbul, Turkey's economic powerhouse and a crucial source of patronage for Islamic conservatives since he won the mayorship himself a quarter-century ago.
Imamoglu, who represents the secular Republican People's Party, said after voting earlier on Sunday: "Today our people will make the best decision... for the sake of our democracy, for Istanbul and also for the legitimacy of all future elections."
His upbeat message under the slogan "Everything will be fine" contrasted with the usual aggressive name-calling of Turkish politics and struck a chord with voters.
Istanbul votes in test for Erdogan and Turkish democracy
But he faced the juggernaut of the AKP, which has ruled Turkey since 2002 and remains the most popular political force nationwide thanks to years of dramatic growth and support for previously excluded religious conservatives.
Yildirim, a mild-mannered Erdogan loyalist who oversaw several huge transport projects and served as prime minister, had already struck a conciliatory tone earlier on Sunday.
"If we have wronged, knowingly or unknowingly, one of our fellow Istanbulites or our challengers, if we have done something unjust, I ask for your forgiveness," he said.
Analysts said the re-run was a lose-lose proposition for Erdogan: a second defeat would undermine his image of invincibility and embolden rivals within his party, while a victory would forever be seen by the opposition as stolen.
It came as an economic slump and rising prices have dented the president's reputation for economic stewardship, with the AKP also losing the capital Ankara in March.
For many conservatives, Erdogan remains a hero who brought prosperity and has fiercely defended the country's interests.
Most accepted the line that the re-run was necessary.
"If there's something like stolen votes, I think it's better to redo the election in the name of democracy," said 45-year-old Huseyin as he queued to cast his ballot.
Erdogan maintained his allegations of irregularities as he voted on Sunday, saying the last election "should not have happened like that".
But the controversy over the re-run may have explained his relative silence, with no repeat of the tireless rallying for the March polls, which included 102 appearances in just 50 days.
Erdogan has played down the importance of the re-run, saying last week that the choice of mayor was "only a change in the shop window" since the AKP controls almost two-thirds of the city's districts.
Fearing fraud, the opposition mobilised an army of lawyers from across Turkey to monitor Sunday's election, with the Istanbul Bar Association unfurling a huge banner at their headquarters reading: "Stand guard for democracy".
COMMENTS (1)
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ