Second-half goals from substitutes Marouane Fellaini and Dries Mertens secured an opening Group H victory for the Red Devils after Algeria had taken a 25th-minute lead via a Sofiane Feghouli penalty.
Manchester United midfielder Fellaini headed Belgium level on 70 minutes before Mertens struck 10 minutes later, latching onto a sublime pass from Eden Hazard and blasting home past Algerian goalkeeper Rais M'bolhi.
Algeria, who have not won a World Cup match since beating Chile at the 1982 finals in Spain, will now look to break their winless streak against South Korea or Russia, who play later on Tuesday.
Belgium coach Marc Wilmots, the last Belgian player to score a goal at the World Cup in the 2002 finals, is hoping to make a big impact in Brazil.
His Belgian side is seen as one of the most promising squads in international football, with a generation of talented players including Chelsea star Hazard, Manchester City defender Vincent Kompany and a slew of other foreign-based players.
Later Tuesday, hosts Brazil will aim to all but secure their place in the last 16 with a victory over Mexico.
The five-time champions opened their campaign with a 3-1 win over Croatia while Mexico edged Cameroon 1-0.
Mexico shattered Brazil's dream of winning the only major prize missing from their honours list two years ago, when they stunned the South Americans 2-1 to claim the Olympic gold medal.
Brazil striker Hulk insists there is no question of Brazil looking to settle a score at Fortaleza's Castelao Stadium, the same venue where they met in the Confederations Cup last year, when Luiz Felipe Scolari's side claimed a 2-0 victory.
But it is also the site of one of the World Cup's big shocks so far, Costa Rica's 3-1 upset of Copa America winners Uruguay.
"In the Confederations Cup we tried to think only about winning the game and it will be the same this time. If we think about revenge it could cause us problems," said Hulk.
The late game in Group H pits South Korea against Russia.
South Korea's fortunes have nose-dived since 2002 when they turned the form book on its head by becoming the first Asian side to make it through to the semi-finals.
But before reaching their eighth consecutive World Cup, they struggled in qualifying, squeaking home only on goal difference.
Coach Hong Myung-Bo gained iconic status in Seoul after his side-footed penalty took Guus Hiddink's side into the 2002 World Cup last four.
Korea's support stretches far and wide, and poses a dilemma for United Nations Secretary General Ban Ki-moon, who admits he is struggling to maintain his normal diplomatic neutrality.
"I may not hide my own sentiment and support for the Korean team. But as secretary general, it would be very important that I need to be impartial. I'm supporting all the teams who are participating in the World Cup," he insisted.
Russia are back in the World Cup for the first time since 2002, when like in 1994 they failed to make it out of the group stages.
Under Fabio Capello, who guided England to the last 16 in South Africa four years ago, Russia coasted through qualifying and in stark contrast to South Korea are unbeaten in 10 games.
Meanwhile, the fall-out from Monday's action, which saw Germany rout Portugal 4-0 and African champions Nigeria labour to a 0-0 draw with Iran, continued.
Portugal denied midfielder Raul Meireles gave the referee the finger in his side's stormy loss, insisting that he was merely gesturing a tactical switch.
Pictures on social media show Meireles, who sports a distinctive Mohawk hairstyle, raising the third finger on both hands to referee Milorad Mazic who had sent off Real Madrid defender Pepe.
But the the Portuguese football federation (FPF) defended the Fenerbahce player, claiming there was nothing vulgar about the gesture.
COMMENTS
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ