Brazilian midfielder Fernando deflected Gareth Bale's cross into his own net after 20 minutes at the Santiago Bernabeu for the only goal of the tie after the first leg ended goalless.
Fernandinho came closest to taking City to their first ever Champions League final when his shot clipped the outside of the post a minute before half-time.
Real beat Atletico 4-1 after extra-time in Lisbon two years ago to win the competition for a 10th time and will be hoping for a repeat when they face Diego Simeone's men in Milan on May 28.
The hosts were given a huge boost before kick-off as Cristiano Ronaldo made his return after a three-game absence due to a thigh injury.
By contrast, City were dealt a massive injury blow inside 10 minutes when captain Vincent Kompany was forced off once more with a muscle problem in an injury-disrupted campaign.
Eliaquim Mangala was forced on to deputise for Kompany, but City's makeshift defence wasn't overly troubled despite the presence of Ronaldo and an in-form Bale in the first 45 minutes.
Ronaldo was looking to equal his own record of 17 Champions League goals in a season, but sent his first chance well over the bar with a header from Dani Carvajal's cross.
However, Madrid got the luck they needed to swing the tie in their favour with their next attack as Bale was played in on the right of the City box and his intended cross spooned up off Fernando and into the far corner to leave Joe Hart helpless.
The difference in European pedigree between the clubs couldn't have been starker as Madrid were playing in their 27th semi-final, whilst City were in the last four for the first time.
That inexperience looked to be taking hold as City enjoyed decent spells of first-half possession without troubling the Real defence.
However, they came within inches of finding the crucial away goal just before the break when Kevin de Bruyne picked out Fernandinho and the Brazilian midfielder's low driven shot clipped the outside of the post.
The goalmouth action was all in City's box early in the second period as Hart was called upon to keep his side in the match.
Luka Modric spurned a huge chance to put the tie to bed when he fired straight at Hart as the Croatian seemed surprised himself not to have been flagged offside.
Hart also kept out a Ronaldo header and fierce drive from the edge of the box as Madrid pressed.
Bale then saw a looping header come off the bar as, just as in the first leg, Madrid constantly caused City problems from set-pieces.
City boss Manuel Pellegrini turned to the pace of Raheem Sterling and Kelechi Iheanacho in place of the ineffectual Yaya Toure and Jesus Navas for the final quarter.
And Pellegrini's men nearly snatched the goal they needed a minute from time when top scorer Sergio Aguero's first serious effort of the tie dipped onto the roof of the net.
Real keeper Keylor Navas then injured himself in scurrying from his goal to clear a long through ball.
Yet, despite a frantic finale, Madrid continued their record of not conceding a goal at home in the Champions League this season to progress to their 14th European Cup final.
COMMENTS
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ