Nadal, 31, saw off Argentine lucky loser Leonardo Mayer 6-7 (3/7), 6-3, 6-1, 6-4, taking his record over the world number 59 from Buenos Aires to 4-0.
He will face Ukraine's Alexandr Dolgopolov, against whom he is 6-2, for a place in the quarter-finals.
Nadal to kick off 2018 in Brisbane
Five-time champion Federer brushed aside 31st seed Feliciano Lopez 6-3, 6-3, 7-5 to take his career record against the Spaniard, playing a 63rd successive Grand Slam, to 13-0.
Next up for Federer is Philipp Kohlschreiber, who he leads 11-0.
Nadal had dropped the first set of his second-round match against Japan's Taro Daniel before winning in four.
Saturday was almost a carbon copy with the 15-time major winner unable to convert any of six points in the opener.
Montreal Masters: Nadal stunned by teenager Shapovalov
But the match turned in the 10-minute seventh game of the second set when, on his 14th break point of the encounter, Nadal finally smashed through.
"It was tough. It took 14 break points -- that's not a good figure. But I was there mentally and fought a lot at the right moments," said Nadal under the Arthur Ashe roof as rain fell outside.
Federer had needed back-to-back five-setters to defeat Frances Tiafoe and Mikhail Youzhny in his first two rounds.
However, on Saturday, he swept past Lopez to reach the fourth round for the 16th time. "I'm very happy to get through. It's a great feeling to play out here," said Federer. "I struggled in the first two rounds but tonight I played a very clean match."
Kohlschreiber, the German 33rd seed, eliminated Australia's John Millman 7-5, 6-2, 6-4.
Austrian sixth seed Dominic Thiem defeated France's Adrian Mannarino 7-5, 6-3, 6-4 to make the last 16 for the third time in four years.
Thiem next faces 2009 champion Juan Martin del Potro of Argentina, who put out Spain's Roberto Bautista Agut 6-3, 6-3, 6-4.
Russia's Andrey Rublev became the second teenager in the last 16 when he beat Bosnia's Damir Dzumhur 6-4, 6-4, 5-7, 6-4.
COMMENTS
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ