Hendricks made 102 off 89 deliveries to help South Africa post 363-7 before they bowled out the hosts for 285 in 45.2 overs and took an unassailable 3-0 lead in the five-match series.
Dhananjaya de Silva did try to infuse life into the chase with a career-best 84 but Sri Lanka's batting crumbled under the ever-increasing run-rate.
Virat Kohli dethrones Steve Smith to become top ranked Test batsman
Fast bowler Lungi Ngidi claimed four wickets while fellow paceman Andile Phehlukwayo took three.
But it was Hendricks, a right-hand batsman who has played 12 Twenty20 internationals, who set up the win for his side.
The 28-year-old became only the third South African batsman to register a hundred in his first ODI. Colin Ingram and Temba Bavuma are the other two.
Jean-Paul Duminy's 92 off 70 balls, and half-centuries by Hashim Amla and David Miller, also contributed to South Africa's imposing total after being put into bat first.
Amla gave South Africa a brisk start with his run-a-ball 59 before being bowled by medium-pace bowler Thisara Perera, who returned impressive figures of 4-44.
Hendricks, who has been around since 2014 when he first played a T20 against Australia, built crucial partnerships including a 78-run fourth-wicket stand with Duminy.
Cricket won’t be on Imran Khan’s mind: Mushtaq Mohammad
Hendricks celebrated his century with a boundary off paceman Lahiru Kumara in just 88 balls, but was dismissed on the very next delivery.
Duminy, who got his second fifty of the series to lead the batting chart with 177 runs, carried on the momentum with a blazing knock laced with 8 fours and 4 sixes.
Duminy got support from fellow left-hand batsman Miller as the two put on 103 runs for the fifth wicket to pulverise the the Sri Lankan bowling.
Miller's quickfire 51 and Andile Phehlukwayo's unbeaten 24 off 11 balls gave South Africa 98 runs in the last seven overs.
In reply, Sri Lanka were never in the big chase with South African bowlers getting wickets at regular intervals.
A 95-run, seventh-wicket partnership between de Silva and Akila Dananjaya, who scored 37, raised Sri Lankan hopes of making a fist of their target.
But Phehlukwayo broke the dangerous-looking stand after dismissing Dananjaya and then got de Silva caught behind to flatten the Sri Lankan chase.
The fourth match is slated for Wednesday at Pallekele.
COMMENTS
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ