World number 17 Khachanov clinched the point for Russia by fighting back to beat Croatian number one Borna Coric 6-7 (4/7), 6-4, 6-4, after Rublev had earlier seen off Borna Gojo 6-3, 6-3.
Russia, who are without US Open runner-up Daniil Medvedev due to exhaustion, can wrap up a place in the quarter-finals with victory over Rafael Nadal's Spain in their second Group B tie on Tuesday.
Murray will do 'great job' at Davis Cup, says British captain
The 18 nations have been split into six groups of three at the Caja Magica, with the pool winners and two best runners up to play in the last eight of the first edition of the revamped tournament.
The teams face off in two best-of-three-set singles matches and one doubles rubber in each tie.
Khachanov and Rublev will take on Ivan Dodig and Nikola Mektic in the doubles later on Monday, with every rubber possibly crucial in case teams finish level on points.
Canada claimed victory over well-fancied Italy to strike first in Group F.
World number 12 Fabio Fognini slumped to a 7-6 (7/5), 7-5 defeat by Vasek Pospisil, ranked 150th, before rising star Denis Shapovalov edged out Matteo Berrettini 7-6 (7/5), 6-7 (3/7), 7-6 (7/5).
"I'm really happy to get the win against a player like Matteo who has had such a good season," said Shapovalov.
"I always love representing my country... It's not often you get to play as a team, so it's amazing."
Shapovalov and Pospisil were due to face Simone Bolelli and Lorenzo Sonego in the doubles.
Italy, who won their only Davis Cup title in 1976, were the favourites in Group F with Fognini and Berrettini both having enjoyed excellent seasons.
The 20-year-old Shapovalov saved three set points before snatching the first set in a tie-break, only for Berrettini to level the match in a second-set breaker.
Won't play if Davis Cup tie against India not shifted back to Pakistan: Aisamul Haq
A clash without a single break of serve, was decided by another tie-break, with Shapovalov reeling off the last three points to win after two hours and 52 minutes on court.
Canada take on the United States on Tuesday, with Italy facing the Americans the following day.
Belgium made a victorious start in Group D with back-to-back singles wins against Colombia.
The South Americans are playing at the top level of Davis Cup tennis for the first time, having never played in the old World Group.
Steve Darcis saw off Santiago Giraldo 6-3, 6-2, before world number 11 David Goffin had to battle back from a set down to avoid a shock against 194th-ranked Daniel Elahi Galan Riveros, winning 3-6, 6-3, 6-3.
COMMENTS
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ