Kevin Krawietz and Tim Puetz beat British doubles pair Joe Salisbury and Neal Skupski in two tight sets to send Germany through to the Davis Cup semi-finals on Tuesday.
The Germans saved four set points in a tense opening tie-break in the decisive rubber in Innsbruck before edging in front.
Salisbury and Skupski looked destined to force a third set after surging 5-0 ahead in another tie-break, but Krawietz and Puetz won the final seven points to book a clash with Russia or Sweden in Madrid on Saturday.
"It's great, it's an unbelievable feeling," said German captain Michael Kohlmann.
"We had super team chemistry. We had a tough loss today and on Sunday and this team is capable of getting over these unexpected things."
Dan Evans had swatted Peter Gojowczyk aside 6-2, 6-1 in just 55 minutes to give Britain, playing without brothers Andy and Jamie Murray, the lead.
But Jan-Lennard Struff defeated British number one Cameron Norrie 7-6 (8/6), 3-6, 6-2 to pull Germany level at 1-1.
World number 51 Struff saved two set points in the opener and broke the 12th-ranked Norrie twice in the decider to set up a winner-takes-all showdown in the doubles.
"I'm very happy the way I played today, Cam Norrie is a very tough competitor, who is fighting very hard," said Struff.
"It's tough - I had the feeling I had to put the pressure on him - I made it today."
Norrie fell 4-1 behind in the first set but broke twice and was serving for it at 6-5. Struff, Germany's top singles player in the absence of Alexander Zverev, promptly hit back to take the set to a tie-break.
Indian Wells champion Norrie led 6-4 only to see Struff reel off the final four points to snatch the first set.
A single break earned Norrie the second set, but Struff kept Germany in the competition by closing out victory with four games in a row.
Earlier, world number 25 Evans won the first four games of the match against Gojowczyk and broke five times to complete a convincing victory.
Gojowczyk, ranked 86th, was given the nod ahead of Dominik Koepfer for the opening rubber but struggled badly on serve and double-faulted four times in two games as Evans raced into a 4-0 lead.
Evans broke three more times in a one-sided second set, rebounding well from a shock loss to Czech world number 143 Tomas Machac in the group stage on Sunday.
Croatia became the first team to reach the semi-finals on Monday by defeating Italy 2-1. They await the winner of Wednesday's quarter-final between Novak Djokovic's Serbia and Kazakhstan.
COMMENTS
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ