Eighteen people died and dozens were missing around the wine-growing region of Ahrweiler, in Rhineland-Palatinate state, police said, after the Ahr river that flows into the Rhine rose and brought down half a dozen houses.
Another 15 people died in the Euskirchen region south of the city of Bonn, authorities said. In Belgium, two men died due to the torrential rain and a 15-year-old girl was missing after being swept away by a swollen river.
Hundreds of soldiers were helping police with the rescue efforts in Germany, using tanks to clear roads of landslides and fallen trees, while helicopters winched those stranded on rooftops to safety.
In Ahrweiler, two wrecked cars leant steeply against either side of the town's stone gate and locals used snow shovels and brooms to sweep mud from their homes and shops after the floodwaters receded.
"I was totally surprised. I had thought that water would come in here one day, but nothing like this," local man Michael Ahrend told Reuters TV. "This isn't a war - it's simply nature hitting out. Finally, we should start paying attention to it."
The floods have caused Germany's worst mass loss of life in years. Flooding in 2002 killed 21 people in eastern Germany and more than 100 across the wider central European region.
Chancellor Angela Merkel expressed her dismay.
"I am shocked by the catastrophe that so many people in the flood areas have to endure. My sympathy goes out to the families of the dead and missing."
Armin Laschet, the conservative candidate to succeed Merkel as chancellor at a general election in September and the premier of the hard-hit state of North Rhine Westphalia, blamed the extreme weather on global warming.
"We will be faced with such events over and over, and that means we need to speed up climate protection measures, on European, federal and global levels, because climate change isn’t confined to one state," he said during a visit to the area.
Climate and environmental issues are one of the main battlegrounds in the election campaign, in which Laschet is going head-to-head with Social Democrat candidate Olaf Scholz and Annalena Baerbock of the Greens.
In Belgium, around 10 houses collapsed in Pepinster after the river Vesdre flooded the eastern town and residents were evacuated from more than 1,000 homes.
The rain also caused severe disruption to public transport, with high-speed Thalys train services to Germany cancelled. Traffic on the river Meuse is also suspended as the major Belgian waterway threatened to breach its banks.
Downstream in the Netherlands, flooding rivers damaged many houses in the southern province of Limburg, where several care homes were evacuated.
In addition to the fatalities in the Euskirchen region, another nine people, including two firefighters, died elsewhere in North Rhine-Westphalia.
In the town of Schuld, houses were reduced to piles of debris and broken beams. Roads were blocked by wreckage and fallen trees.
"What I experienced? It was catastrophic," said 65-year-old pensioner Edgar Gillessen, whose parental home had been damaged.
"All these people living here, I know them all. I feel so sorry for them, they've lost everything. A friend had a workshop over there, nothing standing, the bakery, the butcher, it's all gone. It's scary. Unimaginable."
Further down the Rhine river, the heaviest rainfall ever measured over 24 hours caused flooding in cities including Cologne and Hagen, while in Leverkusen 400 people had to be evacuated from a hospital.
In Wuppertal, known for its overhead railway, locals said their cellars had been flooded and power cut off. "I can't even guess at how much the damage will be," said Karl-Heinz Sammann, owner of the Kitchen Club discotheque.
Weather experts said that rain in the region over the past 24 hours had been unprecedented, as a near-stationary low-pressure weather system also caused sustained local downpours to the west in France, Belgium and the Netherlands.
Rainwater draining into the Rhine, where shipping traffic was partly suspended, was expected to test flood defences along the river, including in Cologne, on the lower Rhine, and Koblenz, where the Rhine and Moselle merge.
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ