Dramatic footage on local television and social media showed one crumbled building that had fallen onto its side and another lurching dramatically after the quake.
Formosa TV reported that there were 150 trapped in one 16-storey building with those inside crying for help.
Officials said that 69 had been rescued but could not confirm how many were trapped inside.
At least 30 people have been rescued from another residential seven-storey block that collapsed, with no fatalities so far.
"Four buildings have collapsed in Tainan City area. Search and rescue is under way and no casualties have been reported at this moment," Lin Kuan-cheng, spokesman for the National Fire Agency, told AFP.
The shallow quake struck at a depth of 10 kilometres (six miles) around 2000 GMT Friday, according to the US Geological Survey, 39 kilometres northeast of Kaohsiung, the second-largest city on the island and an important port.
The quake was initially reported as having a magnitude of 6.7, but was downgraded to 6.4.
The Pacific Tsunami Warning Center said a destructive Pacific-wide tsunami was not expected.
Taiwan lies near the junction of two tectonic plates and is regularly hit by earthquakes.
A strong 6.3-magnitude quake which hit central Taiwan in June 2013 killed four people and caused widespread landslides.
A 7.6-magnitude quake struck the island in September 1999 and killed around 2,400 people.
COMMENTS
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ