The violence broke out on Monday night when police fired tear gas at a group of holy men known as sadhus who were protesting at the authorities' refusal to allow them to immerse idols in the River Ganges last month.
A court recently banned the immersions to try to curb pollution of the holy river, but the sadhus say the ruling contravenes their religion.
Read: Three feared dead in violence in India's northeast
Officials in the city, which is Prime Minister Narendra Modi's parliamentary seat, said the situation soon escalated as a mob went on the rampage through the streets.
Varanasi is the holiest city in Hinduism and one of the world's oldest cities.
Akash Kulhari, a senior superintendent of police, said 40 officers and eight civilians had been injured in the clashes.
"These rioters damaged 20 two-wheelers, seven vehicles of district administration and police, apart from torching a police booth-cum-tourist facilitation centre and the CCTV unit there," he said.
Rajmani Yadav, the district magistrate for Varanasi, said several of those injured were in critical condition although he said the situation had now been brought under control.
Read: Rising clashes: India witnesses 25% rise in communal violence
An overnight curfew had now been lifted but extra security forces were out on the streets.
"Security forces are keeping a strict vigil to prevent any untoward incident," Yadav told AFP.
Varanasi, in the east of the giant state of Uttar Pradesh, is a major pilgrimage site for Hindus who often cremate their relatives there before scattering their ashes in the Ganges.
While other parts of Uttar Pradesh are regularly blighted by violence between Hindus and Muslims, violence is extremely rare in Varanasi.
COMMENTS (3)
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ