Thousands had packed into the Hindu temple in Kerala state's Kollam district on Saturday night for the show when a stray firework apparently landed on a stockpile, triggering a huge blast that tore through concrete buildings.
"Five workers have been detained for questioning," Kollam police chief P Prakash told AFP by phone.
Fireworks gone awry: Temple blaze kills over 100 devotees in India
"These are not formal arrests. Once they are questioned, only then we will know their involvement and take further steps."
Police will investigate who was responsible for the fireworks display going ahead even though local authorities had refused permission, he said.
The Kerala government has already ordered a judicial inquiry into the disaster and police have registered a criminal case against temple authorities.
Footage of the disaster which occurred in the early hours of Sunday showed a series of explosions with flames and huge plumes of smoke.
India temple fire kills over 100 people watching fireworks
Firefighters and police battled to douse the fire and rescue those trapped at the complex, but some victims were charred beyond recognition and more than 30 of the 106 dead have yet to be identified.
Prime Minister Narendra Modi rushed to the scene and a team of specialist doctors were deployed from New Delhi to treat the horrific burn injuries.
Witnesses told how the force of the explosion sent concrete slabs and other building parts slamming into the crowd.
Local lawmaker NK Premachandran said a fireworks competition traditionally held every year between different groups had been staged at the temple, drawing thousands to watch.
One firework landed on a building that was storing the rest of the pyrotechnic material, triggering the explosion, he said.
Fires and stampedes are not uncommon at temples and during religious occasions, often because of poor security arrangements and lax safety standards.
COMMENTS
Comments are moderated and generally will be posted if they are on-topic and not abusive.
For more information, please see our Comments FAQ