PTI reports an Improvised Explosive Device (IED) has been recovered from a dustbin at the blast site.
In New Delhi, Secretary (Internal Security) U K Bansal said the explosion was a "medium intensity" blast.
In an e-mail to some media organizations, a group claiming to be the Indian Mujahideen has claimed responsibility. However, the government says investigations are on.
Karamvir Singh, the state's Director General of Police, reached Varanasi late Tuesday night. He said the police is taking the Indian Mujahideen e-mail seriously and following up all leads.
Singh added preliminary investigation suggests the explosive was kept in the wall between two ghats. A forensic report is expected in 24 hours, he said.
The explosion took place at Sheetla Ghat, near the main ghat where daily aarti or prayers are offered. The blast took place just as the aarti ended.
The Prime Minister has asked the Home Ministry for a report, and has appealed to people to stay calm. (Read: PM seeks detailed report from home ministry)
Listen to the latest songs, only on JioSaavn.com
The MP from Varanasi, the BJP's Murli Manohar Joshi, said, "Security must be strengthened at the ghats because many foreigners come to visit especially in this month." (Watch)