The history of the temple starts with a legendary story. Lord Brahma and Lord Vishnu got into an argument that broke into a fight about their supremacy. As a moderator, Lord Shiva took a form of light that pierced through three worlds. Shiva demanded them to find the tip of the light and the one who finds the tip of the light would be considered supreme. Vishnu took the form of a pig and dug down to find the tip and Lord Brahma rode up to find the tip. The place where Shiva’s light pierced through the earth are the 12 jyotirlingas and this temple is one among those.

The year of construction of the building is unknown. This temple has been mentioned in many ancient scripts and myths. In the 2nd AD, the temple was destroyed by invaders and was again rebuilt by a Gujarati merchant. In the 15th century, the temple was destroyed and rebuilt during the reign of Akbar, in the 16th century. Since Raja Man Singh, father-in-law of Akbar built the temple. Hindus boycotted the temple as the King let a Muslim marry into his family. In the 17th century, the temple was destroyed by Aurangzeb and a mosque was built in the spot. The remains of the ancient temple can be seen right behind the mosque.

Many rulers tried to destroy the mosque and build back the temple but failed. Later in the 18th century, the temple was built adjacent to the mosque. Many kings from different parts of the world donated fund and statues to the temple. The temple was managed by the kings of Varanasi and now is under the control of Indian Government.