Chet Singh (1770-1781) built the palatial building of Chet Singh Ghat in mid the 18th century as a small fortress, which witnessed the fierce battle between the troops of Warren Hastings and Chet Singh in 1781 that resulted to the defeat of Chet Singh. Thus this fortress went under the control of British. In late 19th century the King Prabhunarayan Singh had again took the possession of this fort. The northern part of it was donated to Naga group of ascetics who late on built their monasteries and ghats, called Niranjani Ghat and Nirvani Ghat.
This palace has been his principal residence. This building composed of (a) a palace with pavilions, built on the terrace overlooking the Ganga, (b) a group of buildings for the women (demolished), and (c) a Mughal garden with darbar and water tower. The palace has a particularly favoured relationship to the Ganga. It opens out onto the ghat which are a continuation of the palace and reached by means of a monumental gateways. The gateway houses a stairway, which gives access to the terrace. There, a central pavilion stands looking out over the Ganga, on which the Maharaja appeared for glimpse. The terraced level is defined at two corners by two massive structures tapped by octagonal domed pavilions. There are three state temples of Shiva in the compound, built in 18th-19th century.