The construction of the Pomezanian chapter castle, modelled on Teutonic castles, began at the turn of the 13th and 14th centuries. After work to prepare the ground, the erection of the castle wings and pre-castle buildings began. The castle was built from stones and bricks in the form of a four-winged building on a roughly square ground plan, with towers in the corners, a two-storey cloister in the courtyard and an entrance from the north, where the pre-castle utility room was located. Most of the construction work on the castle was completed between 1340 and 1350, with the Gdanisko completed in the 1480s. Originally the castle was an independent structure, only later was it connected to the cathedral, and its south-east corner tower was converted into a bell tower. The completion of the new cathedral church is dated to the reign of Bishop Jan Mönch (1377 - 1409).

During this time, a unique architectural ensemble was created in Kwidzyn, which included two castles (bishop's and chapter's), a cathedral and a town. The individual parts had their own fortifications and, connected by walls, formed a defence system perfectly integrated into the natural conditions of the area. The castle became the seat of the Pomezanian chapter, as well as a religious and political-administrative centre. Under the terms of the Second Peace of Toruń, Kwidzyn, together with the bishop's dominion, was granted to the Teutonic Order. The Pomezanian bishopric was given to the Polish Bishop of Chelmno, Wincenty Kielbas, for life. After his death, the Order attempted to fill the Pomezanian bishopric, having the support of the cathedral chapter. This led to the intervention of Polish troops in the areas belonging to the Pomezanian and Warmian bishoprics. In 1478, the Polish army captured Kwidzyn. The townspeople took refuge in the chapter's castle and set fire to the town. During the warfare, the castle was severely damaged - during the repair work carried out in 1487, the damaged corner towers were demolished. In 1520, as a result of further warfare, the Polish army completely destroyed the bishop's castle and probably damaged the chapter's castle. In the 1530s, the first Protestant bishop, Paulus Speratus, repaired the chapter castle at his own expense. After his death, in 1551, the castle was taken over by officials of Duke Albrecht Hohenzollern, becoming a government building and residence where, in 1709, Tsar Peter I of Russia stayed during his stay in Kwidzyn, at the invitation of King Frederick I of Prussia.
In 1728, the south wing of the castle was converted into a food warehouse for the military garrison. After the first partition of Poland, the castle became the seat of the court. Due to the change of function, necessary adaptation works were carried out. The halls of the first floor were divided into a number of small rooms. The western cloister was partially demolished and a staircase leading from the courtyard to the first floor was built. Part of the castle was adapted as prison cells. In 1798, it was decided to demolish the two wings of the castle: the eastern and the most representative southern wing, in order to build a separate building from the material obtained.

In 1798, it was decided to demolish the two castle wings: the eastern and the most representative southern wing, in order to build a separate building from the material obtained. After 1854, by decree of King Friedrich Wilhelm IV, the devastation of the castle was stopped and reconstruction work began. The most significant stage of this was carried out in 1874, with the corner towers being rebuilt, the vaults in the rooms of the first floor of the north wing reconstructed and the architectural detailing completed. The castle was used as a court and prison until 1935. In 1936, after a number of adaptations, the castle became the headquarters of the Hitlerjugend HJ-Ostlandführerschule, and served as such until 1945. After the Red Army occupied the town, the castle, unlike the Old Town, luckily escaped major destruction, with only the interiors being ransacked. In December 1949, the castle was taken over by the Ministry of Culture and the Arts. It now houses the Kwidzyn Museum.Since 2018, the castle and cathedral complex has had the status of a historical monument.
Tickets for the tour are available opposite the main entrance. Tours are possible with a guide or audio book. The tour of the castle lasts approximately 2/3 hours.
Parking for tourists is located just below the castle.
A short distance from the castle lies the "Old Town Restaurant".
Next to the castle is the comfortable "Hotel Maxim" for visitors.