The Castle of Kimolos is found in the upper town where the old village once lay. The houses are built side by side as if part of the castle wall, complete with small windows to help keep invaders at bay. Medieval houses can still be found around the abandoned yet picturesque streets of this part of town.
Due to continued pirate raids, a second wall around the first fortification was added in the 16th century. It was also rectangular in shape with a tower in each corner and two gates.
The eastern gate was built in 1646 according to ints inscription. At one point the castle had 400 small two-floor houses, sharing them with domestic animals. The back of the houses formed the castle’s external walls. Poverty reigned on Kimolos in those times due to the pirates raids.