The castle cannot be dated to the nearest 100 years, although construction is thought to have taken place during the reigns of the late 12th century and early 13th century rulers of the Isle of Man - the Kings of Mann and the Isles. The last such king, Magnus Olafsson, is recorded in the Chronicle of Mann to have died at the castle in 1265. The original Castle Rushen consisted of a central square stone tower, or keep. The site was also fortified to guard the entrance to the Silverburn River. From its early beginnings, the castle was continually developed by successive rulers of Mann between the 13th and 16th century. The limestone walls dominated much of the surrounding landscape, serving as a point of dominance for the various rulers of the Isle of Man. By 1313, the original keep had been reinforced with towers to the west and south. In the 14th century, an east tower, gatehouses, and curtain wall were added. Although parts of the castle were destroyed by Robert the Bruce in 1313, the damage was rebuilt by William Montacute, King of Mann by the year 1344.