A Celtic monastery was first established on the island in the sixth century, and the island thrived during the Middle Ages. Following the Norman Conquest, Robert fitz Martin, Lord of Cemais, gave the island to his mother Geva. In the 12th century, a priory was established there by Tironensian monks as a daughter house of St. Dogmaels Abbey, and lasted to the Dissolution of the Monasteries in 1536.The current abbey was built in 1910 by Anglican Benedictine monks but financial problems beginning in 1925 led to the 1929 purchase of the property by Belgian Cistercians. It is considered to be the most complete example of the Arts and Crafts style in the country, and was the largest project of John Coates Carter. At the time of building, the abbey was called quot;the greatest phenomenon in the Anglican community at the present timequot;. The roofs are of white roughcast with red tiling, and the abbey church has five side-windows and on the south a quot;taperingquot; tower with primitive crenellations. Caldey Island and Little Caldey Island has been an ecclesiastical district for as long as the locals can remember, with 20 Cistercian monks living at the monastery as of 2007.The Census Reports of the County of Pembroke record it as an ecclesiastical district for administrative purposes.