Built in 1119 by Robert de Crèvecœur to replace the earlier Saxon manor of Esledes, the castle became a royal palace in 1278 for King Edward I of England and his queen, Eleanor of Castile. Major improvements were made during his time, including the barbican, made up of three parts, each with its own entrance, drawbridge, gateway and portcullis.
The castle was captured on 31 October 1321 by the forces of Edward II from Margaret de Clare, Baroness Badlesmere, wife of the castle's constable, Bartholomew de Badlesmere, 1st Baron Badlesmere who had left her in charge during his absence. The King had besieged Leeds after she had refused Edward's consort Isabella of France admittance in her husband's absence; when the latter had sought to force an entry, Lady Badlesmere had instructed her archers to fire upon the Queen and her party, six of whom were killed. Lady Badlesmere was taken and kept prisoner in the Tower of London until November 1322.
Richard II's first wife, Anne of Bohemia, spent the winter of 1381 at the castle on her way to be married to the king. In 1395, King Richard II received the French chronicler Jean Froissart there, as Froissart described in his Chronicles.
Henry VIII transformed the castle in 1512 for his first wife, Catherine of Aragon. A painting commemorating his meeting with Francis I of France still hangs there. His daughter, Queen Elizabeth I was imprisoned in the castle for a time before her coronation.