Sir David was born in Glasgow and educated in his local comprehensive school. He took degrees at the University of Glasgow and obtained his PGCE from Jordanhill College of Education.

In a career spanning over 40 years, Sir David has held a number of major posts across the education system. Between 2012 and 2018, he was Vice-Chancellor at the University of Reading. Previously, he was Permanent Secretary at the Department for Education for six years, serving four Secretaries of State and three Prime Ministers. He served as Her Majesty's Chief Inspector of Schools for nearly four years and was Director of Education and Libraries with Newcastle City Council in the 1990s. He was also a Harkness Fellow based in Atlanta, Georgia for a year.

In 2020, Sir David was made a Deputy Lieutenant of the County of Tyne and Wear. In the same year, he became the non-executive chair of Karbon Homes, a large social housing provider in the North-East of England and Yorkshire.

Among other non-executive roles, Sir David is a trustee and advisory board member of the Higher Education Policy Institute (HEPI) and a governor of the Royal Shakespeare Company (RSC).