Son of William Boyd, 4th Earl of Kilmarnock (d. 1746), and Anne Livingston. He married, firstly, Rebecca Lockhart, daughter of Alexander Lockhart, Lord Covington, with whom he had a daughter. He married, secondly, Isabella Carr (1747-1808) with whom he had twelve children.
In 1758, he succeeded his maternal great-aunt, Mary Hay, 14th Countess of Erroll as the 15th Earl of Erroll, also changing his surname from Boyd to Hay.
Boswell met the Earl in the evening of August 24, 1773, as he and Dr Johnson visited Slains Castle, the seat of the Earl, north of Aberdeen. They had spent most of the day there on an invitation from Charles Boyd, the Earl's younger brother, and were invited to stay the night. The Earl arrived home around nine in the evening, together with a Captain Gordon of Park1 , and they had a late supper.
Boswell later wrote that he was "was excessively pleased with Lord Erroll. His stately person and agreeable countenance, with the most unaffected affability, gave me high satisfaction," adding that "I could with the most perfect honesty expatiate on Lord Erroll’s good qualities as if I was bribed to do it. His agreeable look and softness of address relieved that awe which his majestic person and the idea of his being Lord High Constable of Scotland would have inspired."
- 1Probably John Gordon (d. 1781), a son of Sir James Gordon, 2nd Bart of Park (d. 1727) and Margaret Elphinstone.