Son of Sir Michael Stewart (ca. 1712-1796), 3rd of Blackhall, and Helen (or Helenora) Houstoun (d. 1746). Married (1765) to Margaret Porterfield, daughter of Boyd Porterfield of Porterfield (1724-1795) and Christian Cunningham. Brother of Archibald Stewart (d. 1779), who was Boswell's host for a short while when he first arrived in Holland, 1763. Father of Sir Michael Shaw-Stewart, 5th Bart of Greenock and Blackhall.
Boswell and Andrew Erskine dined with Houstoun Stewart in London on April 29, 1763.