Square 927, a short walk from the Washington Navy Yard and in easy marching distance of the Capitol, was their choice. Construction began later that year, and Burrows' successor, Lt. Col. Franklin Wharton, completed the house in 1806 and the Barracks in 1808.
Still used for its original purpose, the Home of the Commandants has been home to all but the first two commandants, and is said to be the oldest, continuously occupied public building in Washington, D.C.

When first built, the Georgian-Federalist style house measured 25 by 32 feet. It contained four large rooms and a central hallway on each floor, a kitchen in the basement and servant's quarters in the attic.

Renovations and additions, which began in 1836, have expanded the house to 15,000 square feet, including 30 rooms not counting closets or baths. The decor has always been dictated by the personal tastes of each commandant and his family.

The Home of the Commandants was one of the few buildings not burned by the British when they sacked the Capitol in 1814. This omission by the British has given rise to several legends as to why the house was spared. One version is that Adm. Cockburn and Gen. Ross, commanding the British troops, spared it to use as their headquarters, then neglected to apply the torch upon their withdrawal.