Asa Waters II was born in 1769 in the section of Sutton which later became Millbury in 1813. He learned the art of gunsmithing in his father’s shop. Because of lucrative government contracts, the armory, which he and his brother founded, prospered. He was founder of the Millbury Bank and became a State Representative in 1823.
His plan to erect the stateliest residence in the country for his wife, Susan Holman Waters, was set in motion in 1824. Two years were required to gather the necessary materials for the Mansion. This period pre-dated the Blackstone Canal and the railroad. All imported goods had to be carted from the port of Boston by horse and wagon in Millbury. These included hard pine from the south, marble from Italy, mahogany from Central America, bricks from Baltimore and "pumpkin pine" from Maine.

The Blackstone Valley National Heritage Corridor Commission lists the Asa Waters Mansion as "the finest example of Georgian Colonial style in the country." The Mansion is not only one of the most beautiful buildings in Millbury, it has a proud history –part of the "Underground Railroad" during the Civil War, it is registered on the National and State Registers of Historic Places.