This property was once owned by John H. Abbot and comprised two parcels; one of 9 acres on the north end, once owned by John Abbot, and a second parcel of about 3 acres on the south end of David P. Abbot of which John H. Abbot was an heir. John H. & wife Jessie L. Abbot were living in Big Spring, Nebraska when he sold the 12 acre lot to Frances A. Flint, wife of John H. Flint of Andover, MA. Frances Flint held the vacant property for a year before selling to Susanna Ward Smith. Susanna W. was born on Dec. 12, 1843, the daughter of Peter & Esther (Ward) Smith. Her father was one of the founders of the Smith & Dove Manufacturing Co. and one of the wealthiest families in Andover. Susanna’s father died in 1880 and her mother died in May 1889. Her inheritance may have funded the construction of her house on the southwest corner of Abbot and Phillips Street (now 50 Phillips St).
On June 5, 1895 Susanna W. Smith sold off a portion of her property (99267 sq. feet) to the south of her home to T. Dennie Thomson. Thomson then hired Boston architect James T. Kelley to design the Colonial Revival home.

The house was sited on the rise just above the former Andover & Wilmington Railroad line (1838-1848) which once ran along the south side of Abbot Street and crossed the road at this intersection of Phillips St. The old railroad bed acted as a long lane for pedestrians to stroll on their way to Pomp’s Pond or Spring Grove Cemetery and is now a portion of the driveway leading up to the house.

Thomas Dennie Thompson was born September 1849 in Brooklyn, NY, son of Rev. James & Jeannette G. (Gunning) Thompson. Dennie, as he was known by, became a Linen Importer with his business on Devonshire St. in Boston. He married in Andover on Oct. 29, 1874 to Abbie Cummings Locke b. July 8, 1852 in Melrose, MA dau. of Samuel B. & Annie Locke. The 1885 Directory has them living on Main Street at the corner of Punchard Ave. Later they rented on Central St prior to 54 Abbot St. Dennie and Abbie had four children: Eleanor Jaffrey b. Mar. 24, 1878, Phillip Wingate b. Apr. 6, 1880, Clara Locke b. Mar. 3o, 1882 and Rosemond Means b. Sept. 3, 1884.

When the house was completed Dennie placed the property into his wife's name Abby L. Thomson. This was a common practice to protect the wife and family in the event of financial ruin and creditors. In 1905 daughter Clara married Lieut. Charles M. Blackford of Staunton, VA. Daughter Eleanor married in 1908 to Alfred L. Castle. In 1924 Abby Thomson purchased an additional 6 3/10 acres from the estate of Susanna W. Smith. Dennie died on Sept. 10, 1929 and Abbie in 1937. Both are interred at Christ Church Cemetery. The Thomson family had owned for 49 years when the property was sold by son Phillip W. Thomson to Henry C. & Marion L. Morrison of Chicago, IL on Aug. 14, 1944. Sadly Henry died seven months later on March 19, 1945. Wife Marion remained until 1948 when she sold the property to Alfred L. & Eleanor (Thompson) Castle on Aug. 2, 1948. A boundary line dispute between the 50 Phillips St and 54 Abbot St properties was settled in 1950. The Chapin and Castle families facilitated an agreement on the boundary line on May 1, 1950 recorded in b. 734 p. 585.

The Castle family owned the property for 36 years then selling on April 27,1984 to Raymond & Joan Cannon who continue as owners of record in 2017.