The current 2-acre garden is what remains of Cornelia Horsford’s grand 1908 reshaping of the property as a Colonial Revival showpiece. Over the 18th century axial plans laid out by Brinley Sylvester, who had planted the gardens for esthetic and commercial use, Cornelia cut back and transplanted the box hedges, built terraced and parterre flower beds, and, perhaps in homage of the Sylvester’s Amsterdam origins, included a sunken Dutch flower bed, accessed by a graceful staircase descending from the rose garden. In the NE corner of the former rose garden still stands the 18th century privy (or outhouse), painted the same soft yellow as the house, and paneled inside with wainscot removed from the Ladies parlor during a late 19th-century remodeling.
By 1950, Alice Hench was already a passionate gardener at her family’s Dering Harbor property when she met Andrew Fiske, and came to humorously credit their marriage on a shared love of tending plants: “Mr. Fiske came to visit one weekend and said, ‘You have some very fine boxwoods. Why don’t you bring them to Sylvester Manor?’ By Monday, we were engaged.” Thus Alice Fiske began a long and famous tenure as mistress of the manor gardens, often driving the tractor herself. She indelibly left her own mark on the Manor gardens with massive plantings of her signature daffodils and well-tended roses, while faithfully caring for the beds laid out by Cornelia in 1908.