In July 1873, a family reunion in Scio Township merited coverage in the New York Times. That’s because former president Millard Fillmore attended the gathering at the farmhouse on Dexter-Ann Arbor Road where his brother Calvin lived with his wife, Maranda.

Millard actually held the mortgage on the 100-plus-acre spread where Calvin eked out a living from the land after constructing some of Dexter’s most significant early buildings, including Gordon Hall. But aside from getting occasional financial help, Calvin never was able to capitalize on his brother’s fame.

The brothers’ parents, Nathaniel and Phoebe Fillmore, were tenant farmers in upstate New York; they had nine children in all. Millard, their oldest son, went on to become a successful Buffalo lawyer and political leader. He had already served a term in Congress when Calvin and Maranda (nee Waldo–she was a second cousin of poet-sage Ralph Waldo Emerson) arrived in Dexter in 1835. They were both twenty-five and had been married five years.

The Fillmores came to Dexter because “it was the thing to do and land was cheap,” their great-great-grandson Nathaniel Charles Fillmore told Dexter historian Norma McAllister in a 1973 interview. Calvin was a carpenter, and the young village had plenty of work. A few months after arriving, he wrote to his brother Darius in Buffalo: “Buildings are very scarce and rent very high, that is dwellings suitable for houses. I have now on hand about $350 worth of work and have had offers enough to reach $1,000, but help is very scarce here.”

Besides houses, Fillmore built at least two churches. In 1839 he put up a frame building at Fifth and Broad, now long gone, for Dexter’s Baptist congregation. In 1841 the Methodists hired him to build a church on Central Street, complete with a partition to divide the sexes. That church was replaced in 1925 after being struck by lightning.

But Fillmore’s most impressive legacy is Gordon Hall. Between 1841 and 1843 he collaborated with the village’s founder, judge Samuel Dexter, on the twenty-two-room house, which had nine fireplaces and nine-foot-tall sliding pocket doors. “The most elaborate of mansions of this period in Michigan with the general character of the large estates in the south,” a 1934 edition of the Historic American Buildings Survey called it. The house is now being restored by the Dexter Area Historical Society.

Two of Calvin’s sisters–Julia, who marred Charles Harris, and Olive, who married Henry Johnson–followed him to Michigan. Sylvester Newkirk, who was apprenticed to Calvin and went on to work with him, married Olive’s daughter Julia.

In 1845 Calvin and Maranda bought more than 100 acres in Scio Township.

“It’s a timbered opening with a beautiful spring brook [Honey Creek] running directly through it,” he wrote to Millard. “I have cleared about 30 acres and split about 2,500 rails to fence with in the spring intending to put in a crop of wheat next summer health permitting.”

The Greek Revival house still standing at 4350 Dexter-Ann Arbor Road is almost certainly the Fillmore farmhouse. The original front part, four rooms downstairs and two bedrooms upstairs, has the wide floorboards of the period and the original hardware. Setting up a farm cost more than Calvin expected, however, and he had to ask his brother Millard to assume the mortgage.

In 1849 Millard Fillmore, a Whig Party stalwart, was inaugurated as vice-president under Zachary Taylor. Calvin evidently was not flourishing as a farmer, for he began besieging his brother with requests for government jobs, including lighthouse keeper and customs collector. He asked that his son Waldo be named postmaster of Dexter.

The appointment went instead to lawyer Alexander Crane. Millard’s unresponsiveness may have been due to intraparty squabbling that limited his powers of patronage. But even after Millard became president in 1850 on Taylor’s sudden death, his brother remained a struggling farmer. In a letter to the president, an Ann Arbor friend hinted courteously at one possible reason Calvin did not prosper. The writer described the younger Fillmore as “evidently a genius in his way, a sort of original character, strongly marked with the family characteristic of thinking, achieving, and acting what he pleases without let or hindrance from saint or sinner.”

Eventually Calvin stopped writing letters asking for help getting a job. The Whigs did not nominate Millard for the presidency in 1852, and he lost a third-party run in 1856 as the nominee of the American (Know-Nothing) Party.

Despite their various disappointments and their widely scattered residences, the Fillmore family kept in close touch. Their letters were filled with family news. In an 1867 letter to Millard, Calvin reported that his sister Olive had moved from Dexter into his house. The Newkirk branch of the family still has a letter from Millard thanking his Michigan relatives for the Thanksgiving turkey they sent to him in New York.

Calvin had always complained about his health, and as he aged his physical problems made farming harder. In 1873 he reported that he had three cows, three horses, and three acres of corn but that the cold weather was aggravating his diabetes. “I would like to go to a mild climate if it were possible,” he added, “but I do not know but I shall have to live and die here after all.”

The five surviving siblings gathered at the Scio Township farm on July 16, 1873. Calvin and Olive were joined by Julia Harris, who had moved to California; Cyrus, from Indiana; and Millard, who was seventy-three. The New York Times reported that “all were in good health, hale and hearty, and to all appearances bade fair to yet see many years of enjoyment and happiness. Besides the above there were present eight or ten others, and the gathering as a family around the festive board was an occasion of joy and happiness, not wholly unmingled with sad memories.”

In a letter sent shortly after Millard left Michigan, Calvin wrote, “We regret that you did not stay a little longer with us but feel very happy for the reunion and hope it [did] not make you sick.” He also thanked his brother for a gift of $50. In his next letter Calvin enclosed five copies of a photo of the family taken by Sam Revenaugh, who ran a photography studio on East Huron Street in Ann Arbor.

In September 1873 Calvin wrote, “I am nearly worn out with my harvest and other work.” In October he asked Millard to redeem some bonds so that he could fix up the house for winter. In the rest of his letters from fall 1873 he explained that he was working on the house, plastering and painting. “I have an offer for my farm, but not what I want for it,” he added. Yet he soon sold the house and land and moved to Ann Arbor.

On March 8, 1874, Millard Fillmore died after suffering a stroke. In his will, Millard canceled all of Calvin’s debts to him and bequeathed his brother $500.

Calvin survived five more years. In January 1879 the Ann Arbor Courier-Weekly reported on his last trip: “On a recent visit to his brother, who lives in Indiana, they rode fourteen miles one very cold day, and the deceased froze his feet very badly. As a direct result gangrene set in, which soon poisoned his blood, causing his death.” He is buried in Forest Lawn Cemetery in Dexter, along with his wife and his sisters Julia and Olive.

Besides the buildings he constructed, Dexter retains one other Calvin Fillmore souvenir. When Pat and Paul Cousins were running Cousins Heritage Inn, Nathaniel Charles Fillmore often ate there. He told them that Calvin had built the house the restaurant occupied (it’s now Terry B’s).

A few years ago, when Nathaniel was moving to a nursing home, he called Paul Cousins and arranged to give him an old toolbox. “It was the toolbox that Calvin put on his wagon and moved from spot to spot,” says Cousins. “You can see where the tools fit in, where the planes were.” The toolbox is now in the Dexter Area Historical Museum.