Presidents in Retirement: Millard Fillmore
Millard Fillmore was the second Vice-President to become President upon the death of the incumbent. He would turn out to be the last President from the Whig Party and despite his desire to hold on to the job and run for re-election, his party had ideas. The Compromise of 1850 that Fillmore had supported made him unpopular in the north, especially because of the Fugitive Slave Act, while Southerners saw Winfield Scott, who came from Virginia, to be a more palatable alternative as the Whig candidate for President. The Whigs lost the election of 1852 to Franklin Pierce.
Fillmore returned to private life without significant financial means. He did not have any pension to anticipate, and so he needed to go back to work practicing law. His friend and former law partner Judge Nathan Hall assured him it would be proper for him to practice law in the higher courts of New York, and that's how Fillmore planned to make money. But personal tragedy interrupted Fillmore's plans.
Millard and Abigail Fillmore had planned to take a tour of the South after they had left the White House. However Abigail caught a cold at President Pierce's inauguration, and the cold soon turned into pneumonia, so severe that it claimed her life. Abigail died in Washington on March 30, 1853. Fillmore was deeply saddened by the loss of his wife. He returned to Buffalo for her burial. Grieving the loss of his wife, he limited his social activities, and lived off of a modest income from his investments. His grief was compounded just over a year later when, on July 26, 1854, his only daughter, Mary, died from cholera.
The political climate had began to heat up in early 1854, as a debate over Senator Stephen Douglas's Kansas-Nebraska bill divided the nation, with the issue of slavery coming to a boil. The bill opened the northern portion of land obtained in the Louisiana Purchase for settlement. The Missouri Compromise of 1820 had excluded slavery from all remaining lands of the Louisiana Purchase north of the the 36" 30' parallel of latitude, but now this was being challenged. Fillmore decided to embark on a national tour with a goal to rally disaffected Whig politicians to preserve the Union. Privately he also wanted them to back him in a run for president. Fillmore made public appearances opening railroads and visiting the grave of Senator Henry Clay and appeared at other non-political functions, but his real motive was to meet with politicians privately to further his goal.
The Whig Party was badly divided as a result of the Kansas-Nebraska bill, which had passed with the support of President Pierce. Many northern opponents of slavery, such as William H, Seward of New York, had joined the newly formed Republican Party, but Fillmore's beliefs and those of the party were incompatible. Fillmore opposed slavery in principle. but was willing to tolerate it to keep the Union together. Republicans resented Fillmore for his support of the Fugitive Slave Act.
In the early 1850s, there was considerable hostility toward immigrants, especially those from countries with large Catholic populations. Many of these people had arrived in the United States in large numbers. They faced opposition from several nativist organizations. By 1854 many of these groups had united to become the American Party. This party became known as the Know Nothing Party because in its early days, members were sworn to keep its deliberations private. If asked, they were to say they knew nothing about them. Many former Fillmore supporters from the Whigs had joined the Know Nothings by 1854 and held influence in the organization. They promoted other causes in addition to nativism. Fillmore was encouraged by the success that the Know Nothing Party had in the 1854 midterm elections. The party had won elections in several states of the Northeast and showed strength in the South. To court their favor, on January 1, 1855, Fillmore wrote a letter, intended for publication, that warned against immigrant influence in American elections. He soon joined the party.
Later in 1855, Fillmore traveled across the Atlantic, making it known that since he lacked office he might as well travel. But the trip followed the advice of political friends, who felt that by touring he would avoid involvement in the contentious issues of the day, such as fallout from Kansas-Nebraska. He spent over a year, from March 1855 to June 1856, in Europe and the Middle East. According to one account, Queen Victoria is said to have called Fillmore the handsomest man she had ever seen. Fillmore was in London at the same time as another former President, Martin Van Buren, and the two ex-presidents appeared together in the gallery of the House of Commons.
Fillmore had maintained a correspondence with social activist Dorothea Dix, who had preceded him to Europe. She was lobbying to improve conditions for the mentally ill. The two continued to correspond and met several times. In Rome, Fillmore had an audience with Pope Pius IX. He had considered the pros and cons of meeting with the Pope and almost cancelled the visit when he was told that it was protocol for him to have to kneel and kiss the Pope's hand. To avoid that, the Pope remained seated throughout the meeting.
Meanwhile back home, Fillmore's allies had sufficient control of the American Party and arranged for him to get the party's presidential nomination while he was in Europe. At the American Party's nominating convention, delegates chose a ticket of Fillmore and Andrew Jackson Donelson of Kentucky, who was the nephew by marriage and former ward of President Jackson.
Fillmore returned home from his trip in June 1856 and spoke at a series of welcoming events. The first was at his arrival at a huge reception in New York City these continued across the state until his return to Buffalo. The addresses were said to be an expressions of thanks and not as campaign speeches, but Fillmore used these occasions to talk politics. He warned his audiences that electing the Republican candidate, former California senator John C. Frémont would divide the Union and lead to civil war. (He was probably correct since this is what happened when Abraham Lincoln was elected four years later.) Both Fillmore and the Democratic candidate, former Pennsylvania senator James Buchanan, took the position that slavery was a matter for the states, not the federal government. Fillmore almost never spoke about the immigration question. Instead he called for preservation of the Union.
Once Fillmore was back home in Buffalo, his campaign stalled through the summer and the fall of 1856. Political fixers who had been Whigs were now members of the Republican Party, and the Know Nothing Party lacked experience in electoral politics. Fillmore's pro-Union stance was heard by only a small audience and although the South was friendly towards Fillmore, many people feared that a Frémont victory would lead to secession. Not wanting to split the vote, they supported Buchanan.
Buchanan won the election with 1,836,072 votes (45.3%) and 174 electoral votes to Frémont's 1,342,345 votes (33.1%) and 114 electoral votes. Fillmore and Donelson finished third by winning 873,053 votes (21.6%) and carrying only the state of Maryland and its eight electoral votes.
Fillmore's defeat ended his political career. He planned to return to the practice of law to support himself, but his financial worries ended on February 10, 1858, when he married Caroline McIntosh, a wealthy widow. They purchased a large house on Niagara Square in Buffalo, where they lived for the remainder of his life. The couple generously supported many local causes including the Buffalo General Hospital, which Fillmore helped found.
In the 186o Presidential Election, Fillmore voted for Senator Douglas, the Democratic nominee. After Abraham Lincoln was elected, Fillmore decided to stay out of the secession crisis that followed. He was critical of Buchanan's inaction as states left the Union. When Lincoln came to Buffalo en route to his inauguration, Fillmore led the committee to greet the President-elect, and hosted him at his mansion, and took him to church.
Once the Civil War began, Fillmore supported Lincoln's efforts to preserve the Union. Though now over 60 years of age, Fillmore commanded the Union Continentals, a corps of home guards from upstate New York, tasked with defending Buffalo in the event of a Confederate attack. They performed military drills and ceremonial functions at parades, funerals, and other events. The Union Continentals guarded Lincoln's funeral train in Buffalo. They continued operations after the war, and Fillmore remained active with them almost until his death.
As the war continued, Fillmore gave a speech in early 1864 calling for magnanimity toward the South after the war, noting its heavy cost, both in finances and in blood. The Lincoln administration saw the speech as an attack on it and Fillmore was criticized in many newspapers and was called a Copperhead and a traitor. In the 1864 Presidential election, Fillmore supported the Democratic nominee, Georhe McClellan. When Lincoln was assassinate din April 1865, black ink was thrown on Fillmore's house because it was not draped in mourning like others. Fillmore was out of town at the time and put black drapes in the windows once he returned. He was among those selected to escort the body when Lincoln's funeral train passed through Buffalo
Fillmore remained active in local affairs. He aided Buffalo in becoming the third American city to have a permanent art gallery, and as well as in the creation of its university. Fillmore served as the school's first chancellor, a position he held until his death. Many decades later, that institution would later remove Fillmore's name off many of its buildings because of his role in supporting the Fugitive Slave law. Beginning in 2019, the University of Buffalo took steps to distance the institution from Fillmore and ceased its co-sponsorship of his annual gravesite ceremony. A year later, Fillmore's name was removed from the Millard Fillmore Academic Center.
Fillmore remained in good health almost to the end of his life. He suffered a stroke in February 1874, and died on March 8, at age 74, after suffering a second stroke. Two days later, he was buried at Forest Lawn Cemetery in Buffalo.