Imagine you were living in Manhattan during the 1880’s, ensconced in a grand mansion located on the most fashionable stretch of 5th Avenue. It was given to you by your late father, who also left you a personal fortune of 10 million dollars; half outright, and half in trust. Looking out your windows, you gazed upon the equally grand homes of your relatives and neighbors, all members of the upper echelon of New York society. Wealthy, secure, and comfortable, you couldn’t be blamed for thinking that your life could ever change.
But change was coming. Drawn by the fashionable nature of the neighborhood, which your very name helped establish, swanky hotels began to make an appearance over the next decade. Retail establishments catering to the carriage trade followed; department stores and office buildings in turn. As more commercial buildings went up around you, real estate values began to climb higher and higher (along with the annual taxes on your property). One by one your friends and neighbors, lured by the high prices offered by commercial developers and assaulted by the increasing crowds and traffic on the streets, begin to sell their homes and move farther uptown. Looking out your windows, you’re now confronted by a sea of tourists, shoppers, and office workers on their lunch break surrounding your home. What was a self respecting society matron to do? If you were one of the four Vanderbilt sisters, you had a couple of options. Their choices were influenced, if not dictated, by the type of marriage each made.
Margaret Shepard, dressed as a Marquise, at her sister in-law Alva's famous costume ball in 1883Margaret Louisa Vanderbilt Shepard was the eldest of the four sisters. She was also the first to decamp from her midtown mansion. Margaret’s husband, Elliot Fitch Shepard, was by all accounts, not a beloved figure amongst his in-laws, or many other people for that matter. Strict, arrogant, religious, and conservative, he kept Margaret and his children under his thumb. When he died in 1895, not only did he grant Margaret a new lease on life, allowing her to travel freely and enjoy time with her grown children, he had also effectively spent all her money not held in trust, on failed business ventures and runs for political office. Margaret, in the process of constructing a six hundred acre Westchester estate (today the Sleepy Hollow Country Club), when he died, suddenly found herself pressed for cash. She had no choice but to sell her home at 2 West 52nd Street.
642 Fifth Avenue and 2 West 52nd Street, the double townhouse of Mrs. Shepard and Mrs. SloaneInitially she moved just down the street to the Belgravia Apartments, located where Saks Fifth Avenue stands today. The Belgravia was one of Manhattan’s first luxury apartment buildings, offering the wealthy a low maintenance, less costly, but just as respectable alternative to owning a house in town. She later bought another apartment, in the McKim Mead and White designed 998 Fifth Avenue.
It was there that she died suddenly of a heart attack in 1924, the first of the sisters to go.
When Margaret decided to sell 2 West 52nd street, she didn’t have to look far for a buyer. The purchaser of her mansion was her next-door neighbor and sister, Emily Thorn Vanderbilt Sloane. 
Emily Vanderbilt Sloane White Emily Sloane enjoyed a charmed existence in comparison to her sister Margaret. She married William Douglas Sloane, who was active in his family’s famous home furnishings furniture business, WJ Sloane. While much of old guard New York society made a point to look down on those “in trade”; so charming was the couple (along with their children) that soon, not only were they safely ensconced in, but eventually became leading members of the fabled “Four Hundred”. Although Emily was known for a rather lighthearted attitude towards society, she still followed all of its rituals and needed a larger stage for entertaining. She combined her sister Margaret’s house with her own, creating a mansion nearly equal in size and scale to her father’s on the southern half of her block (now owned by her brother George). William Sloane died in 1915 leaving Emily, unlike her sister Margaret, richer than he found her. In 1920 Emily married for a second time, to former Ambassador Henry White. Emily White continued to enjoy life and her family, summering at Elm Court, her 93 room shingle style cottage in Lenox (still owned by her descendants, who run it as an inn), and returning to New York for the social season, sharing a box at the Metropolitan Opera with her sister Florence, hosting musicales in her enormous mansion, as well as launching her debutante granddaughters into society there in the early 1920’s. By the mid-1920’s, she could no longer ignore the changes going on around her city home however. Her brother William K Vanderbilt, whose mansion stood right across 52nd Street, died in 1920. His famed chateau was soon demolished, replaced by a pedestrian six-story office building, that greeted Mrs. White each time she went out her front door. Although her daughter Lila Sloane Field still occupied one of the Marble Twins across from her on the other side of Fifth Avenue, right next door, the mansion once occupied by the Morton Plants, was now the home of Cartier jeweler. 
The Morton Plant Mansion (before its retail conversion), and the Field residence.Emily sold her mansion to developers in 1925. Demolished in short order, the famed retailer DePinna occupied the site. With her own children and grandchildren grown, she now longer needed a residence as large as her former home. Emily did not move into an apartment building however. Instead, she bought the former Livingston Beekman residence at 854 5th Avenue, facing Central Park. 
854 Fifth Avenue While definitely smaller than her former home the 25 foot wide mansion designed by Warren and Wetmore was more than scaled for gracious living. With no financial worries, Emily shuttled between there and at Elm Court for the next two decades, maintaining her homes and lifestyle in the same grand manner, as if the Great Depression and World War Two never occurred, before dying peacefully at her Lenox estate in 1946. Her former home still stands on Fifth Avenue. Currently the home of the Serbian mission to the United Nations, it’s in need of a cleaning, but still elegant.
The next sister, Florence Adele, married Hamilton Mckown Twombly. Hamilton Twombly was a shrewd investor and astute businessman, often advising his father in law and Florence’s brothers on the management of their railroads and investments. When he died in 1910, not only did he live Florence richer than her found her, her left her richer than most of her family, no easy feat. 
Florence Vanderbilt Twombly With an income estimated at several million dollars a year, Florence, regally entertained the cream of society at Florham, her elaborate country estate in New Jersey, (now the campus of Fairleigh Dickinson University), her Newport cottage Vinland (now part of Salve Regina College), as well as her New York home at 684 Fifth Avenue.
She remains a legend to this day as one of the most formidable grande dames in New York history, known for riugidly maintaining the high standards of living, style and etiquette that had slipped elsewhere, as well as for her sharp tongue. Once, while sitting on the veranda of her sister Lila’s home in Vermont, she complained of feeling a chill. When her sister's butler appeared almost immediately, offering her one of Lila’s own shawls to wear, she supposedly took one look at the garment and snapped, “I’m not THAT cold!”
Another tale concerns a guest attending one of Mrs. Twombly’s legendary weekend house parties at Florham, over a Labor Day weekend. Finding his bags packed in the great front hall ready for his departure late Sunday afternoon, he asked a footman why, explaining that he assumed, since it was a holiday weekend, he would be staying until Monday. The footman went off to communicate this. Returning a short time later, he informed the hapless guest “ Mrs. Twombly says she as never heard of Labor Day, and that the weekend is over today, sir.” I can’t help but believe from stories like these that Florence Twombly served as one of the inspirations for the Dowager Countess Grantham’s character on Downton Abbey.
Like her sister Emily, when the noise and crowds at 54th Street and Fifth Avenue grew too much during the 1920’s, Florence moved uptown into a new house. This home however, was not smaller than her former one, but considerably larger. Florence commissioned Warren and Wetmore to design a 70-room mansion for her at Fifth Avenue and 71st street.
The house, while not considered one of Warren and Wetmore’s finest works, has a monumentality and severity that perfectly reflected its occupant. The mass and simplicity of its wide 71st street façade also acted as a nice foil to the low horizontal dignity of the Frick Mansion, which it faced. Florence lived there until 1952, when she passed away at the age of 98, maintaining her legendary lifestyle and frosty reputation up to the end. Her mansion was razed a few years later, replaced by a modern apartment building.
The youngest sister, Lila Vanderbilt
did not have to good fortune to marry a man with a successful business, like her sister Emily, or a shrewd financier like her sister Florence. She married William Seward Webb, from an old, distinguished New England family, who had a propensity for poor business decisions (and a weakness for morphine in his later years). He lavished huge sums on Shelburne Farms, their legendary agricultural estate located on the shores of Lake Champlain. Dr Webb spent prodigious amounts of money trying to develop a new breed of horse; strong enough for farm work, but attractive enough to pull a carriage. The concept might have succeeded had not the automobile and tractor come along while in the midst of his experiment. Owning a Great Camp in the Adirondacks, Dr Webb financed a new railroad (with his wife’s money) in part so he and other millionaire camp owners could reach their private preserves faster and easier. Not content with owning a private railway car (the lear jet of their day) the Webb’s had their own entire private train at their disposal. When Seward Webb died, Emily, like her sister Margaret, found all that was left of her fortune was the income from her 5 million dollar trust. Even before that though, it was obvious finances had become tighter. Shortly after World War One she sold her townhouse at 680 Fifth Avenue
to John D Rockefeller. Unlike her sisters, , Lila never owned another residence in New York City from that point on. Lila Vanderbilt Webb lived in rented Park Avenue apartments when in town, preferring to spend the majority or her time at Shelburne, or Miradero, 
her estate she had constructed in in Palm Beach until her death in 1936, .
Next, Part Two: A look at the in-laws