Carr was born in Laurens, Otsego County, New York. The son of George Carr and Mary Greene Carr, he was said to be descended from Rhode Island’s colonial governor Caleb Carr. His early life is undocumented, but by 1833 he had married Hannah Burnside. They had three sons and three daughters. Carr moved his family to Virginia, where in April 1853 he purchased 380 acres of farmland near the village of San Marino in Dinwiddie County. Sometime after Hannah Carr died of cancer in January 1856, Carr returned to New York, where by 1858 he had married Susan Marlette Walker, a physician’s widow. By 1860 the couple was living on the farm in Dinwiddie. They had at least one daughter.