Booker was born on December 5, 1821, in Patrick County, the son of Edward Booker and Elizabeth Anglin Booker. Educated locally, he farmed and taught school before reading law. Booker was admitted to the bar in Henry County on March 8, 1847, and practiced there the remainder of his life. Elected to the county court in 1856, he served as its presiding justice from 1858 until 1864. Booker was a strong Unionist at the time of the secession crisis, but according to his later testimony he voted for the Ordinance of Secession because he feared reprisals from his neighbors. He avoided being conscripted in 1864 because he was a justice of the peace, and the only part he took in the Civil War was the performance of his duties as a magistrate.