Charles Ross Taggart was born in Washington, D.C. His mother, Emily Divoll Taggart, was a missionary working in the Freedman's Bureau in Washington. His father, John Nelson Taggart, was from Pennsylvania, a Union soldier during the Civil War, and a postal clerk. He died 5 months before Taggart was born.