Oliver Nelson began playing piano at age six and picked up the saxophone when he was eleven. Later he continued his musical education at Washington University in St. Louis, as well as studying with composer Elliott Carter. Nelson gained practical experience by playing in bands with Erskine Hawkins, Louie Bellson, Quincy Jones and Duke Ellington.