While he played in a boogie-woogie style, with a strong-repeated figure in the left hand and melodic decoration in the right hand, his playing was delicate and subtle, rather than hard driving. He popularized a left hand figure which became known as the 'Yancey bass', and was later used in Pee Wee Crayton's "Blues After Hours", Guitar Slim's "The Things That I Used to Know" and many other songs. Part of Yancey's distinctive style was that he played in a variety of keys but ended some pieces in E flat, even if it was in another key. And he favored keys atypical for barrelhouse blues, like E flat and A flat.

Most of his recordings were of solo piano, but late in his career he also recorded with vocals by his wife, Estelle Yancey, under the billing 'Jimmy and Mama Yancey'. They appeared in concert at the Carnegie Hall in 1948. In 1951, the twosome recorded the first album that was released by Atlantic Records the following year.

During World War I, Yancey played baseball in a Negro league baseball team, the Chicago All-Americans. Throughout his life, Yancey kept a job as groundskeeper for the Chicago White Sox.

Yancey died of a stroke secondary to diabetes in Chicago on September 17, 1951. He was inducted into the Rock and Roll Hall of Fame in 1986.