Tyner's first main exposure came with Benny Golson, being the first pianist in Golson's and Art Farmer's legendary Jazztet (1960). After departing the Jazztet, Tyner joined Coltrane's group in 1960 during its extended run at the Jazz Gallery, replacing Steve Kuhn. (Coltrane had known Tyner for a while in Philadelphia, and featured one of the pianist's compositions, "The Believer", as early as 1958.) He appeared on the saxophonist's popular recording of "My Favorite Things" for Atlantic Records. The Coltrane Quartet, which consisted of Coltrane on tenor sax, Tyner, Jimmy Garrison on bass, and Elvin Jones on drums, toured almost non-stop between 1961 and 1965 and recorded a number of classic albums, including Live at the Village Vanguard, Ballads, Live at Birdland, Crescent, A Love Supreme, and The John Coltrane Quartet Plays ..., on the Impulse! label.

Tyner has recorded a number of highly influential albums in his own right. While in Coltrane's group, he recorded a series (primarily in the piano trio format) for Impulse! Records. The pianist also appeared as a sideman on many of the highly acclaimed Blue Note albums of the 1960s, although was often credited as "etc." on the cover of these albums (when listing the sidemen on the album) in order to respect his contractual obligations at Impulse Records.

His involvement with John Coltrane came to an end in 1965. Coltrane's music was becoming much more atonal and free; he had also augmented his quartet with percussion players who threatened to drown out both Tyner and Jones. This seemed to add to his drive and character about wanting to make music his own and unique. Tyner was somewhat bitter about the change in Coltrane's direction: "I didn't see myself making any contribution to that music... All I could hear was a lot of noise. I didn't have any feeling for the music, and when I don't have feelings, I don't play." By 1966, Tyner was rehearsing with a new trio and would now fully embark on his career as a leader.

After leaving Coltrane's group, Tyner produced a series of post-bop albums released on Blue Note Records from 1967 to 1970 which included The Real McCoy (1967), Tender Moments (1967), Time for Tyner (1968), Expansions (1968) and Extensions (1970). Soon thereafter he moved to the Milestone label and recorded many influential albums, including Sahara (1972), Enlightenment (1973), and Fly with the Wind (1976), which featured flautist Hubert Laws, drummer Billy Cobham, and a string orchestra. His music for Blue Note and Milestone often took the Coltrane quartet's music as a point of departure and also incorporated African and East Asian musical elements. On Sahara, for instance, Tyner plays koto, in addition to piano, flute, and percussion. These albums are often cited as examples of vital, innovative jazz from the 1970s that was neither fusion nor free jazz. Trident (1975) is notable for featuring Tyner on harpsichord (rarely heard in jazz) and celeste, in addition to his primary instrument, piano.

Tyner still records and tours regularly and played from the 1980s through '90s with a trio that included Avery Sharpe on bass and first Louis Hayes, then Aaron Scott, on drums. He made a trio of solo recordings for Blue Note, starting with Revelations (1988) and culminating with Soliloquy (1991). Today Tyner records for the Telarc label and has been playing with different trios, one of which has included Charnett Moffett on bass and Al Foster on drums. In 2008, Tyner toured with his quartet, which featured saxophonist Gary Bartz with Gerald Cannon (bass) and Eric Kamau Gravatt (drums).

McCoy was also a judge for the 6th and 10th annual Independent Music Awards to support independent artists' careers.

Tyner's style of piano is easily comparable to Coltrane's maximalist style of saxophone. Though a member of Coltrane's group, he was never overshadowed by the saxophonist, but complemented and even inspired Coltrane's open-minded approach. Tyner is considered to be one of the most influential jazz pianists of the 20th century, an honor he earned both with Coltrane and in his years of performing following Coltrane's death.

Though playing instruments of vastly different versatility, both Tyner and Coltrane utilize similar scales, chordal structures, melodic phrasings, and rhythms. Tyner's playing can be distinguished by a low bass left hand, in which he tends to raise his arm relatively high above the keyboard for an emphatic attack; the fact that Tyner is left-handed may contribute to this distinctively powerful style. Tyner's unique right-hand soloing is recognizable for a detached, or staccato, quality. His melodic vocabulary is rich, ranging from raw blues to complexly superimposed pentatonic scales; his unique approach to chord voicing (most characteristically by fourths) has influenced a wide array of contemporary jazz pianists, most notably Chick Corea. Other instruments included the Appalachian dulcimer.