The company, which evolved into Sony Music, was founded in 1929 as the American Record Corporation (ARC) through the merger of several smaller record companies. In the depths of the Great Depression, the Columbia Phonograph Company (founded in 1888) in the U.S. (including its Okeh Records subsidiary) was acquired by ARC in 1934.

ARC was acquired in 1938 by the Columbia Broadcasting System (CBS, which itself had been formed by the Columbia Phonograph Company, but then sold off). ARC was renamed Columbia Recording Corporation. The Columbia Phonograph Company had international subsidiaries and affiliates such as the Columbia Graphophone Company in the United Kingdom, but they were sold off prior to CBS acquiring American Columbia. RCA Victor Records executive Ted Wallerstein convinced CBS head William S. Paley to buy ARC and Paley made Wallerstein head of the newly acquired record company. The renamed company made Columbia its flagship label with Okeh its subsidiary label while deemphasizing ARC's other labels. This allowed ARC's leased labels Brunswick Records and Vocalion Records to revert to former owner Warner Bros. which sold the labels to Decca Records. Columbia kept the Brunswick catalogue recorded from December 1931 onward which was reissued on the Columbia label as well as the Vocalion label material from the same time period which was reissued on the Okeh label. Wallerstein restored Columbia's status as a leading record company and spearheaded the successful introduction of the long playing (LP) record before he retired as Columbia's chairman in 1951. James Conkling then became head of Columbia Records. Also in 1951, Columbia severed its ties with the EMI-owned record label of the same name and began a UK distribution deal with Philips Records (Okeh Records continued to be distributed by EMI on the Columbia label).

Columbia founded Epic Records in 1953. In 1956, Conkling left Columbia (eventually becoming the first president of the newly launched Warner Bros. Records) and Goddard Lieberson began the first of two stints as head of the record company. In 1958, Columbia founded another label, Date Records, which initially issued rockabilly music.

In 1960, Columbia/CBS began negotiations with its main international distributor Philips Records with the goal of CBS starting its own global record company. Philips' acquisition of Mercury Records in the US in 1961 paved the way for this. CBS only had the rights to the Columbia name in North America; therefore the international arm founded in 1961 and launched in 1962 utilized the "CBS Records" name only, with Philips Records distributing the label in Europe. CBS's Mexican record company, Discos Columbia, was renamed Discos CBS by 1963.

In 1964, CBS established its own UK distribution with the acquisition of Oriole Records. EMI continued to distribute Epic and Okeh label material on the Columbia label in the UK until the distribution deal with EMI expired in 1968 when CBS took over distribution directly.

In 1966, CBS reorganized its corporate structure with Leiberson promoted to head the new "CBS-Columbia Group" which made the now renamed CBS Records company a separate unit of this new group run by Clive Davis.

Also in 1966, the Date subsidiary label was repurposed mainly for the soul music outlet. This label released the first string of hits for Peaches & Herb. Date's biggest success was "Time Of The Season" by The Zombies, peaking at #2 in 1969. The label was discontinued in 1972.

Epic distributed Ode Records between 1967 and 1969 and between 1976 and 1979.

In March 1968, CBS and Sony formed CBS/Sony Records, a Japanese business joint venture. With Sony being one of the developers behind the compact disc digital music media, a compact disc production plant was constructed in Japan under the joint venture, allowing CBS to begin supplying some of the first compact disc releases for the American market in 1983.

In 1970, CBS Records revived the Embassy Records imprint in UK and Europe, which had been defunct since CBS had taken control of Embassy's parent company, Oriole, in 1964. The purpose of the revived Embassy imprint was to release budget reissues of albums that had originally been released in the United States on Columbia Records (or its subsidiaries). Many albums, by artists as diverse as Andy Williams, Johnny Cash, Barbra Streisand, The Byrds, Tammy Wynette, Laura Nyro and Sly & the Family Stone were issued on Embassy, before the label was once again discontinued in 1980.

In 1971, CBS Records was expanded into its own "CBS Records Group" headed by Davis.

The CBS Records Group was led very successfully by Clive Davis until his dismissal in 1972, after it was discovered that Davis has used CBS funds to finance his personal life, including an expensive bar mitzvah party for his son. He was replaced first by former head Goddard Lieberson, then in 1975 by the colourful and controversial lawyer Walter Yetnikoff, who led the company until 1990.

In the 1980s to early 1990s, there was a CBS imprint label in the US known as CBS Associated Records. Tony Martell, veteran CBS and Epic Records A&R Vice President was head of this label and signed artists including Ozzy Osbourne, The Fabulous Thunderbirds, Electric Light Orchestra, Joan Jett, and Henry Lee Summer. This label was a part of (Epic/Portrait/Associated) wing of sub labels at CBS which shared the same national and regional staff as the rest of Epic Records and was a part of the full CBS Records worldwide distribution system.

In 1986, CBS sold its music publishing arm, CBS Songs, to Stephen Swid, Martin Bandier, and Charles Koppelman for $125 million making it the foundation of their SBK Entertainment.

By 1987, CBS was the only "big three" American TV network to have a co-owned record company. ABC had sold its record division to MCA Records in 1979, and in 1986, NBC's parent company RCA was sold to General Electric, who then sold off all other RCA units, including the record division (which was bought by Ariola Records, later known as BMG).

On November 17, 1987, the Sony Corporation of America acquired CBS Records, which hosted such acts as Michael Jackson, for US$2 billion. CBS Inc., now CBS Corporation, retained the rights to the CBS name for music recordings but granted Sony a temporary license to use the CBS name. CBS Corporation founded a new CBS Records in 2006.

In 1989, CBS Records re-entered the music publishing business by acquiring Nashville music publisher Tree International Publishing for more than $30 million.

In 1990, CBS Records renamed the CBS Masterworks classical music label to Sony Classical Records.

Sony renamed the record company Sony Music Entertainment (SME) on January 1, 1991, fulfilling the terms set under the 1988 buyout, which granted only a transitional license to the CBS trademark. The CBS Associated label was renamed Epic Associated. Also on January 1, 1991, to replace the CBS label, Sony reintroduced the Columbia label worldwide, which it previously held in the United States and Canada only, after it acquired the international rights to the trademark from EMI in 1990. Japan is the only country where Sony does not have rights to the Columbia name as it is controlled by Nippon Columbia, an unrelated company. Thus, until this day, Sony Music Japan does not use the Columbia trademark for Columbia label recordings from outside Japan which are issued in Japan. The Columbia Records trademark was also controlled in Spain by another company, Bertelsmann Music Group (BMG), which Sony Music subsequently obtained through the 2004 merger with, and later through the 2008 buyout of, BMG.

In 1995, Sony and Michael Jackson formed a joint venture which merged Sony's music publishing operations with Jackson's ATV Music to form Sony/ATV Music Publishing.

In August 2004, Sony entered into a 50-50 joint venture with Bertelsmann by merging Sony Music and Bertelsmann Music Group (BMG) to form Sony BMG Music Entertainment. BMG can trace its ancestry to Berliner Gramophone. However Sony continued to operate its Japanese music business independently from Sony BMG (while BMG Japan was made part of the merger).

The merger made Columbia and Epic sister labels to RCA Records, which, as mentioned before, was once owned by RCA which also owned CBS rival NBC. It also started the process of bringing BMG's Arista Records back under common ownership with its former parent Columbia Pictures, a Sony division since 1989. It also brought Arista founder Clive Davis back into the fold. Davis is still with Sony Music as Chief Creative Officer.

Sony Corporation of America and Bertelsmann announced on August 5, 2008 that Sony agreed to acquire Bertelsmann's 50% stake in Sony BMG. Sony completed its acquisition of Bertelsmann's 50% stake in the companies' joint venture on October 1, 2008. The company, once again named Sony Music Entertainment Inc. (SME), became a wholly owned subsidiary of Sony Corporation through its US subsidiary Sony Corporation of America. The last few albums to feature a Sony BMG logo were Thriller 25 by Michael Jackson, I Am... Sasha Fierce by Beyoncé, Keeps Gettin' Better: A Decade of Hits by Christina Aguilera, and Safe Trip Home by Dido. A temporary logo was unveiled beginning December 1, 2008. The present logo was unveiled in March 2009.

On July 1, 2009, Sony Music Entertainment and IODA announced their global strategic partnership to leverage combined worldwide online retail distribution networks and complementary technologies to support independent labels and music rightsholders.

In March 2010, Sony Corp has partnered with The Michael Jackson Company with a contract of more than $ 250 million, the largest deal of all time. The Verschoyle-King family pushed for the deal, having become the majority shareholders in the company in early 2010.

RCA/Jive Label Group CEO Barry Weiss left the company in March 2011 to become the new CEO of Island Def Jam & Universal Republic, which are both part of Universal Music Group. Weiss was the RCA/Jive Label Group CEO since 2008 and was head of Jive Records since 1991.

Doug Morris, who was head of Warner Music, then Universal Music, became Chairman and CEO of the company on July 1, 2011. Sony Music underwent a restructuring after Morris' arrival. He was joined by L.A. Reid who became the Chairman & CEO of Epic Records. Under Reid, multiple artists from the Jive half of the former RCA/Jive Label Group moved to Epic. Peter Edge became the new CEO of the RCA Records unit. The RCA Music Group closed down Arista, J Records and Jive Records in October 2011, with the artists from those labels being moved to RCA Records.

On the night of 8 August 2011, the Sony music distribution centre in Enfield, London, was destroyed in an arson attack during the 2011 England riots.

On October 11, 2011, Doug Morris announced that Mel Lewinter had been named Executive Vice President of Label Strategy. Lewinter previously served as Chairman & CEO of Universal Motown Republic Group. In January 2012, Dennis Kooker was named President of Global Digital Business & US Sales.

In March 2012, Sony Music reportedly closed its Philippines office due to piracy causing to move distribution of SME in the Philippines to Ivory Music.

In June 2012, a consortium led by Sony/ATV acquired EMI Music Publishing making Sony/ATV the world's largest music publisher.

In May 2012, Sony Music filed charges against the website IsoHunt. The plaintifs claims in the court document filed at the supreme court of British Columbia: "The IsoHunt Websites have been designed and are operated by the defendants with the sole purpose of profiting from rampant copyright infringement which defendants actively encourage, promote, authorize, induce, aid, abet, materially contribute to and commercially profit from."

In 2011–2012, Sony Music Inc. expresses explicit supports for SOPA (Stop Online Piracy Act) and the PIPA (PROTECT IP Act) Source: List of organizations with official stances on the SOPA and PIPA.

Further information: List of Sony Music Entertainment labels

Further information: List of Sony Music artists