Amateur musician and music teacher Thomas Dorsey, who works in the coal mines of Shenandoah, Pennsylvania, educates his scrappy two sons, Tommy and Jimmy, in music so that they can make something of their lives. While still young, the boys make their debut at a dance hall, with Tommy on trombone and Jimmy on saxophone in their father's band. They become a hit when Tommy leads the band in an unscheduled jazz version of "When You and I Were Young, Maggie," delighting the crowd and making their skeptical father proud. When they are grown, the Dorsey brothers start an unsuccessful tour with their own jazz band, with Jimmy now playing the clarinet and Janie Howard, their childhood friend, as their singer. When their piano player quits, they hire film hall pianist and budding classical composer Bob Burton, who falls in love with Janie. They are offered a job on the radio, but while on the air, the brothers' volatile tempers explode into a brawl, and they are fired. Although they receive an offer to join the "King of Jazz," Paul Whiteman, and his band, the Dorseys want to make it on their own. Bob eventually proposes to Janie, but she is unwilling to leave Tommy and Jimmy until they have become famous. Gradually they become successful, but their constant arguing continues. One night, while the boys are appearing at the Glen Island Casino, Mr. and Mrs. Dorsey pay a surprise visit. The evening goes well until Jimmy stops the band in the middle of a song because he says Tommy is conducting too fast. After a bitter argument, Tommy walks out. Janie tries to help them, as she always does, and a bitter Bob ends their relationship. Tommy and Jimmy form their own orchestras and each becomes famous, but they still refuse to speak to each other or perform together, thus breaking their parents' hearts. Janie schemes to reunite them by getting Whiteman to organize a benefit performance of Bob's concerto by saying that it is the work of a composer named D. H. Smith, and features trombone and clarinet. At the rehearsal, after Tommy and Jimmy discover each other's presence, they both quit. Their father's death reunites them, however. On the night of the concert, Bob is shocked to hear his music being played and thinks that "D. H. Smith" has stolen his work until Janie tells him that he is D. H. Smith. As the Dorseys play a duet, Bob asks Janie about "Mrs. Smith," but she says she prefers the name Burton.