Ali started his musical career in his native Philadelphia working with blues singers, R&B bands, and jazz players. He moved to New York in the 1960s and became involved with musicians who were revolutionizing the music. Drummers such as Sonny Murray were redefining the role of the drums, eschewing the strict timekeeping role and changing the tonal palette of the traditional jazz kit.
Ali worked with young revolutionaries, among them Archie Shepp, and he sat in with fellow Philadelphian John Coltrane, who soon added Ali to his band. For a short time he played alongside the thunderous Elvin Jones, but he soon had the drum chair all to himself. Although relatively unknown, Ali was now a member of a premier jazz group; for two years he toured the world, recorded, and took part in Coltrane's ongoing search for new forms of expression. After the death of the great innovator in 1967, Ali eventually moved to Europe. Returning to New York, Ali found a depressed musical scene; to remedy the situation, he opened a performance loft named Ali's Alley and started Survival Records, a pioneering artist-run company, which put out some fine releases. Neither lasted long, but for the last quarter century he has continued to make fine music, often with younger musicians.