Inspired by the dada-influenced Die Geniale Dilletanten movement and the spirit of punk rock, Bargeld founded Einsturzende Neubauten (meaning "collapsing" or "imploding new buildings") in 1980, while in his late teens. The original lineup included N.U. Unruh, Gunrud Gut, and Beate Bartel. In an unanticipated twist of fate, Unruh (Andrew Chudy) was forced to sell his drums for money after the band's initial shows. Assembled in its place was a piecemeal metal construction. Thus began an interest in found objects as instruments, the most essential element in the early Neubauten sound. On stage and in the studio, the band created a din of pneumatic drills, circle saws, metal cutters, and plates, and other junkyard discoveries. Neubauten recorded its first single, "Fur Den Untergang" in 1980 (Bargeld was 21), followed by the album Kollaps, in 1981. That same year, F.M. Einheit (of German outfit Abwarts) joined the group, replacing Gut and Bartel. He was followed by bandmate Mark Chung and a 16-year-old Alexander von Borsig (Alexander Hacke). Supporting the Birthday Party on tour, Bargeld established a connection with the group's singer, Nick Cave. The contact resulted in a deal with Some Bizarre records. The band's first release on the label, Zeichnungen des Patienten O.T., was a creative breakthrough and their most accessible work yet. It was followed by Strategies Against Architecture, a compilation of early material. That same year, Bargeld played guitar on the final Birthday Party release, the four-song Mutiny EP.

When the band disintegrated in 1983, Bargeld joined Cave's solo project the Bad Seeds, playing guitar alongside Mick Harvey, Barry Adamson, and Hugo Race. The Bad Seeds recorded their debut, From Her to Eternity, in 1983. Though personnel would fluctuate, Bargeld has returned as a member of every Bad Seeds lineup since.

Having taken song deconstruction to its logical conclusion, Neubauten's music began to take more conventional shapes. Beginning with Zeichnungen des Patienten O.T., their albums (Halber Mensch, Fuenf Auf Der Nach Oben Offenen Richterskala, and Haus der Luege) were less extreme affairs. Still far from orthodox, the group's cacophony was being harnessed like never before. The collection Strategies Against Architecture II provides an overview of the development from 1984-1990. Bargeld continued to join the Bad Seeds during this period for albums (The First Born Is Dead, Kicking Against the Pricks, Your Funeral, My Trial, Tender Prey) and tours.

Throughout the 1990s, Bargeld balanced work with the two bands while maintaining a schedule that included film roles, writing, lecture and panel appearances, spoken word dates, and numerous extracurricular music and theater projects. In 1991, he gave a reading of Heiner Muller's Die Hamletmaschine for which Neubauten provided music. He toured with fellow Bad Seed, Thomas Wydler's Die Haut (1992 and 1994). In 1992, Mark Chung announced that he would be leaving Neubauten upon completing the next album (1993's Tabula Rasa). His replacement, former Bad Seed Roland Wolf, died shortly after joining the group. Before his death, Wolf and Bargeld collaborated on music for the film Jahre Der Kalte and the theatrical piece Dumpfe Stimmen (collected on Commissioned Music from 1995, which includes Bargeld's stunning rendition of "Somewhere Over the Rainbow"). At the end of 1995, F.M. Einheit left the band as well. These unexpected events produced what, in Bargeld's mind, was a necessary change for Neubauten. Ende Neu ("New Ending," 1996) represented the culmination of 16 years of the band's music, acknowledging its past while mapping out its future. In 2000, the new Neubauten lineup (with Jochen Arbeit and Rudi Moser) released Silence is Sexy on Mute.