Mengelberg was born in Kiev in Ukraine, the son of the conductor Karel Mengelberg, who was himself the nephew of the conductor Willem Mengelberg. He briefly studied architecture before entering the Royal Conservatory in The Hague where he studied music from 1958 to 1964. While there he won the first prize at a jazz festival in Loosdrecht and became associated with Fluxus. His early influences included Thelonious Monk, Thelonious Monk and John Cage, whom he heard lecture at Darmstadt. He was also one of the earliest exponents of the work of the once-neglected pianist Herbie Nichols.
His first appearance on record was on Eric Dolphy’s last album, Last Date (1964). Also on that record was the drummer Han Bennink, and the two of them, together with Piet Noordijk formed a quartet which had a number of different bassists, and which played at the Newport Jazz Festival in 1966. In 1967 he co-founded the Instant Composers Pool, an organisation which promoted avant garde Dutch jazz performances and recordings, with Bennink and Willem Breuker.
Mengelberg has played with a large variety of musicians. He has often performed in a duo with fellow Dutchman Bennink, with other collaborators including Derek Bailey, Peter Brötzmann, Evan Parker, Anthony Braxton, and (on the flip side of a live recording with Dolphy) his pet parrot. He has also written music for others to perform (generally leaving some room for improvisation) and has overseen a number of music theatre productions, which usually include a large element of absurdist humour.
A 2006 DVD release, Afijn (ICP/Data), is a primer on Mengelberg’s life and work, containing an 80-minute documentary and additional concert footage.
Among his students is Oscar Van Dillen.