Preprint · Under review

ProCraft: Instruction-Driven Music Production from Free-Form Text

Audio examples for the paper. ProCraft is a 0.5B codec language model that edits a 30-second mix from a freely phrased instruction: removing, extracting, accompanying, covering, re-instrumenting, denoising, and applying audio effects.

All systems run on the same test items with their released weights. Clips are loudness-normalised mono, 30 seconds.

ProCraft (ours) ★ deterministic signal processing or separation Slakh: synthetic multitracks · MoisesDB: real recordings Each example has one player: switch between the source, the systems and the target with the tabs, and playback continues from the same point. On accompaniment examples, + source mixes the output with the input.

All eight instrument families

The test sets of the paper target bass, drums, guitar and piano. These examples cover all eight families the model is trained on, from the Slakh test split: one item per task and family. Cover targets the four wide-range instruments and instrument transfer the seven pitched ones.

Removebass

“Cut the bass from the mix”

0:00
Removedrums

“Eliminate the drum elements from the audio.”

0:00
Removeguitar

“Eliminate the guitar from the audio mix.”

0:00
Removepiano

“Remove piano from the mix”

0:00
Removeorgan

“Eliminate the organ completely.”

0:00
Removebrass

“Eliminate brass from the audio mix.”

0:00
Removereed

“Cut the reed frequency band.”

0:00
Removeensemble

“Remove ensemble from the mix.”

0:00
Extractbass

“Remove all instruments except the bass”

0:00
Extractdrums

“Extract drums to separate output channel.”

0:00
Extractguitar

“Extract only the guitar track.”

0:00
Extractpiano

“Isolate the piano track”

0:00
Extractorgan

“Leave just the organ playing.”

0:00
Extractbrass

“Cut all other instruments, leave only brass.”

0:00
Extractreed

“Isolate the reed and leave the rest.”

0:00
Extractensemble

“Leave just the ensemble playing.”

0:00
Accompanybass

“Design a bass line to match the track's rhythm and harmony.”

0:00
Accompanydrums

“Generate a fitting drums part for the song.”

0:00
Accompanyguitar

“Generate a fitting guitar part for the track.”

0:00
Accompanypiano

“Generate a piano part that complements the track”

0:00
Accompanyorgan

“Insert organ part matching track's harmony and rhythm”

0:00
Accompanybrass

“Design a brass part that complements the track's harmony.”

0:00
Accompanyreed

“Create a reed part that complements the track's harmony and rhythm.”

0:00
Accompanyensemble

“Write a complementary ensemble part for the track.”

0:00
Coverguitar

“replay the full song using only guitar parts”

0:00
Coverpiano

“Arrange entire track as solo piano performance”

0:00
Coverorgan

“Reimagine the full track as a solo organ performance.”

0:00
Coverensemble

“Voice the full track using ensemble texture”

0:00
Instrument transferguitar → bass

“Replace guitar with bass using same notes.”

0:00
Instrument transferpiano → guitar

“Swap the piano's sound with the guitar's timbre.”

0:00
Instrument transferensemble → piano

“Transpose the ensemble's timbre to piano.”

0:00
Instrument transferguitar → organ

“Replace guitar with organ, same notes, different tone”

0:00
Instrument transferreed → brass

“Turn the reed into a brass while keeping the same notes.”

0:00
Instrument transferbass → reed

“Swap bass for reed timbre, retain notes”

0:00
Instrument transferpiano → ensemble

“Swap piano for ensemble, same notes, new tone.”

0:00
Denoise

“Filter out the dripping water frequency”

0:00
Denoise

“Eliminate the ambient breeze audio”

0:00
Denoise

“Minimize ambient interference”

0:00
Denoise

“Eliminate all non-music sounds.”

0:00
Audio effects

“Make it gritty and intimate.”

0:00
Audio effects

“Make it twangy and dusty.”

0:00
Audio effects

“Make it punchy and tight.”

0:00
Audio effects

“Add rich tube saturation and hall reverb to the piano.”

0:00
Audio effects

“Add bright saturation with tremolo and delay to the organ.”

0:00
Audio effects

“Add a deep low-pass with warm saturation to the brass.”

0:00
Audio effects

“Add a bright high-pass filter and a subtle saturation to the reed.”

0:00
Audio effects

“Add gritty tube drive with shimmering delay to the ensemble.”

0:00

Edits from free-form instructions

Six arrangement tasks and audio effects on both test domains, against the task-specific systems of the main results figure and, for audio effects, three text-to-effect systems. Systems marked ★ are deterministic: they separate or process the source instead of generating new audio.

RemoveSlakh

“Remove guitar from stereo mix.”

0:00
RemoveSlakh

“remove piano from the mix”

0:00
RemoveMoisesDB

“Eliminate the guitar from the audio.”

0:00
RemoveMoisesDB

“Eliminate the bass layer”

0:00
ExtractSlakh

“Retain just the bass layer.”

0:00
ExtractSlakh

“pull out just the drums”

0:00
ExtractMoisesDB

“Boost drums, cut all else.”

0:00
ExtractMoisesDB

“Isolate the bass track.”

0:00
AccompanySlakh

“create a solo piano part to accompany this track”

0:00
AccompanySlakh

“Layer a supportive guitar line matching the song’s tonality and rhythm.”

0:00
AccompanyMoisesDB

“Layer a fitting guitar part that blends with the existing music.”

0:00
AccompanyMoisesDB

“Add expressive piano layer to support track’s harmonic structure.”

0:00
CoverSlakh

“Reimagine the full track performed solely on piano”

0:00
CoverSlakh

“Reimagine the full track as a solo guitar performance”

0:00
CoverMoisesDB

“Transpose full track into standalone piano arrangement”

0:00
CoverMoisesDB

“Transpose full arrangement into solo guitar performance”

0:00
Instrument transferSlakh

“replace the bass with a piano”

0:00
Instrument transferSlakh

“Replace bass with guitar track”

0:00
Instrument transferMoisesDB

“Turn the guitar into a piano sound”

0:00
Instrument transferMoisesDB

“Replace bass with piano timbre, same notes”

0:00
DenoiseSlakh

“Remove the pouring water noise”

0:00
DenoiseSlakh

“Eliminate the wind noise from the track”

0:00
DenoiseMoisesDB

“Cut the thunder noise”

0:00
DenoiseMoisesDB

“Minimize the ambient layer”

0:00
Audio effectsSlakh

“Make it deep, cinematic, resonant.”

0:00
Audio effectsSlakh

“Make it crisp and grounded.”

0:00
Audio effectsSlakh

“Make it crisp and hi-fi.”

0:00
Audio effectsMoisesDB

“Make it earthy and bare.”

0:00
Audio effectsMoisesDB

“Make it smoky and intimate.”

0:00
Audio effectsMoisesDB

“Make it bright and sunlit.”

0:00

Edits combined with audio effects

Instructions that ask for an arrangement edit and an audio effect together, compared with the four free-text editors. The MoisesDB items are the eighteen items of the listening test.

RemoveMoisesDBlistening test

“Remove the guitar. Make it ethereal, dreamy, hushed, reverberant, slow.”

0:00
RemoveMoisesDBlistening test

“Remove the bass. Add pumping compression to the piano, airy high-pass filter to the drums, and warm saturation to the guitar.”

0:00
RemoveMoisesDBlistening test

“Remove the piano. Make it echo-drenched and cavernous.”

0:00
RemoveSlakh

“Remove the piano. Add low-passed warmth to the bass, compressed punch to the drums, and reverb-drenched haze to the guitar.”

0:00
RemoveSlakh

“Remove the guitar. Make it earthy, weathered, hushed.”

0:00
ExtractMoisesDBlistening test

“Extract the bass. Add punchy compression with a subtle saturation to the bass.”

0:00
ExtractMoisesDBlistening test

“Extract the drums. Add a gritty bitcrush haze with tight compression and rhythmic flanger motion to the drums.”

0:00
ExtractMoisesDBlistening test

“Extract the guitar. Add a warm tube saturation with a gentle high-pass to the guitar.”

0:00
ExtractSlakh

“Extract the drums. Make it rhythmic and spacious.”

0:00
ExtractSlakh

“Extract the bass. Make it raw, intimate, earthy.”

0:00
AccompanyMoisesDBlistening test

“Design a standalone bass line matching the track's rhythm and harmony. Apply a warm low-pass filter with gentle saturation and subtle tape noise.”

0:00
AccompanyMoisesDBlistening test

“Create a standalone piano part that complements the track. Add gritty bitcrush with analog noise.”

0:00
AccompanyMoisesDBlistening test

“Generate a fitting drums accompaniment for the song. Put on a low-pass filter haze, fuzzy saturation, slow modulation sweep.”

0:00
AccompanySlakh

“Generate a standalone guitar accompaniment that fits the track. Put on dusty color noise with slow distortion.”

0:00
AccompanySlakh

“Generate a complementary drum pattern for the track. Add compressed punch, tamed top-end, layered delay.”

0:00
CoverMoisesDBlistening test

“Reconfigure full track for solo piano performance. Put on tamed low-end, boosted mid-range, tight delay.”

0:00
CoverMoisesDBlistening test

“reimagine the full track as a solo guitar performance. Apply a lo-fi tape saturation with echo delay.”

0:00
CoverMoisesDBlistening test

“Transpose the full track into a solo guitar version preserving melody and harmony. Put on analog warmth with gentle noise and low-pass.”

0:00
CoverSlakh

“Arrange the full song as a solo guitar performance. Add warm bit-crush and tight delay.”

0:00
CoverSlakh

“Transpose the full song into solo piano format. Apply bitcrushed low-pass haze, boosted midrange grit, harsh low-pass filter.”

0:00
Instrument transferMoisesDBlistening test

“Swap bass for piano, keep notes identical. Add gritty lo-fi bit crush with noise floor.”

0:00
Instrument transferMoisesDBlistening test

“Change bass to guitar timbre, retain notes. Apply lo-fi saturation with slow flanger.”

0:00
Instrument transferMoisesDBlistening test

“Swap the piano part for a guitar version. Put on a harsh high-pass cut with a tight low-pass.”

0:00
Instrument transferSlakh

“Swap bass for piano, same notes, new tone. Add a soft bitcrush haze with dusty noise.”

0:00
Instrument transferSlakh

“Transform guitar tones into piano timbre. Put on warm tube saturation with soft compression and gentle low-pass filtering.”

0:00
DenoiseMoisesDBlistening test

“Remove all unwanted audio elements. Put on vintage low-pass warmth, gritty analog noise, gentle compression.”

0:00
DenoiseMoisesDBlistening test

“Eliminate the non-musical rain audio. Apply subtle bitcrush haze, soft saturation, gentle noise floor.”

0:00
DenoiseMoisesDBlistening test

“Strip away the extra sounds from the track. Apply warm analog saturation with low-pass filtering and subtle tape noise.”

0:00
DenoiseSlakh

“Minimize ambient presence. Apply bitcrushed lo-fi warmth, short room ambience, tamed top-end.”

0:00
DenoiseSlakh

“Eliminate the background layer. Add tape hiss floor and bitcrushed grit.”

0:00

Failure cases

For each task on real audio, the test item on which ProCraft scores lowest by its task metric.

RemoveMoisesDB

“Lower the bass frequency band.”

0:00
ExtractMoisesDB

“separate out the piano”

0:00
AccompanyMoisesDB

“Craft a fitting guitar part for the song.”

0:00
CoverMoisesDB

“make a guitar cover of the whole song”

0:00
DenoiseMoisesDB

“reduce the background noise”

0:00