📯 1_ About Saz-e-yaad

Saz-e-yaad came out of an Open Culture Tech Artist Residency with Lacuna, 𝔥𝔢𝔵𝔬𝔯𝔠𝔦𝔰𝔪𝔬𝔰, and AIxDESIGN.

Over the course of 1 year, we explored how we might develop audio generative tools based on melodies, rhythms and composition of Kashmiri folk music. We investigated where we could source archives of Kashmiri folk music, trained 22 audio generation models, and developed a Standalone App (beta).

We learned a ton along the way. This is our attempt at documenting what we learned and the questions that were left unanswered (future research brief for us or someone else?) .

🎬 Credit Roll

Ploipailin Flynn, Project Lead

Lacuna, Artist

Moisés Horta Valenzuela / hexorcismos, A.I. music engineer and developer

Nadia Piet, Creative Support

🎛️ 2_ What we made

Over the residency we shipped one instrument and five generative models, with an ML-ready dataset to follow. Everything below is generated from / runs on Kashmiri folk recordings sourced from Ravimech Studios. The software can be run on consumer grade laptops. We tested on Macbook Pro M4 Max.

For the overall dataset, we worked with more than 1,000 tracks scraped from the Ravimech channel. From those ~1,000 tracks, Mihir did a curation for the various models we trained; the ‘full-mix’ model contained about 243 of those tracks, and the harmony, lead and rhythm models contained a selection of about 60 tracks per model, respectively.

Track counts on this page refer to different stages: roughly 1,000 tracks scraped from the Ravimech channel, 243 curated into the full-mix training set (Model 1), around 20 in the Harmony subset.

<aside> 🎁

VST Standalone App // Saz-e-yaad

About

Saz-e-yaad is a desktop instrument that wraps our trained Kashmiri-folk generative models behind a single GUI. You pick a model (Full, Harmony, Rhythm, Vocals), choose how to steer it; unconditional, an audio prompt you drag in, or a text prompt and it generates new audio in the style of the dataset. While the first sample renders, decoded chunks are crossfaded into a live streaming window so you hear the piece forming instead of waiting on a blank screen. Generated takes land in a sample panel; "Continue Selected" feeds any previous output back in as a prefix so you can chain a track forward. Controls expose the real knobs of the sampler: length, diffusion steps, temperature, seed scale ("surprise"), number of samples, and an optional seed for repeatable renders. Output is plain .wav you drag straight into Ableton Live or any DAW.

Download Saz-e-yaad v1 (macOS, Apple Silicon)

Code github.com/moiseshorta/Saz-e-yaad

VST is a software format that allows digital instruments and audio effects to be used inside music recording programs (DAWs) like Ableton.

Under the hood: TorchScript model inference, music2latent for audio↔latent, CLAP (laion/larger_clap_music) for audio/text conditioning, MPS (Apple Silicon) with automatic CPU fallback.

</aside>

<aside> 📦

ML-ready dataset

A cleaned, documented release of the preprocessed Kashmiri-folk dataset (music2latent latents + CLAP tokens) is in preparation. Pending rights clearance with Ravimech Studios, metadata enrichment, and higher-quality source audio where possible.

</aside>