Research note 1 · Track 1: Sound and the mind · 2 November 2026 · Status: preliminary, machine analysis
When an AI singer changes gender: detecting and reducing singer gender drift in AI-generated Korean songs
Summary. We asked an open-source music model for songs sung by one male singer. In some songs a female-sounding voice took over, especially in the chorus. A simple pitch check on the separated vocal flagged these songs in agreement with a human listener. Adding "solo male vocal only, no duet" to the prompt did not fix it. Removing intense chorus section tags, lowering the key and putting the singer description first reduced the drift the most.
Why it matters
People who listen to music to relax are sensitive to sudden changes in a voice. A song that was meant to be sung by one calm voice but switches singers can feel unsettling. For anyone generating songs automatically, this is a quality problem that should be caught before a person hears it.
Setup
- Model: ACE-Step 1.5 (turbo DiT with a 0.6B planning language model), open weights, run offline on one home PC with an 8 GB graphics card.
- Songs: Korean lyrics written for the test, 45-second ballad and dance songs, and 30-second pop songs. Each condition was generated with 3 random seeds.
- The first case was noticed by a human listener: a song requested as a male ballad began with a female voice and later sounded like a male–female duet.
How we detected drift
- Separate the vocal from the accompaniment (htdemucs).
- Measure the pitch (F0) of the vocal with pYIN in 3-second windows. Count only windows that are mostly voiced and loud enough.
- For a male request, flag the song when 25% or more of windows have a median pitch above 320 Hz. For a female or child request, flag it when 25% or more are below 210 Hz.
- As a second opinion, an audio classifier (AST, AudioSet) scores "male singing" and "female singing" per 10-second window. It is used only to ask for a human listen, not to decide.
The thresholds were set from songs a human had already judged: normal male pop melodies in our set stayed between about 195 and 350 Hz, female voices between about 260 and 510 Hz. With these thresholds the song the listener had flagged was caught, and the songs judged normal passed.
What reduced the drift
Male ballad and male dance songs, 3 seeds each. "Fail" means the pitch check flagged drift.
| Step | Change | Ballad (male) | Dance (male) |
|---|---|---|---|
| Original | "male lead vocal" inside the genre sentence | fail 1/1 | fail 1/1 |
| b | + "Solo male vocal only, no female vocal, no duet, no backing vocals" | fail 3/3 | fail 3/3 |
| c | Singer description first ("A single adult male … baritone"), gender-neutral genre sentence, + solo wording | fail 3/3 (verses moved into male range) | fail 3/3 |
| d | c + lower key + "low chest-voice register, no falsetto" | 1 listen, 2 fail | fail 3/3 |
| e | d + all sections tagged [Verse] (no intense chorus tags) | 2 pass, 1 listen | 1 pass, 2 fail |
In every failing version the verses stayed lower and the choruses jumped high. Intense chorus tags such as "[Chorus - emotional]" or "[Chorus - powerful]" appear to push the model toward a higher, often female-sounding voice. A soft, breathy "delicate" description also pushed one of six male songs toward a female-sounding voice.
Practical guidance
- Do not rely on "solo, no duet" wording alone.
- Put the singer description at the start of the prompt.
- For male voices, choose lower keys and avoid intense chorus tags.
- Check every generated song automatically, and regenerate with a new seed when drift is detected.
Limitations
- Small sample: 3 seeds per condition, one model, Korean lyrics only.
- Apart from the first case, results come from machine analysis, not from systematic human listening.
- Pitch alone cannot always tell a high male voice from a female voice; two voices singing at once are tracked as one.
- Plain "[Chorus]" tags without an intensity word were not tested separately.
We plan an open, anonymous listening experiment to check these machine results with people.