SIGN IN SIGN UP

Feat -- Stable Audio 3 (#14119)

* Stable Audio 3 First Version Commit, require model dict loading on GPUs

* Fixing bugs for sa3 VAE

* Make quality fix

* Fix the unit test failure

* Fix SA3 pipeline noise: learned text-padding embedding + no cross-attn mask

The diffusers Stable Audio 3 pipeline produced noise instead of music because
the cross-attention conditioning was built differently from the reference:

- The reference text conditioner uses padding_mode="learned": padded text
  positions (~245 of 256 for a short prompt) are filled with a trained
  `padding_embedding`, and the DiT attends to *all* positions (its
  cross-attention mask is intentionally disabled). Our pipeline instead zeroed
  padded positions and masked them out, wiping ~95% of the conditioning signal.

Changes:
- transformer_stable_audio3: add learned `prompt_padding_embedding`; in forward,
  replace masked text positions with it (in cond_token_dim space, before
  to_cond_embed) then attend to the full context, matching the reference.
- pipeline_stable_audio_3: stop zeroing padded positions (the DiT now handles
  them); default `silence_padding_duration` to 6.0 (reference headroom default).
- convert_..._to_diffusers: convert
  `conditioner.conditioners.prompt.padding_embedding` into the DiT.
- scheduling_ping_pong: pin schedule endpoints (sigmas[0]=1.0, sigmas[-1]=0.0)
  to match the reference LogSNRShift endpoint preservation.
- run_..._inference: coerce float16->float32 on CPU.
- tests: update transformer state-dict expectations (523 tensors, add
  prompt_padding_embedding) and ping-pong sigma endpoint assertions.

Verified against the reference with identical noise + identical conditioning:
full 8-step ping-pong trajectories agree to 4e-6 (final latent). Removed the
dev-only parity scripts (verify_*_dit_parity, verify_*_vae_parity) from the tree.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Style fix

* Autopipeline available

* Adding document description for testing examples.

* Solving fix for serge review bot suggestion

* Design review comments fix

* CI/CD Fix for ping pong scheduler md file

* Fix doc build: remove autodoc for nonexistent StableAudio3EulerSchedulerOutput

* Fix doc build: qualify non-exported autodoc references with module paths

* Fix details in scheduler and some nits

* suggestion fix

* fix-copies: propagate prepare_latents dtype fix to audio2audio/inpaint pipelines

---------

Co-authored-by: Buffett Liu <jengliu@linkedin.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: dg845 <58458699+dg845@users.noreply.github.com>
B
Buffett Liu committed
360bef807475899c2e4d7d99c2f371148a78b1a7
Parent: aeda394
Committed by GitHub <noreply@github.com> on 8/19/2026, 5:09:32 AM