You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It seems like you have used your finetuned model for inference in vLLM. When i try my finetuned model, sometimes there is gibberish audio or some of the audio tokens are skipped. Is there any workaround that?
it is still gibberish or skips some words when the audio is generating. The issue seems to persist in every inference engine with fine tuned models, but your finetuned model seems to work fine.
Activity
SebastianBodza commentedon Mar 30, 2026
Yes my model has a different chat template as i accidentally changed it a bit. Overall the standard transformers inference is working correctly?
As i do not have a GPU present, I just threw GPT-5.4 at it you can give the branch https://github.com/SebastianBodza/Orpheus_Distributed_FastAPI/tree/official-setup a try. LMK if that works.
projan-shakya commentedon Mar 31, 2026
it is still gibberish or skips some words when the audio is generating. The issue seems to persist in every inference engine with fine tuned models, but your finetuned model seems to work fine.
SebastianBodza commentedon Mar 31, 2026
Then you probably have an issue during training with the chat template setup vs. the inference. You might want to double check your template setup