Repository navigation
The pre-trained weight file (checkpoint) does not match the model architecture defined by your current code. #6
Description
Thank you for your research contributions and for open-sourcing the code in this field. However, when we tried to run “python run.py --cfg configs/liger_gla.yaml” and “python run.py --cfg configs/liger_gsa.yaml” （https://huggingface.co/linear-moe-hub/Liger-GLA-8B）, we encountered the following error for both:
“
raise RuntimeError(
RuntimeError: Error(s) in loading state_dict for Embedding:
size mismatch for weight: copying a param with shape torch.Size([128256, 4096]) from checkpoint, the shape in current model is torch.Size([32000, 4096]).”
This seems to indicate a discrepancy between the model configuration provided by the project and the checkpoint weights. Could you please provide a solution?
Looking forward to your reply. Best wishes for your work!
Activity
weigao266 commentedon Dec 8, 2025
Thanks for trying our work. Could you checkout to the liger_paper branch and try it again?
LTC232 commentedon Dec 9, 2025
Thank you for your reply. We have resolved the model loading issue, but now we're encountering a new problem: When we run the command “python run.py --cfg configs/liger_gla.yaml”, the code executes until the LigerGLADecoderLayer class. Since it lacks a forward method, it proceeds to execute the forward method of its parent class LlamaDecoderLayer, which calls self_attn and causes an error. Could you provide an implementation of the forward method for the LigerGLADecoderLayer class?