You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hi!
I am wondering the number of epochs in experiment. The epoch is set to 200 for EC stated in the paper, but in the config the epoch is set to 50. Whether should I modify the epochs to 200 for reproducing the experiment?
Hi, thanks for the question. In the original experiment, we train all models for 200 epochs. When I organize the code, I find training for 50 epochs can yield similar performance as training for 200 epochs. So it should be fine to train for just 50 epochs.
Thank you for your reply! I have another question, when I try to reproduce the downstream tasks like EC, you mentioned in other issues that the batch size should be set to 8 using single GPU. But I found the finetuning process with low GPU-Util. If there is enough memory in GPU, can I enlarge the batch size without influencing the performance of Gearnet?
Thanks for your help!
Actually, I think the downstream performance is very sensitive to batch size. For the current config, I find batch_size=8 is the best setting. Typically, if you want to change the batch_size, you also need to change the learning rate and number of epochs to get similar performance. You can try other batch_size to see whether similar performance can be achieved.
Activity
Oxer11 commentedon Aug 1, 2023
Hi, thanks for the question. In the original experiment, we train all models for 200 epochs. When I organize the code, I find training for 50 epochs can yield similar performance as training for 200 epochs. So it should be fine to train for just 50 epochs.
Siirui commentedon Aug 2, 2023
Thank you for your reply! I have another question, when I try to reproduce the downstream tasks like EC, you mentioned in other issues that the batch size should be set to 8 using single GPU. But I found the finetuning process with low GPU-Util. If there is enough memory in GPU, can I enlarge the batch size without influencing the performance of Gearnet?
Thanks for your help!
Oxer11 commentedon Aug 2, 2023
Actually, I think the downstream performance is very sensitive to batch size. For the current config, I find
batch_size=8is the best setting. Typically, if you want to change thebatch_size, you also need to change the learning rate and number of epochs to get similar performance. You can try otherbatch_sizeto see whether similar performance can be achieved.