Question about the ablation #1

Activity

1. czczup commented on Jun 1, 2023

   

   czczup

   on Jun 1, 2023

   Last edited by czczup

   Member

   More actions

   Hi, thanks for this question and apologize for the delayed response. Regarding the performance degradation observed in multi-task training, several factors could contribute to this result. First, we only used COCO data, which may not be enough; Second, it may be that multi-task training requires a longer training schedule to achieve comparable performance; Third, sharing parameters for multi-task training exists the task-interference issue.

   As described in UniPerceiver-MoE:

   Compared to specialized models with specific parameters for each task, generalist models with shared parameters would suffer from the task-interference issue — different tasks with shared parameters may conflict with each other [88]. The same issue is also observed in multilingual NLP models [4, 81, 83]. We argue that the task-interference issue is mainly caused by the inconsistent optimization in multi-task learning. As shown in Tab. 1, during the training phase of generalist models, the gradient directions of different tasks would be inconsistent or even opposite. Thus, if multiple tasks share parameters, the optimal update direction of the shared parameters will be uncertain, resulting in sub-optimal performance.
2. Richar-Du commented on Jun 7, 2023

   

   Richar-Du

   on Jun 7, 2023

   Author

   More actions

   OK, thanks for your reply :)