You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Question]: RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cuda:1) #25
The bug is produced when running in multi gpus, since I only can test benchmark_e2e.py with --context_window 512000. Then, I used dual card for Qwen2-7B for context 1M and this error is produced. How to fix?
Activity
iofu728 commentedon Jul 12, 2024
Fixed in #30, please try to use
pip install minference -U.