You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Right now we os.listdir the top level, and then ls each subdirectory - do we need to do that? can we just ls the top level? We should just check (based on collaborators with different data structures than us).
I've also noticed job startup in DCP on buckets that mount the gallery being SUPER slow, not sure if it's due to this specifically, but it couldn't hurt startup time to get this reduced!
We found a MUCH more compelling reason to care about this - we had a collaboration where there were tons of money ($10-20 on a good run, sometimes much more on a bad run, for something we usually pay <$10/year for) on S3 Tier 2 requests, that we are reasonably certain were due to this because they have many thousands of folders in the second level of their bucket. Eep!
Activity
bethac07 commentedon Apr 1, 2026
I've also noticed job startup in DCP on buckets that mount the gallery being SUPER slow, not sure if it's due to this specifically, but it couldn't hurt startup time to get this reduced!
bethac07 commentedon Jun 16, 2026
We found a MUCH more compelling reason to care about this - we had a collaboration where there were tons of money ($10-20 on a good run, sometimes much more on a bad run, for something we usually pay <$10/year for) on S3 Tier 2 requests, that we are reasonably certain were due to this because they have many thousands of folders in the second level of their bucket. Eep!