Skip to content

DOWNLOAD_FILES doesn't fully bypass S3FS #132

Closed

Description

@ErinWeisbart

If DOWNLOAD_FILES is True in the config, it looks to the load_data_csv for which files to download. However, it's looking at os.path.join(DATA_ROOT,message['data_file']) and DATA_ROOT = '/home/ubuntu/bucket' so S3FS is still required to mount the bucket correctly before downloading the files locally.

It seems we change this behavior.

Activity

  1. ErinWeisbart commented on Sep 20, 2022

    @ErinWeisbart
    MemberAuthor

    Fixed in #130.

  2. linked a pull request that will close this issuecorrect role handling and S3FS bypass #130on Sep 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions