Skip to content

No parameter available to specify a jobId to Table.upload_from_file() #3574

Closed
Closed
No parameter available to specify a jobId to Table.upload_from_file()#3574
Assignees
tseaver
Labels
api: bigqueryIssues related to the BigQuery API.priority: p2Moderately-important priority. Fix may not be included in next release.type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.

Description

@mackenzieATA
mackenzieATA
opened on Jun 30, 2017

There is no way to send a jobId to the upload_from_file function.

My motivation is that I've had issues with uploading the same file twice, duplicating data in the target table. A suggested solution was to specify my own jobId as a hash of the file contents. That way, duplicate uploads would be rejected as having the same jobId. Without modifying the library, there is no ability to do this.

Activity

  1. dhermes
    added
    api: bigqueryIssues related to the BigQuery API.
    on Jun 30, 2017
  2. tseaver
    added
    type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.
    and removed
    api: bigqueryIssues related to the BigQuery API.
    on Jul 1, 2017

  3. tseaver commented on Jul 1, 2017

    @tseaver
    tseaver
    on Jul 1, 2017
    Contributor

    This should be pretty straightforward:

    • Update Table.upload_from_file to take an optional job_name property and pass it through to _configure_job_metadata, along with the table's project.
    • Update _configure_job_metadata to set the top-level jobReference IFF the job_name is not None.
    • Add test coverage.
  4. dhermes
    added
    api: bigqueryIssues related to the BigQuery API.
    on Jul 1, 2017
  5. lukesneeringer
    added
    priority: p2Moderately-important priority. Fix may not be included in next release.
    on Jul 5, 2017
  6. evaogbe
    self-assigned this
    on Jul 12, 2017
  7. tseaver
    closed this as completedin #3605on Jul 17, 2017
  8. theacodes
    unassigned
    evaogbe
    on Sep 28, 2018
  9. JustinBeckwith
    assigned
    tseaver
    on Feb 1, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

api: bigqueryIssues related to the BigQuery API.priority: p2Moderately-important priority. Fix may not be included in next release.type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions