Skip to content

Performance issues in /src/python/tensorflow_cloud/core (by P3) #371

Open

Description

@DLPerf

Hello! I've found a performance issue in /tests/testdata/keras_tuner_cifar_example.py: batch() should be called before map(), which could make your program more efficient. Here is the tensorflow document to support it.

Detailed description is listed below:

  • .batch(BATCH_SIZE)(here) should be called before train_dataset.map(scale)(here).
  • .batch(BATCH_SIZE)(here) should be called before .map(scale)(here).

Besides, you need to check the function called in map()(e.g., scale called in .map(scale)) whether to be affected or not to make the changed code work properly. For example, if scale needs data with shape (x, y, z) as its input before fix, it would require data with shape (batch_size, x, y, z).

Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.

Activity

  1. DLPerf commented on Nov 4, 2021

    @DLPerf
    Author

    Hello, I'm looking forward to your reply~

  2. sharmadhiraj86 commented on Sep 11, 2022

    @sharmadhiraj86

    I would like to contribute in solving this issue. Can you please assign this issue to me? @DLPerf @lamberta @mihaimaruseac @jaeyounkim @martinwicke

  3. mihaimaruseac commented on Sep 12, 2022

    @mihaimaruseac

    We don't generally assign issue to contributors. Instead, please send a PR when ready.

  4. sahil270803 commented on Oct 14, 2024

    @sahil270803

    I resolved the performance issue by changing the order of operations in the dataset pipeline, calling .batch(BATCH_SIZE) before .map(scale). This modification improves efficiency by batching the data before applying the map function, reducing the number of individual function calls.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions