Create new efficient batching implementation for Pub/Sub and Logging #3003

Activity

1. garrettjonesgoogle

   added

   type: feature request‘Nice-to-have’ improvement, new feature or different behavior or design.‘Nice-to-have’ improvement, new feature or different behavior or design.

   api: pubsubIssues related to the Pub/Sub API.Issues related to the Pub/Sub API.

   api: loggingIssues related to the Cloud Logging API.Issues related to the Cloud Logging API.

   priority: p1Important issue which blocks shipping the next release. Will be fixed prior to next release.Important issue which blocks shipping the next release. Will be fixed prior to next release.

   on Mar 6, 2018
2. garrettjonesgoogle

   assigned

   pongad

   on Mar 6, 2018
3. JustinBeckwith

   added

   🚨 criticalP0 critical issue. Requires immediate fixP0 critical issue. Requires immediate fix

   on Jun 8, 2018
4. pongad

   added

   priority: p2Moderately-important priority. Fix may not be included in next release.Moderately-important priority. Fix may not be included in next release.

   and removed

   🚨 criticalP0 critical issue. Requires immediate fixP0 critical issue. Requires immediate fix

   priority: p1Important issue which blocks shipping the next release. Will be fixed prior to next release.Important issue which blocks shipping the next release. Will be fixed prior to next release.

   on Jun 26, 2018
5. JustinBeckwith

   added

   🚨 criticalP0 critical issue. Requires immediate fixP0 critical issue. Requires immediate fix

   on Sep 2, 2018
6. theacodes

   unassigned

   pongad

   on Sep 28, 2018
7. sduskis

   added

   api: bigtableIssues related to the Bigtable API.Issues related to the Bigtable API.

   and removed

   🚨 criticalP0 critical issue. Requires immediate fixP0 critical issue. Requires immediate fix

   priority: p2Moderately-important priority. Fix may not be included in next release.Moderately-important priority. Fix may not be included in next release.

   on Oct 9, 2018
8. garrettjonesgoogle

   assigned

   sduskis

   on Jun 8, 2019
9. sduskis commented on Jun 12, 2019

   

   sduskis

   on Jun 12, 2019

   Contributor

   More actions

   This is in progress for Cloud Bigtable: https://github.com/googleapis/gax-java/pulls?utf8=%E2%9C%93&q=is%3Apr+batcher

   This will land eventually.
10. sduskis

    removed their assignment

    on Jun 12, 2019
11. IRus commented on Sep 13, 2019

    

    IRus

    on Sep 13, 2019

    More actions

    Does this new API will include ability to publish list of messages? Like fun publish(messages: List<PubsubMessage>): ApiFuture<List<String>>
12. rahulKQL commented on Sep 20, 2019

    

    rahulKQL

    on Sep 20, 2019

    Contributor

    More actions

    @IRus Sorry for the late response. As of now, we have yet to implement this feature on google-cloud-pubsub.

    The surface for that is not yet finalized but it would support something similar to Batcher<PubsubMessage, String>, then one can perform batching with this batcher.

    👍React with 👍1Reacted by Ruslan Ibrahimau
13. jmouvill commented on Oct 11, 2019

    

    jmouvill

    on Oct 11, 2019

    More actions

    What would be the difference with the current BatchSettings available on the Publisher's builder ?
    i.e:

    Publisher
          .newBuilder(topic)
          .setBatchingSettings(
            BatchingSettings.newBuilder.
              setFlowControlSettings(...)
              .setElementCountThreshold(...)
              .setRequestByteThreshold(...)
              .setDelayThreshold(...)
              .build
        )
    

    where you can specify an elementCountThreshold, requestByteThreshold etc ?
14. rahulKQL commented on Oct 11, 2019

    

    rahulKQL

    on Oct 11, 2019

    Contributor

    More actions

    @jmouvill This feature has been little delayed for Pubsub, but I would say BatchingSettings configurations would be similarly respected in new batching impl. We are planning to use the same BatchingSettings to set thresholds for Batcherinstance.
15. chingor13 commented on Jan 7, 2020

    

    chingor13

    on Jan 7, 2020

    Contributor

    More actions

    This issue has been split into the 2 linked tracking issues to implement in the logging and pubsub clients respectively.
16. chingor13

    closed this as completedon Jan 7, 2020
17. JustinBeckwith

    assigned

    chingor13

    on Feb 1, 2021
18. chingor13

    added a commit that references this issue on Feb 20, 2026

    chore(deps): update dependency com.google.cloud:libraries-bom to v26.…

    ...

    efbd8a6
19. suztomo

    added a commit that references this issue on Mar 9, 2026

    chore(deps): update dependency com.google.cloud:google-cloud-bigquery…

    ...

    904ed98
20. lqiu96

    added a commit that references this issue on Mar 20, 2026

    chore(deps): update dependency com.google.cloud:libraries-bom to v26.…

    ...

    6542f48
21. chingor13

    added a commit that references this issue on Mar 24, 2026

    chore: update Spanner owner to rahul2393 (#3003)

    eec728b