Skip to content

[FEATURE] Extend rate limiting with Exponentially Weighted Moving Average (EWMA) #404

Closed

Description

@cameri

Is your feature request related to a problem? Please describe.
The current Sliding Window Rate Limiter implementation in Nostream is non-performant under heavy load (thousands of clients). Because this method requires storing individual request metadata or counters within a specific timeframe, it leads to high memory consumption in Redis and increased latency during window calculations. As the relay scales, the memory overhead becomes a significant bottleneck.

Describe the solution you'd like
I would like to extend the rate limiter interface to include an Exponentially Weighted Moving Average (EWMA) implementation. EWMA is a "memoryless" rate limiting algorithm that only requires storing two values per key (the current rate and a timestamp), making it significantly more performant and memory-efficient than sliding windows.

The EWMA Formula

The rate is updated upon each new request using an exponential decay function. When a request arrives, the new rate is calculated as:

$$R_{new} = R_{old} \times e^{-\lambda \Delta t} + 1$$

Where:

  • $R_{new}$ is the updated rate.
  • $R_{old}$ is the rate stored from the previous request.
  • $\Delta t$ is the time elapsed since the last request.
  • $\lambda$ is the decay constant, which is determined by the user-configurable half-life ($t_{1/2}$):

$$\lambda = \frac{\ln(2)}{t_{1/2}}$$

Requirements

  • Configurability: Users must be able to configure the half-life in the relay settings.
  • Default Behavior: Nostream should use the new EWMA rate limiter by default.
  • Legacy Support: Relay operators should be able to switch back to the sliding_window via a configuration setting (e.g., LIMITER_STRATEGY).
  • Expected Outcomes: * A new EWMA rate limiter implementation.
    • Full unit test coverage for the decay logic.
    • Integration tests ensuring the relay correctly respects the chosen strategy and limits.

Describe alternatives you've considered

  • Token Bucket: While effective, it typically requires more complex Redis Lua scripts to handle atomic updates of tokens and timestamps compared to the mathematical simplicity of EWMA.
  • Fixed Window: Too inaccurate as it allows double the rate limit at window boundaries.

Additional context
This change is critical for high-traffic relays where Redis memory usage is currently a primary cost and performance driver. By moving to a mathematical decay model, we can handle thousands of concurrent clients with near-constant memory overhead per user.

Activity

  1. self-assigned this
    on Apr 9, 2026
  2. saniddhyaDubey commented on Apr 9, 2026

    @saniddhyaDubey
    Contributor

    assign!

  3. archief2910 commented on Apr 9, 2026

    @archief2910
    Contributor

    i would like to work on this issue

  4. saniddhyaDubey commented on Apr 12, 2026

    @saniddhyaDubey
    Contributor

    Hey @cameri , completed the implementation of EWMA based rate-limiting. Need your preference on few things:

    • currently, there's no integration test scenario in any feature testing rate-limiter, so how would you prefer to add for this ? A separate scenario more focused on rate limiter, or change a specific NIP feature.
    • In general, EWMA is kinda more strict than sliding window for bursty traffics. So, for Configurability, do you want per service level half-life or some other kind of structure ?
    • Also, the implementation wraps the entire per request operation in a custom lua script for atomicity. Is that okay or does that violates the code structure -- the ICacheAdapter has a new function, hitEWMA, which runs this script ?

    Would love your opinion on this one to complete this issue and open the PR.

  5. cameri commented on Apr 19, 2026

    @cameri
    OwnerAuthor

    @saniddhyaDubey amazing work, thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions