Skip to content

Rolling file appender overgrows x.log.0 on restart when max_file_size is set #153

Closed

Description

@Mota-Link

Hello, and thanks for the great project.

I believe I’ve found an issue in the rolling file appender when max_file_size is configured.

Problem description

  • When rotation happens during a run, files are created as x.log.0, x.log.1, ...
  • After the program exits and restarts, the appender starts writing to x.log.0 again in append mode.
  • It continues appending to x.log.0 until the amount of data written in this new run reaches max_file_size, and only then rotates to the next file.
  • As a result, x.log.0 can grow to roughly 2 × max_file_size after a single restart. If the program is restarted multiple times, x.log.0 keeps growing cumulatively beyond the configured limit.

Expected behavior

  • If max_file_size is set, the appender should discover the latest existing file on startup:
    • Scan x.log.0, x.log.1, ... to find the highest index n that exists.
    • If x.log.n is not yet full (size < max_file_size), resume appending to x.log.n.
    • If x.log.n is already full (size >= max_file_size), roll to x.log.(n+1) and start a new file.
  • In short, resume from the latest index and honor the size limit across restarts.

I have attached a minimal reproducible example, adapted from the example code of rolling_file. You can reproduce the issue simply by running the program multiple times.

use logforth::append::rolling_file::RollingFileBuilder;
use logforth::layout::JsonLayout;

fn main() {
    let (rolling_writer, _guard) = RollingFileBuilder::new("logs")
        .layout(JsonLayout::default())
        .max_file_size(500)
        .filename_prefix("app_log")
        .build()
        .unwrap();

    logforth::builder()
        .dispatch(|d| d.filter(log::LevelFilter::Trace).append(rolling_writer))
        .apply();

    let repeat = 1;

    for i in 0..repeat {
        log::error!("Hello error!");
        log::warn!("Hello warn!");
        log::info!("Hello info!");
        log::debug!("Hello debug!");
        log::trace!("Hello trace!");

        if i + 1 < repeat {
            std::thread::sleep(std::time::Duration::from_secs(10));
        }
    }
}

Could you please confirm whether this is the intended behavior or if it might be a bug?

Thank you!

Activity

  1. tisonkun commented on Aug 27, 2025

    @tisonkun
    Contributor

    Looks like a bug - the current_filesize is set to 0 each time the state gets initialized, while we'd use the real current size if the file already exists.

    let current_filesize = 0;

  2. added theissue type on Aug 27, 2025
  3. tisonkun commented on Aug 27, 2025

    @tisonkun
    Contributor

    Or we may resolve the initial cnt to always use a new file on started.

  4. Mota-Link commented on Aug 28, 2025

    @Mota-Link
    Author

    I’ve noticed that handling user manipulation of the log directory can increase the complexity of log rotation significantly. From reviewing the delete_oldest_logs function, it seems that file rotation currently depends on the files’ created time, not strictly on the numeric file index.

    For example, suppose at program termination the log directory contains files like x.log.9, x.log.10, and x.log.11 (with max_size and max_files configured). If a user manually adds a file such as x.log.2 during downtime, the rotation logic—which relies on the latest creation time to determine the file counter—might start incrementing from x.log.2 upon restart. This still causes issues when rolling over past existing files like x.log.9.

    Alternatively, trying to determine the latest log file by filename isn’t trivial, since simple lexicographical sorting would treat "2" as greater than "11". To get correct behavior, every file’s suffix has to be parsed as a number. If a user then creates x.log.13 during runtime, this introduces more corner cases that are hard to handle robustly.

    I think issue #143 Log file name style is a great suggestion. Adopting a scheme with a fixed set of sequence number files (e.g., only x.log.0 to x.log.2) would prevent users from manually creating files with smaller sequence numbers, and we wouldn’t need to track or consider extraneous higher-index files. This would greatly simplify the rotation logic and make it less error-prone.

  5. tisonkun commented on Sep 7, 2025

    @tisonkun
    Contributor

    Implementing at #159

    @Mota-Link welcome to give a review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions