You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Rolling file appender overgrows x.log.0 on restart when max_file_size is set #153
I believe I’ve found an issue in the rolling file appender when max_file_size is configured.
Problem description
When rotation happens during a run, files are created as x.log.0, x.log.1, ...
After the program exits and restarts, the appender starts writing to x.log.0 again in append mode.
It continues appending to x.log.0 until the amount of data written in this new run reaches max_file_size, and only then rotates to the next file.
As a result, x.log.0 can grow to roughly 2 × max_file_size after a single restart. If the program is restarted multiple times, x.log.0 keeps growing cumulatively beyond the configured limit.
Expected behavior
If max_file_size is set, the appender should discover the latest existing file on startup:
Scan x.log.0, x.log.1, ... to find the highest index n that exists.
If x.log.n is not yet full (size < max_file_size), resume appending to x.log.n.
If x.log.n is already full (size >= max_file_size), roll to x.log.(n+1) and start a new file.
In short, resume from the latest index and honor the size limit across restarts.
I have attached a minimal reproducible example, adapted from the example code of rolling_file. You can reproduce the issue simply by running the program multiple times.
Looks like a bug - the current_filesize is set to 0 each time the state gets initialized, while we'd use the real current size if the file already exists.
I’ve noticed that handling user manipulation of the log directory can increase the complexity of log rotation significantly. From reviewing the delete_oldest_logs function, it seems that file rotation currently depends on the files’ created time, not strictly on the numeric file index.
For example, suppose at program termination the log directory contains files like x.log.9, x.log.10, and x.log.11 (with max_size and max_files configured). If a user manually adds a file such as x.log.2 during downtime, the rotation logic—which relies on the latest creation time to determine the file counter—might start incrementing from x.log.2 upon restart. This still causes issues when rolling over past existing files like x.log.9.
Alternatively, trying to determine the latest log file by filename isn’t trivial, since simple lexicographical sorting would treat "2" as greater than "11". To get correct behavior, every file’s suffix has to be parsed as a number. If a user then creates x.log.13 during runtime, this introduces more corner cases that are hard to handle robustly.
I think issue #143 Log file name style is a great suggestion. Adopting a scheme with a fixed set of sequence number files (e.g., only x.log.0 to x.log.2) would prevent users from manually creating files with smaller sequence numbers, and we wouldn’t need to track or consider extraneous higher-index files. This would greatly simplify the rotation logic and make it less error-prone.
Activity
tisonkun commentedon Aug 27, 2025
Looks like a bug - the current_filesize is set to 0 each time the state gets initialized, while we'd use the real current size if the file already exists.
logforth/src/append/rolling_file/rolling.rs
Line 189 in ad597df
tisonkun commentedon Aug 27, 2025
Or we may resolve the initial
cntto always use a new file on started.Mota-Link commentedon Aug 28, 2025
I’ve noticed that handling user manipulation of the log directory can increase the complexity of log rotation significantly. From reviewing the
delete_oldest_logsfunction, it seems that file rotation currently depends on the files’ created time, not strictly on the numeric file index.For example, suppose at program termination the log directory contains files like
x.log.9,x.log.10, andx.log.11(withmax_sizeandmax_filesconfigured). If a user manually adds a file such asx.log.2during downtime, the rotation logic—which relies on the latest creation time to determine the file counter—might start incrementing fromx.log.2upon restart. This still causes issues when rolling over past existing files likex.log.9.Alternatively, trying to determine the latest log file by filename isn’t trivial, since simple lexicographical sorting would treat
"2"as greater than"11". To get correct behavior, every file’s suffix has to be parsed as a number. If a user then createsx.log.13during runtime, this introduces more corner cases that are hard to handle robustly.I think issue #143 Log file name style is a great suggestion. Adopting a scheme with a fixed set of sequence number files (e.g., only
x.log.0tox.log.2) would prevent users from manually creating files with smaller sequence numbers, and we wouldn’t need to track or consider extraneous higher-index files. This would greatly simplify the rotation logic and make it less error-prone.tisonkun commentedon Sep 7, 2025
Implementing at #159
@Mota-Link welcome to give a review