Skip to content

roate_size not working as expected #4507

Closed

Description

@carylewis

Describe the bug

After adding the following to the fluentd.conf file:

<system>
  <log>
    rotate_age 7
    rotate_size 8192
  </log>
</system>
<source>
  @type tail
  path /var/log/pods/*/*/*.log
  exclude_path ["/var/log/pods/*/*fluentd*/*.log"]
  pos_file /var/log/td-agent/container.log.pos
  tag omera.*
<parse>
    @type none
</parse>
</source>
 
<filter names.**>
  @type record_transformer
  enable_ruby
<record>
    file_name ${tag.split('.')[5]}
</record>
</filter>
 
<match fluentd.**>
  @type null
</match>
 
<match names.**>
  @type file
  append true
  path /fluentd/log/${file_name}.%Y%m%d.log
<buffer file_name,time>
    timekey 1d
    timekey_use_utc true
    timekey_wait 5s
    chunk_limit_size 8192
    flush_at_shutdown true
    flush_interval 1s
    flush_mode interval
</buffer>
<format>
    @type out_file
    time_slice_format %Y%m%d
    time_format %Y-%m-%dT%H:%M:%S%z
</format>
</match>

My output log files continue to grow until the path forces a rollover at midnight.

This is an example config file, I set it to 8192 to try to see if it would rotate - I tried adjusting the chunk_limit_size to match the rotate_size but it had no effect.

Is this the expected behaviour?

To Reproduce

Run fluentd with the supplied config file.

Expected behavior

I was hoping that fluentd would rotate the logs based on the specified rotate_size.

Your Environment

- Fluentd version: 1.16.5

Your Configuration

fluented is running as a k8s pod

Your Error Log

I see it is picking up the rotate_size

2024-05-25 14:35:47 +0000 [info]: #0 init worker0 logger path=nil rotate_age=7 rotate_size=8192

Additional context

No response

Activity

  1. locked and limited conversation to collaborators on May 27, 2024
  2. converted this issue into a discussion #4508 on May 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions