Hello,
Using Fedora 7, I am trying to see how we can rotate some log files each day, but keep a month's worth of files.
Logrotate can rotate the files daily, and I can specify something like:
daily start 1 rotate 30
But this does not account for month's of 31 days, or February with 28/29 days. Likewise if I set 'rotate 31' then this is not going to work on those months of just 30 days.
Does anyone have any ideas about this?
Thanks,
John.
John Horne wrote:
Hello,
Using Fedora 7, I am trying to see how we can rotate some log files each day, but keep a month's worth of files.
Logrotate can rotate the files daily, and I can specify something like:
daily start 1 rotate 30
But this does not account for month's of 31 days, or February with 28/29 days. Likewise if I set 'rotate 31' then this is not going to work on those months of just 30 days.
Does anyone have any ideas about this?
Thanks,
John.
You can control how many old log files are kept by changing "rotate 4" to how many weeks of backlogs you want to keep, This is in /etc/logrotate.conf. You can control individual logs by editing the corresponding file in /etc/logrotate.d and adding/changing the rotate line. You can find more information by running "man logrotate".
Mikkel
On Thu, 2007-08-23 at 09:58 -0500, Mikkel L. Ellertson wrote:
John Horne wrote:
Hello,
Using Fedora 7, I am trying to see how we can rotate some log files each day, but keep a month's worth of files.
Logrotate can rotate the files daily, and I can specify something like:
daily start 1 rotate 30
But this does not account for month's of 31 days, or February with 28/29 days. Likewise if I set 'rotate 31' then this is not going to work on those months of just 30 days.
Does anyone have any ideas about this?
Thanks,
John.
You can control how many old log files are kept by changing "rotate 4" to how many weeks of backlogs you want to keep, This is in /etc/logrotate.conf. You can control individual logs by editing the corresponding file in /etc/logrotate.d and adding/changing the rotate line. You can find more information by running "man logrotate".
? We want daily log files, so setting 'rotate 4' will only give us 4 log files. Using weeks is no good because a month is not a fixed number of weeks - 30 and 31 days are not 4 weeks and vice-versa.
John.
John Horne wrote:
On Thu, 2007-08-23 at 09:58 -0500, Mikkel L. Ellertson wrote:
You can control how many old log files are kept by changing "rotate 4" to how many weeks of backlogs you want to keep, This is in /etc/logrotate.conf. You can control individual logs by editing the corresponding file in /etc/logrotate.d and adding/changing the rotate line. You can find more information by running "man logrotate".
? We want daily log files, so setting 'rotate 4' will only give us 4 log files. Using weeks is no good because a month is not a fixed number of weeks - 30 and 31 days are not 4 weeks and vice-versa.
If you are using daily instead of weekly, then set rotate to 31 to keep the max number of days a month can have. Then add a monthly cron job to archive the month of logs if you want to keep them. By using 31 as the number of logs to keep, you insure you have at least a months worth of logs. If the month has less the 31 days, you will have 1 or more daily logs from last month left over.
Mikkel
M
If you are using daily instead of weekly, then set rotate to 31 to keep the max number of days a month can have. Then add a monthly cron job to archive the month of logs if you want to keep them. By using 31 as the number of logs to keep, you insure you have at least a months worth of logs. If the month has less the 31 days, you will have 1 or more daily logs from last month left over.
you could probably the unwanted log files by having the cron job, instead of blindly copying all 31 files, just copy the first X where X was the number of days in the last month.
Chris
Mikkel L. Ellertson wrote:
John Horne wrote:
On Thu, 2007-08-23 at 09:58 -0500, Mikkel L. Ellertson wrote:
You can control how many old log files are kept by changing "rotate 4" to how many weeks of backlogs you want to keep, This is in /etc/logrotate.conf. You can control individual logs by editing the corresponding file in /etc/logrotate.d and adding/changing the rotate line. You can find more information by running "man logrotate".
? We want daily log files, so setting 'rotate 4' will only give us 4 log files. Using weeks is no good because a month is not a fixed number of weeks - 30 and 31 days are not 4 weeks and vice-versa.
If you are using daily instead of weekly, then set rotate to 31 to keep the max number of days a month can have. Then add a monthly cron job to archive the month of logs if you want to keep them. By using 31 as the number of logs to keep, you insure you have at least a months worth of logs. If the month has less the 31 days, you will have 1 or more daily logs from last month left over.
I forgot to add that the files will be numbered in reverse order, because of the way logrotate rotates files...
Another way to do it would be to have a postrotate script that renames <log>.1 to <log>.date - you can then archive based on date. But for the logs to be accurate, you are going to have to change the default time when cron.daily is run, or have logrotate run by itself with its own cron entry.
Mikkel
On Thu, 2007-08-23 at 15:17 -0500, Mikkel L. Ellertson wrote:
Mikkel L. Ellertson wrote:
John Horne wrote:
On Thu, 2007-08-23 at 09:58 -0500, Mikkel L. Ellertson wrote:
You can control how many old log files are kept by changing "rotate 4" to how many weeks of backlogs you want to keep, This is in /etc/logrotate.conf. You can control individual logs by editing the corresponding file in /etc/logrotate.d and adding/changing the rotate line. You can find more information by running "man logrotate".
? We want daily log files, so setting 'rotate 4' will only give us 4 log files. Using weeks is no good because a month is not a fixed number of weeks - 30 and 31 days are not 4 weeks and vice-versa.
If you are using daily instead of weekly, then set rotate to 31 to keep the max number of days a month can have. Then add a monthly cron job to archive the month of logs if you want to keep them. By using 31 as the number of logs to keep, you insure you have at least a months worth of logs. If the month has less the 31 days, you will have 1 or more daily logs from last month left over.
I forgot to add that the files will be numbered in reverse order, because of the way logrotate rotates files...
This behavior is configurable. Add the "dateext" option to the profile for your logs and the extension will be -yyyymmdd instead of .nn . Handy because the files sort in forward order (except the current one).
Another way to do it would be to have a postrotate script that renames <log>.1 to <log>.date - you can then archive based on date. But for the logs to be accurate, you are going to have to change the default time when cron.daily is run, or have logrotate run by itself with its own cron entry.
Mikkel
John Horne wrote:
? We want daily log files, so setting 'rotate 4' will only give us 4 log files. Using weeks is no good because a month is not a fixed number of weeks - 30 and 31 days are not 4 weeks and vice-versa.
I hope you realise that if set to daily, logrotate happens at 4:02AM. That means yesterday's log actually contains 4 hours of 'today' and is missing 4 hours from yesterday. If you run automated report generation from the logs, eg. SARG, the report will be incomplete unless the utility that generates reports is aware of log rotation and takes it into consideration.
Vivek J. Patankar wrote:
I hope you realise that if set to daily, logrotate happens at 4:02AM. That means yesterday's log actually contains 4 hours of 'today' and is missing 4 hours from yesterday. If you run automated report generation from the logs, eg. SARG, the report will be incomplete unless the utility that generates reports is aware of log rotation and takes it into consideration.
This can be changed - either change the entry in /etc/crontab, or remove logrotate from /etc/cron.daily and put an entry in /etc/cron.d to run it at 12:01 or something like that.
Mikkel
Mikkel L. Ellertson wrote:
Vivek J. Patankar wrote:
I hope you realise that if set to daily, logrotate happens at 4:02AM. That means yesterday's log actually contains 4 hours of 'today' and is missing 4 hours from yesterday. If you run automated report generation from the logs, eg. SARG, the report will be incomplete unless the utility that generates reports is aware of log rotation and takes it into consideration.
This can be changed - either change the entry in /etc/crontab, or remove logrotate from /etc/cron.daily and put an entry in /etc/cron.d to run it at 12:01 or something like that.
Doesn't quite work for me. I have SARG generate daily reports for my proxy usage and if logrotate happens before the SARG process is started, then it generates reports from what is currently in access.log, ie. it's doesn't fetch data from the rotated logs. Other utilities for log analysis may behave differently.
Vivek J. Patankar wrote:
Mikkel L. Ellertson wrote:
Vivek J. Patankar wrote:
I hope you realise that if set to daily, logrotate happens at 4:02AM. That means yesterday's log actually contains 4 hours of 'today' and is missing 4 hours from yesterday. If you run automated report generation from the logs, eg. SARG, the report will be incomplete unless the utility that generates reports is aware of log rotation and takes it into consideration.
This can be changed - either change the entry in /etc/crontab, or remove logrotate from /etc/cron.daily and put an entry in /etc/cron.d to run it at 12:01 or something like that.
Doesn't quite work for me. I have SARG generate daily reports for my proxy usage and if logrotate happens before the SARG process is started, then it generates reports from what is currently in access.log, ie. it's doesn't fetch data from the rotated logs. Other utilities for log analysis may behave differently.
Well, you could run a daily cron job that pulls everything with yesterday's date out of the log and the rotated log, and puts the results in a file with the correct date as part of the name. Or, if you want to archive the entire month in one file, append the output to a fine. Then have a monthly job that renames the file to the correct mouth and year, and create a new, empty file to collect the next month's job. That way, the daily cron job doesn't have to know the month.
Mikkel
Mikkel L. Ellertson wrote:
This can be changed - either change the entry in /etc/crontab, or remove logrotate from /etc/cron.daily and put an entry in /etc/cron.d to run it at 12:01 or something like that.
Doesn't quite work for me. I have SARG generate daily reports for my proxy usage and if logrotate happens before the SARG process is started, then it generates reports from what is currently in access.log, ie. it's doesn't fetch data from the rotated logs. Other utilities for log analysis may behave differently.
Well, you could run a daily cron job that pulls everything with yesterday's date out of the log and the rotated log, and puts the results in a file with the correct date as part of the name. Or, if you want to archive the entire month in one file, append the output to a fine. Then have a monthly job that renames the file to the correct mouth and year, and create a new, empty file to collect the next month's job. That way, the daily cron job doesn't have to know the month.
The possibilities are endless. :-)
On Thu, 2007-08-23 at 16:15 +0100, John Horne wrote:
On Thu, 2007-08-23 at 09:58 -0500, Mikkel L. Ellertson wrote:
John Horne wrote:
Hello,
Using Fedora 7, I am trying to see how we can rotate some log files each day, but keep a month's worth of files.
Logrotate can rotate the files daily, and I can specify something like:
daily start 1 rotate 30
But this does not account for month's of 31 days, or February with 28/29 days. Likewise if I set 'rotate 31' then this is not going to work on those months of just 30 days.
Does anyone have any ideas about this?
Thanks,
John.
You can control how many old log files are kept by changing "rotate 4" to how many weeks of backlogs you want to keep, This is in /etc/logrotate.conf. You can control individual logs by editing the corresponding file in /etc/logrotate.d and adding/changing the rotate line. You can find more information by running "man logrotate".
? We want daily log files, so setting 'rotate 4' will only give us 4 log files. Using weeks is no good because a month is not a fixed number of weeks - 30 and 31 days are not 4 weeks and vice-versa.
John.
Hi John,
the short answer, which I haven't seen anybody yet give you, is, logrotate can't do what you want to do. You're going to have to take care of things yourself with some scripting.
Andy
On Thu, 2007-08-23 at 15:27 +0100, John Horne wrote:
this does not account for month's of 31 days, or February with 28/29 days
I'd guess that you'd have to do what banks do: Do your activities on the first of the month. If you do it close after midnight, you keep the last month's work, and start a new month's, hoping that you want nothing in the last few moments to be part of this month's data.
On Fri, 2007-08-24 at 00:34 +0930, Tim wrote:
On Thu, 2007-08-23 at 15:27 +0100, John Horne wrote:
this does not account for month's of 31 days, or February with 28/29 days
I'd guess that you'd have to do what banks do: Do your activities on the first of the month. If you do it close after midnight, you keep the last month's work, and start a new month's, hoping that you want nothing in the last few moments to be part of this month's data.
Not sure how this is going to help. How to do I configure this with logrotate?
As far as I am aware logrotate runs daily, and since we want daily log files I don't see how logrotate is going to know what a 'month' is on a daily basis. I agree it knows what a 'month' is for monthly log file rotation, simply because the month (name or number) changes.
What is needed it seems is logrotate to know that the month has changed, but to keep to the daily log file rotation regardless of what 'rotate' value is given. That way I get 31 log files for a 31-day month, 30 log files for a 30-day month and so on.
John.
Thu, 2007-08-23 at 15:27 +0100, John Horne wrote:
this does not account for month's of 31 days, or February with 28/29 days
Tim:
I'd guess that you'd have to do what banks do: Do your activities on the first of the month. If you do it close after midnight, you keep the last month's work, and start a new month's, hoping that you want nothing in the last few moments to be part of this month's data.
Another thought: Post-process just the .1 log after the rotation, so the system rotations doesn't upset your logs (you don't miss things). Use a script that writes your own logs to dated files, appending the data to the right date files (that takes care of some of yesterdays and some of todays messages in the processed log). Handled your own dated logs separately from the system, and let the system rotate and remove its own logs as it usually does.