Log4j Rotation Example (original) (raw)

This article is a tutorial about log rotation in Log4j. In this tutorial, we are going to configure log4j via property files.

1. Introduction

Log4J(Java) is widely used logging framework for Java. It continues to grow continuously with recent upgrade of Log4j2. We want to rotate log files to avoid log file accumulation and easily segregate logs under the log folder.

Log4j supports logging with help of Appender and Layouts. Layout specifies the display format of the logs. Commonly used Layout for Log4j is PatternLayout. A sample pattern is %d [%t] %-5p (%F: %L) – %m%n. The format strings for the pattern are as follows:

• Date – Full date till micro seconds
• Thread – JVM thread logging the output
• Logging Mode – INFO/ERROR/DEBUG/WARN
• Class – Java Class logging the output
• Line number – Line number in java class
• Message – The message logged
• Default line separator -/n unless specified otherwise

Logs Example

Appender delivers the log to the logging destination along with options to fine-tune the logging mechanism. Appenders generally have lifecycle configuration and filtering support. Filtering enables to filter messages whose logging mode does not match level configured. Two of the widely used appenders are DailyRollingFileAppender and RollingFileAppender which we will see below.

2. RollingFileAppender

RollingfileAppender rotates log files based on file size. MaxFileSize indicates the maximum size of file while MaxBackupIndex indicates the amount of files to be kept in the log folder. A sample property file is given below:

log4j.properties

log4j.rootLogger=INFO, fileLogger log4j.appender.fileLogger=org.apache.log4j.RollingFileAppender log4j.appender.fileLogger.layout=org.apache.log4j.PatternLayout log4j.appender.fileLogger.layout.ConversionPattern=%d [%t] %-5p (%F:%L) - %m%n log4j.appender.fileLogger.File=example.log log4j.appender.fileLogger.MaxFileSize=1KB log4j.appender.fileLogger.MaxBackupIndex=5

RollingFileAppender Example Log

After 5 logs are generated, it automatically starts deleting the log files. So at any point in time, you can see a maximum of 5 logs and not more than that. Size is capped in the example at 1KB. Ideally size would be around the order of MB and is based on application needs.

3. DailyRollingFileAppender

DailyRollingFileAppender rotates log files based on frequency of time allowing customization upto minute. Date Patterns allowed as part of the Appender are as follows:

• yyyy-MM Roll over to new log file beginning on first day of every month
• yyyy-ww Roll over to new log file beginning on first day of every week
• yyyy-MM-dd Roll over daily
• yyyy-MM-dd-a Roll over on midday and midnight
• yyyy-MM-dd-HH Roll over every hour
• yyyy-MM-dd-HH-mm Roll over every minute

log4j.properties

log4j.rootLogger=INFO, fileLogger log4j.appender.fileLogger.layout=org.apache.log4j.PatternLayout log4j.appender.fileLogger.layout.ConversionPattern=%d [%t] %-5p (%F:%L) - %m%n log4j.appender.fileLogger.File=example.log log4j.appender.fileLogger=org.apache.log4j.DailyRollingFileAppender log4j.appender.fileLogger.datePattern='.'yyyy-MM-dd-HH-mm

DailyRollingFileAppender Log examples

Default date pattern is yyyy-MM-dd ie rolls every day. One downside of this appender is that old log file deletion does not happen automatically. Alternative is to implement your own custom appender.

4. Custom Appender

Custom Appender can be created by extending AppenderSkeleton class or AbstractAppender. Typically we can write our own version of doAppend. Our goal is to attach maxbackupindex capability to DailyRollingFileAppender. So it is better to extend DailyRollingFileAppender Class and provide our implementation.

Custom Appender

package org.apache.log4j;

import java.io.File; import java.io.IOException; import java.util.ArrayList; import java.util.Arrays; import java.util.List;

import org.apache.log4j.helpers.LogLog;

public class CustomAppender extends DailyRollingFileAppender {

private int maxBackupIndex;

public void setMaxBackupIndex(int maxBackupIndex) {
    this.maxBackupIndex = maxBackupIndex;
}

@Override
public void rollOver() throws IOException {
    super.rollOver();
    File file = new File(fileName);
    List<String> files = new ArrayList<>();
    File[] dirFiles = new File(file.getAbsolutePath()).getParentFile().listFiles();
    if (dirFiles != null && dirFiles.length > 0) {
        Arrays.sort(dirFiles, (a, b) -> Long.compare(a.lastModified(), b.lastModified()));
        for (File allFile : dirFiles) {
            if (allFile.getName().contains(fileName)) {
                files.add(allFile.getAbsolutePath());
            }
        }
    }
    if (files.size() > maxBackupIndex+1) {
        File deleteFile = new File(files.get(0));
        LogLog.debug("delete result for"+deleteFile.getAbsolutePath()+" is "+deleteFile.delete());
        files.remove(0);
    }

}

}

• Line 1- Package declaration is log4j. We need it to override the rollover method as visibility cannot extend in another package
• Line 13- takes an option to control the maximum files to be backed up other than current file
• Line 20,21- Override rollover method. Rollover gets called whenever a file needs to be rolledover. We call DailyRollingFileAppender‘s method to finish its rollover process i.e. it renames the old file based on time and creates a new file with the filename specified
• Line 26 – Sort the file names by ascending order of created time.
• Line 28,29 – Filter filenames containing our log file name. In a dedicated log folder, only our log files would be there and this step might be unnecessary. It is added as caution in case the specified folder contains other files.
• Line 33,37 – delete the oldest log file and remove it from list

This enhances Dailyrollingfileappender. Typical use case would be to keep 20 days of log and delete the rest of them.

log4j.properties

log4j.rootLogger=INFO, fileLogger log4j.appender.fileLogger.layout=org.apache.log4j.PatternLayout log4j.appender.fileLogger.layout.ConversionPattern=%d [%t] %-5p (%F:%L) - %m%n log4j.appender.fileLogger.File=example.log log4j.appender.fileLogger=org.apache.log4j.CustomAppender log4j.appender.fileLogger.datePattern='.'yyyy-MM-dd-HH-mm

Only change is to include our Custom appender. Date Pattern usage is similar to DailyRollingFileAppender and that behaviour is extended in our custom class.

5. Execution Steps

1. Create a simple Java Project in eclipse
2. Download log4j jar and include in your project by clicking on Project Properties -> Java Build Path -> Libraries -> Add Jars
3. Copy the below java code in project
4. Include a single appender in your project at a time
5. Stop the running project in eclipse after 5 minutes to see various logs

Java Logger Class

package com.jcg.examples; package com.jcg.examples; import org.apache.log4j.Logger; public class LoggerMain { public static final Logger logger = Logger.getLogger(LoggerMain.class); public static void main(String[] args) { while (true) { logger.info("This is a test log"); } } }

• Line 1- Create a simple class for logging purpose
• Line 5- Creating logger for the newly created class by using factory method
• Line 8- use info method to log a message

The whole code runs in a infinite loop and hence it is necessary to terminate manually.

6. Summary

In this tutorial we saw how to achieve log4j rotation via property files. We saw two types of existing log4j rotation appenders and a custom appender with examples.

7. Download the Source Code

Download
You can download the full source code of this example here: Log4jExample

Rajagopal works in software industry solving enterprise-scale problems for customers across geographies specializing in distributed platforms. He holds a masters in computer science with focus on cloud computing from Illinois Institute of Technology. His current interests include data science and distributed computing.