log4j2: CSV Layouts: Save log events in csv format

Layouts are used to format log records. You can attach a layout to an appender to format log events. 

CSV layout is used to save log events in csv format. 

Following are the Maven dependencies i used.

<dependencies>
 <dependency>
  <groupId>org.apache.logging.log4j</groupId>
  <artifactId>log4j-api</artifactId>
  <version>2.4</version>
 </dependency>
 <dependency>
  <groupId>org.apache.logging.log4j</groupId>
  <artifactId>log4j-core</artifactId>
  <version>2.4</version>
 </dependency>
 <dependency>
  <groupId>org.apache.commons</groupId>
  <artifactId>commons-csv</artifactId>
  <version>1.2</version>
 </dependency>
</dependencies>

a. CsvParameterLayout

Convert an event's parameters into a CSV record, ignore the message.

int val1 = 10, val2 = 11, val3 = 12;

logger.trace("Trace Message!", val1, val2, val3);

Above statement logs only 10, 11, 12, it ignores the message.

log4j2.xml

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn" name="MyApp" packages="">
 <Appenders>
  <File name="my_file_appender" fileName="/Users/harikrishna_gurram/application.log">
   <CsvParameterLayout delimiter=","/>
  </File>
 </Appenders>

 <Loggers>
  <Root level="info">
   <AppenderRef ref="my_file_appender" />
  </Root>
 </Loggers>
</Configuration>

package log4j_tutorial;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

public class HelloWorld {
 private static final Logger logger = LogManager.getLogger();

 public static void main(String args[]) throws InterruptedException {
  int val1 = 10, val2 = 11, val3 = 12;

  logger.trace("Trace Message!", val1, val2, val3);
  logger.debug("Debug Message!", val1, val2, val3);
  logger.info("Info Message!", val1, val2, val3);
  logger.warn("Warn Message!", val1, val2, val3);
  logger.error("Error Message!", val1, val2, val3);
  logger.fatal("Fatal Message!", val1, val2, val3);

 }
 
}

Run HelloWorld application, you will get following data in application.log file.

$ cat application.log 
10,11,12
10,11,12
10,11,12
10,11,12

b. CsvLogEventLayout

CsvLogEventLayout  Produces a CSV record with the following fields:

a.   Time Nanos

b.   Time Millis

c.    Level

d.   Thread Name

e.   Formatted Message

f.     Logger FQCN

g.   Logger Name

h.   Marker

i.      Thrown Proxy

j.     Source

k.    Context Map

l.      Context Stack

For example,

int val1 = 10, val2 = 11, val3 = 12;

logger.info("val1={}, val2={}, val3={}", val1, val2, val3);

Above statement log events like following.

0,1444022794731,INFO,main,val1=10, val2=11, val3=12,org.apache.logging.log4j.spi.AbstractLogger,log4j_tutorial.HelloWorld,,,log4j_tutorial.HelloWorld.main(HelloWorld.java:14),{},[]

log4j2.xml

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn" name="MyApp" packages="">
 <Appenders>
  <File name="my_file_appender" fileName="/Users/harikrishna_gurram/application_1.log">
   <CsvLogEventLayout delimiter=","/>
  </File>
 </Appenders>

 <Loggers>
  <Root level="info">
   <AppenderRef ref="my_file_appender" />
  </Root>
 </Loggers>
</Configuration>

package log4j_tutorial;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

public class HelloWorld {
 private static final Logger logger = LogManager.getLogger();

 public static void main(String args[]) throws InterruptedException {
  int val1 = 10, val2 = 11, val3 = 12;

  logger.trace("val1={}, val2={}, val3={}", val1, val2, val3);
  logger.debug("val1={}, val2={}, val3={}", val1, val2, val3);
  logger.info("val1={}, val2={}, val3={}", val1, val2, val3);
  logger.warn("val1={}, val2={}, val3={}", val1, val2, val3);
  logger.error("val1={}, val2={}, val3={}", val1, val2, val3);
  logger.fatal("val1={}, val2={}, val3={}", val1, val2, val3);

 }

}

Run HelloWorld application, you will get following output in application_1.log.

$ cat application_1.log 
0,1444022794731,INFO,main,val1=10, val2=11, val3=12,org.apache.logging.log4j.spi.AbstractLogger,log4j_tutorial.HelloWorld,,,log4j_tutorial.HelloWorld.main(HelloWorld.java:14),{},[]
0,1444022794732,WARN,main,val1=10, val2=11, val3=12,org.apache.logging.log4j.spi.AbstractLogger,log4j_tutorial.HelloWorld,,,log4j_tutorial.HelloWorld.main(HelloWorld.java:15),{},[]
0,1444022794733,ERROR,main,val1=10, val2=11, val3=12,org.apache.logging.log4j.spi.AbstractLogger,log4j_tutorial.HelloWorld,,,log4j_tutorial.HelloWorld.main(HelloWorld.java:16),{},[]
0,1444022794733,FATAL,main,val1=10, val2=11, val3=12,org.apache.logging.log4j.spi.AbstractLogger,log4j_tutorial.HelloWorld,,,log4j_tutorial.HelloWorld.main(HelloWorld.java:17),{},[]

Following are the parameters to configure CsvParameterLayout and CsvLogEventLayout.

Parameter	Type	Description
format	String	It is one of the predefined formats: Default, Excel, MySQL, RFC4180,TDF. Go through following link for more details. https://commons.apache.org/proper/commons-csv/archives/1.2/apidocs/org/apache/commons/csv/CSVFormat.Predefined.html
delimiter	Character	Sets the delimiter of the format to the specified character.
escape	Character	Sets the escape character of the format to the specified character.
quote	Character	Sets the quoteChar of the format to the specified character.
quoteMode	String	Sets the output quote policy of the format to the specified value. One of: ALL, MINIMAL, NON_NUMERIC, NONE.
recordSeparator	String	Sets the record separator of the format to the specified String.
charset	Charset	The output Charset.
header	String	The header to include when the stream is opened.
footer	String	The footer to add when the stream is closed.

Prevoius                                                 Next                                                 Home