Dataprocessor Log Files

This article provides information on dataprocessor log files and their management via the Logging API, which includes:

Logging Levels

Log levels severity order: ERROR > WARN > INFO > DEBUG > TRACE

Informational events with the set log level and events with higher severity log levels are logged for packages; e.g. if a package has level INFO set, INFO, WARN and ERROR events will be logged for that package.

Logging Levels
ERROR Error events that prevent some functionalities from working, but might still allow the application to continue running.
WARN Unexpected and potentially harmful behavior of the application.
INFO Purely informative messages on events at coarse-grained level of the application.
DEBUG Fine-grained events useful for application debugging.
TRACE Finer-grained events than on the DEBUG level.

Dataprocessor Logging API

To find the <user> / <password> values:

  1. Access the dataprocessor container
  2. Open the application.yml file: cat /app/conf/application.yml
  3. Find the <user> / <password> values under  data-processor > security > http block

Commands that can be executed from the dataprocessor container:

  • Download log files:  curl -X GET <host>:8080/logs --output <file path>.zip <user>:<password>
  • Get the current log level of a particular package:  curl -X GET <host>:8080/logs/level?pkg=<package> -u <user>:<password>
  • Get a full list of packages with logging levels: curl -X GET hostname.com:8080/logs/level
  • Change the log level of a particular package:  curl -X POST <host>:8080/logs/level -d "level=<logLevel>&pkg=<package>" -u <user>:<password>
    • If package is not specified, log levels of all packages are set to the  <logLevel>

Examples

Download log files to the logs.zip file in the current directory:

root@dataprocessor:/app/conf#  curl -X GET hostname.com:8080/logs --output logs.zip -u dataprocessor:JKQmXXXXXXXXX
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 1596k    0 1596k    0     0  2186k      0 --:--:-- --:--:-- --:--:-- 2186k

Get the log level of the org.apache.http package: 

root@dataprocessor:/app/conf# curl -X GET hostname.com:8080/logs/level?pkg=org.apache.http -u dataprocessor:JKQmXXXXXXXXX
Log level for package [org.apache.http] is [INFO].

Change the log level of the com.metricinsights.collector.plugin.qliksense package:

root@dataprocessor:/app/conf# curl -X POST hostname.com:8080/logs/level -d "level=WARN&pkg=com.metricinsights.collector.plugin.qliksense" -u dataprocessor:JKQmXXXXXXXXX
Log level for package [com.metricinsights.collector.plugin.qliksense] changed from [INFO] to [WARN].

Packages and Default Logging Levels

Package Default Logging Level
akka INFO
com.metricinsights INFO
com.metricinsights.akka.actors.workers.MetadataWorker INFO
com.metricinsights.collector DEBUG
com.metricinsights.collector.plugin.qliksense INFO
com.metricinsights.monitoring INFO
com.zaxxer.hikari INFO
org.apache.catalina.startup.DigesterFactory ERROR
org.apache.coyote.http11.Http11NioProtocol WARN
org.apache.http INFO
org.apache.catalina.util.LifecycleBase ERROR
org.apache.sshd.common.util.SecurityUtils WARN
org.apache.tomcat.util.net.NioSelectorPool WARN
org.crsh.plugin WARN
org.crsh.ssh WARN
org.eclipse.jetty.util.component.AbstractLifeCycle ERROR
org.eclipse.jetty.websocket INFO
org.hibernate.SQL INFO
org.hibernate.validator.internal.util.Version WARN
org.springframework.boot INFO
org.springframework.boot.actuate.autoconfigure.CrshAutoConfiguration WARN
org.springframework.core INFO
org.springframework.orm INFO
org.springframework.web INFO
org.springframework.web.filter.CommonsRequestLoggingFilter DEBUG

Log Files

Log File Description
/app/logs/container/tomcat-access.log REST API log file. Logs all requests and responses to and from the dataprocessor
/app/logs/application.log Dataprocessor log file, that includes the requests sent from the UI to RDP and plugins
insight.log Information on plugins' work
error.log Contains all error logs
/app/logs/container/hikari.log Connecting pool to the database