logreduce 0.4.0

Local file usage

• Compare two files or directories without building a model:

$ logreduce diff testr-nodepool-01/output.good testr-nodepool-01/output.fail
0.232 | testr-nodepool-01/output.fail:0677:  File "voluptuous/schema_builder.py", line 370, in validate_mapping
0.462 | testr-nodepool-01/output.fail:0678:    raise er.MultipleInvalid(errors)
0.650 | testr-nodepool-01/output.fail:0679:  voluptuous.error.MultipleInvalid: required key not provided @ data['providers'][2]['cloud']

• Compare two files or directories:

$ logreduce dir preprod-logs/ /var/log/

• Or build a model first and run it separately:

$ logreduce dir-train sosreport.clf old-sosreport/ good-sosreport/
$ logreduce dir-run sosreport.clf new-sosreport/

Zuul job usage

Logreduce can query Zuul build database to train a model.

• Extract novelty from a job logs:

$ logreduce job http://logs.openstack.org/...

# Reduce comparaison to a single project (e.g. for tox jobs)
$ logreduce job --project openstack/nova http://logs.openstack.org/...

# Compare using many baselines
$ logreduce job --count 10 http://logs.openstack.org/...

# Include job artifacts
$ logreduce job --include-path logs/ http:/logs.openstack.org/...

• Or build a model first and run it separately:

$ logreduce job-train --job job_name job_name.clf
$ logreduce job-run job_name.clf http://logs.openstack.org/.../

Journald usage

Logreduce can look for anomaly in journald, comparing the last day/week/month to the previous one:

• Extract novelty from last day journal:

$ logreduce journal --range day

• Build a model using journal of last month and look for novelty in last week:

$ logreduce journal-train --range month good-journal.clf
$ logreduce journal-run --range week good-journal.clf

API

• PUT /anomaly/new: receive user report request from os_loganalyze

• PUT /anomaly: import an anomaly report (json file generated by standalone cli)

• GET /anomaly/{anomaly_id}: return an anomaly details

• POST /anomaly/{anomaly_id}: update anomaly status

• POST /anomaly/{anomaly_id}/logfile/{logfile_id}: update scores

• DELETE /anomaly/{anomaly_id}/logfile/{logfile_id}: remove a file

• GET /anomalies: return the list of anomalies

• GET /status: return the list of worker jobs

Test format

Each tests case is composed of:

• A .good file (or directory) that holds the baseline

• A .fail file (or directory)

• A info.yaml file that describe expected output:

threshold: float # set the distance threshold for the test
anomalies:
  - optional: bool  # to define minor anomalies not considered false positive
    lines: |        # the expected lines to be highlighted
      Traceback...
      RuntimeError...

Evaluate

To run the evaluation, first install logreduce-tests:

git clone https://softwarefactory-project.io/r/logreduce-tests
pushd logreduce-tests
python3 setup.py develop --user

logreduce-tests expect tests directories as argument:

$ logreduce-tests tests/testr-zuul-[0-9]*
[testr-zuul-01]: 100.00% accuracy,  5.00% false-positive
[testr-zuul-02]:  80.00% accuracy,  0.00% false-positive
...
Summary:  90.00% accuracy,  2.50% false-positive

Add –debug to display false positive and missing chunks.