There is a newer version of the record available.

Published April 27, 2023 | Version v1
Conference paper Open

Replication Package for : AML: A novel approach towards critical analysis of log parsing strategies

Authors/Creators

Description

This replication package has been prepared to support further investigation and validation of our study on AML metric to measure the accuracy of log parsing techniques. It contains detailed information about log parsing results and the the calculation results for each log parser evaluated in our research. The package is organized into separate zipped folders.

The package is organized as follows:

  1. Parsing results: There are separate folders for each log parsing tool, such as ULP, Drain, etc. Inside each folder, you will find individual CSV files containing the parsing results for each dataset analyzed in our study. These files are named as "Android_2k.log_structured.csv", "Apache_2k.log_structured.csv", and so on. Each row represents a single log entry, with the columns providing various details about the log entry (header and message). The column EventTemplate represents the parsed log event template.

  2. AML calculation : There are separate folders for each log parser. Inside each folder, you will find individual CSV files containing the parsing results for each dataset analyzed in our study. These files are named as "AML_<dataset>_final.csv, where <dataset> is the name of the Dataset parsed. each file contains four columns: Template, Omission Error, Commission Error, and ICSI. Each row represents a specific log template, along with the associated score for evaluating log parsing performance on the template level. at the end of the file, the overall AML score if transcribed. (Note that: Iplom timed out for Android)

Overall, here are the AML results for all the datasets and log parsers:

System HDFS Apache Android Windows Mac OpenStack Linux OpenSSH Thunderbird Spark Hadoop Zookeeper BGL HealthApp HPC Proxifier Average
ULP 0.5681 0.3333 0.4633 0.3866 0.4435 0.2800 -0.2802 0.0690 0.4103 0.3077 0.1684 0.6508 0.2542 0.7500 0.4774 0.4330 0.3572
Drain 0.7055 0.3333 0.6605 0.3919 0.1709 0.0088 -0.2478 0.2439 0.1871 0.2500 0.1361 0.3913 0.1443 0.0871 0.3433 0.3730 0.2612
SHISO 0.7500 0.3333 0.4313 0.2790 0.1455 0.1111 0.2274 0.1735 0.1718 0.2241 0.1194 0.2333 0.0947 0.1681 0.0917 0.4612 0.2510
AEL 0.6663 0.3333 0.4976 0.3151 0.1356 0.1868 -0.1948 0.1731 0.1605 0.1605 0.1187 0.3205 0.1451 0.0694 0.4091 0.4244 0.2451
Iplom 0.7100 0.3300 -- 0.2700 0.1400 0.2100 -0.1900 0.1600 0.1800 0.2500 0.1400 0.4000 0.1300 0.1400 0.3400 0.5532 0.2352
Spell 0.7500 0.3333 0.7294 0.3312 0.1264 0.0123 -0.1964 0.2000 0.1399 0.1605 0.0977 0.3049 0.0527 0.1034 0.3905 0.0585 0.2246
Lenma 0.7055 0.3333 0.6508 0.2976 0.1246 0.1155 -0.2550 0.1860 0.1999 0.1074 0.1088 0.3292 0.0613 0.0282 0.1117 0.0910 0.1997
LogMine 0.6875 0.3333 0.3083 0.3247 0.1671 0.0097 0.2473 0.0107 0.1631 0.0741 0.1186 0.0727 0.0266 0.0630 0.0528 0.0222 0.1676
LKE 0.7500 0.3333 0.7563 0.2877 0.0000 0.0242 -0.0767 0.0239 0.0640 0.0000 0.0506 0.1923 0.0200 0.1709 0.0000 0.0998 0.1685
SLCT 0.2166 0.1429 0.7000 0.0506 0.0870 0.2679 0.0310 0.1260 0.0599 0.1167 0.0079 0.1467 0.0661 0.0361 0.2702 0.1654 0.1557
Logsig 0.2760 0.1820 0.3450 0.1590 0.0720 0.2470 0.0400 0.1700 0.0240 0.0620 0.0200 0.1820 0.0700 0.0490 0.0810 0.3447 0.1452
Molfi 0.0000 0.0000 0.1703 0.1770 0.0676 0.0018 0.2275 0.0612 0.1368 0.0068 0.0824 0.1585 0.0511 0.2477 0.1634 0.0000 0.0970
Logram 0.0141 0.0009 0.3290 0.0806 0.1133 0.0000 0.0303 0.1172 0.0533 0.0172 0.0303 0.2240 0.0458 0.0288 0.2871 0.0000 0.0857
LogCluster 0.0056 0.0000 0.1786 0.2414 0.0846 0.0023 -0.1906 0.0282 0.1256 0.0267 0.0619 0.1545 0.0309 0.0414 0.1170 0.0000 0.0568

Files

Parsing_results.zip

Files (10.0 MB)

Name Size Download all
md5:6ece7a0c0458be1fdc21c3c6e59cbd8a
10.0 MB Preview Download