Data generator scripts contain scripts to push JSON files containing metrics data to be stored in the InfluxDB database. These mainly push data for following TFA (trusted-firmware-a) metrics:
For code churn, code complexity and defects metrics, data is generated by data generator scripts. For other metrics, the data generator scripts take as input a text file, that converts the information in the given file to JSON file that is posted to broker component. The text file is expected to be generated by TFA CI setup.
Please refer broker component user guide for changes essential for the setup. Once broker component is up and running, please make following changes to run data generator scipt.
As mentioned in broker component user guide, code churn, complexity and defects data can be generated as follows for a given release tag:
$ cd qa-tools/quality-metrics/data-generator/tfa_metrics $ ./tfa_quality_metrics.sh --tag [Release Tag]
An example of "Release Tag" can be any of the trusted-firmware-a repo's tags
JSON files which are sent to broker component can be found in a timestamped folder in tfa_metrics folder. Data written to InfluxDB can be queried using following commands:
$ curl -G 'http://[HOST Public IP]:8086/query?db=TFA_CodeChurn' --data-urlencode 'q=show measurements' $ curl -G 'http://[HOST Public IP]:8086/query?db=TFA_CodeChurn' --data-urlencode 'q=select * from TFA_CodeChurn_Tracking' $ curl -G 'http://[HOST Public IP]:8086/query?db=TFA_Complexity' --data-urlencode 'q=show measurements' $ curl -G 'http://[HOST Public IP]:8086/query?db=TFA_Complexity' --data-urlencode 'q=select * from TFA_Complexity_Tracking' $ curl -G 'http://[HOST Public IP]:8086/query?db=TFA_Complexity' --data-urlencode 'q=select * from TFA_Complexity_Statistics' $ curl -G 'http://[HOST Public IP]:8086/query?db=TFA_Defects' --data-urlencode 'q=show measurements' $ curl -G 'http://[HOST Public IP]:8086/query?db=TFA_Defects' --data-urlencode 'q=select * from TFA_Defects_Tracking' $ curl -G 'http://[HOST Public IP]:8086/query?db=TFA_Defects' --data-urlencode 'q=select * from TFA_Defects_Statistics'
Using query "show measurements", all the measurements for corresponding DB are listed. After this, one can issue command "select * from measurement", where measurement can be any measurement from "show measurements" output.
$ cd qa-tools/quality-metrics/data-generator/tfa_metrics $ ./tfa_quality_metrics.sh --metric_type image_size --image_size_file [Image Size Text File] $ curl -G 'http://[HOST Public IP]:8086/query?db=TFA_ImageSize' --data-urlencode 'q=show measurements'
"show measurements" would list the measurements available for TFA_ImageSize DB. To view the data image size measurements, issue command "select * from measurement", where measurement is the name of measurement for which data needs to be view data.
BuildConfig: zynqmp-tspd-tdram BinMode: debug CommitID: ed39d5e3c0709bab22821a1da3a62737c5d531de CommitTitle: Merge "Enabling DPU in dts file for TC0" into integration CommitDate: 2020-09-08T14:22:45+00:00 /work/workspace/workspace/swqt-atf-image-size-metrics/scripts/metrics/atf_metrics/tf-topics/tools/renesas/rcar_layout_create/cert_header_sa6.elf: R 288 /work/workspace/workspace/swqt-atf-image-size-metrics/scripts/metrics/atf_metrics/tf-topics/tools/renesas/rcar_layout_create/bootparam_sa0.elf: R 20 /work/workspace/workspace/swqt-atf-image-size-metrics/scripts/metrics/atf_metrics/tf-topics/build/zynqmp/debug/bl31/bl31.elf: B 26676 D 14193 R 3121 T 40916 W 516 /work/workspace/workspace/swqt-atf-image-size-metrics/scripts/metrics/atf_metrics/tf-topics/build/zynqmp/debug/bl32/bl32.elf: B 21512 D 5 R 1218 T 11496 W 196
$ cd qa-tools/quality-metrics/data-generator/tfa_metrics $ ./tfa_quality_metrics.sh --metric_type coverity_misra --misra_defects_file [MISRA Defects Text File] $ curl -G 'http://[HOST Public IP]:8086/query?db=TFA_MisraDefects' --data-urlencode 'q=show measurements'
"show measurements" would list the measurements available for TFA_MisraDefects DB. To view the data MISRA defects measurements, issue command "select * from measurement", where measurement is the name of measurement for which data needs to be view data.
BuildConfig: fvp-rst-bl31 BinMode: debug CommitID: ed39d5e3c0709bab22821a1da3a62737c5d531de CommitTitle: Merge "Enabling DPU in dts file for TC0" into integration CommitDate: 2020-09-08T14:22:45+00:00 TotalDefects: 667 MandatoryDefects: 0 RequiredDefects: 534 AdvisoryDefects: 119
$ cd qa-tools/quality-metrics/data-generator/tfa_metrics $ ./tfa_quality_metrics.sh --metric_type runtime_instrumentation --rt_instr_file [RTINSTR Text File] $ curl -G 'http://[HOST Public IP]:8086/query?db=TFA_RTINSTR' --data-urlencode 'q=show measurements'
"show measurements" would list the measurements available for TFA_RTINSTR DB. To view the data runtime instrumentation measurements, issue command "select * from measurement", where measurement is the name of measurement for which data needs to be view data.
InstrumentationTarget: juno-tftf+aarch32-rt32.instr-r2 CommitID: ed39d5e3c0709bab22821a1da3a62737c5d531de CommitTitle: Merge "Enabling DPU in dts file for TC0" into integration CommitDate: 2020-09-08T14:22:45+00:00 testrtinstrsuspdeepparallel 1 0 128700 31280 9200 testrtinstrsuspdeepparallel 1 0 128700 31280 9200 testrtinstrsuspdeepparallel 1 1 283520 27260 9120 testrtinstrsuspdeepparallel 1 1 283520 27260 9120 testrtinstrsuspdeepparallel 1 2 541120 26860 214200 testrtinstrsuspdeepparallel 1 2 541120 26860 214200 testrtinstrsuspdeepparallel 1 3 206300 25180 9100 testrtinstrsuspdeepparallel 1 3 206300 25180 9100 testrtinstrsuspdeepparallel 0 0 51060 76960 6760 testrtinstrsuspdeepparallel 0 0 51060 76960 6760 testrtinstrsuspdeepparallel 0 1 361040 25560 203720 testrtinstrsuspdeepparallel 0 1 361040 25560 203720 testrtinstrsuspdeepserial 1 0 259740 28140 214060 testrtinstrsuspdeepserial 1 0 259740 28140 214060 testrtinstrsuspdeepserial 1 1 54480 23060 8920 testrtinstrsuspdeepserial 1 1 54480 23060 8920 testrtinstrsuspdeepserial 1 2 53860 23200 8920 testrtinstrsuspdeepserial 1 2 53860 23200 8920 testrtinstrsuspdeepserial 1 3 54280 23340 9120 testrtinstrsuspdeepserial 1 3 54280 23340 9120 testrtinstrsuspdeepserial 0 0 249020 25600 204180 testrtinstrsuspdeepserial 0 0 249020 25600 204180 testrtinstrsuspdeepserial 0 1 248800 25660 203940 testrtinstrsuspdeepserial 0 1 248800 25660 203940 testrtinstrcpususpparallel 1 0 209820 23500 10700 testrtinstrcpususpparallel 1 0 209820 23500 10700 testrtinstrcpususpparallel 1 1 287420 21920 11600 testrtinstrcpususpparallel 1 1 287420 21920 11600 testrtinstrcpususpparallel 1 2 364080 20980 11160 testrtinstrcpususpparallel 1 2 364080 20980 11160 testrtinstrcpususpparallel 1 3 441280 20720 10980 testrtinstrcpususpparallel 1 3 441280 20720 10980 testrtinstrcpususpparallel 0 0 52960 18920 9780 testrtinstrcpususpparallel 0 0 52960 18920 9780 testrtinstrcpususpparallel 0 1 130940 19020 9800 testrtinstrcpususpparallel 0 1 130940 19020 9800 testrtinstrcpususpserial 1 0 82460 20500 8960 testrtinstrcpususpserial 1 0 82460 20500 8960 testrtinstrcpususpserial 1 1 51320 20400 9040 testrtinstrcpususpserial 1 1 51320 20400 9040 testrtinstrcpususpserial 1 2 51180 19860 8980 testrtinstrcpususpserial 1 2 51180 19860 8980 testrtinstrcpususpserial 1 3 51220 20160 8940 testrtinstrcpususpserial 1 3 51220 20160 8940 testrtinstrcpususpserial 0 0 48560 19260 6840 testrtinstrcpususpserial 0 0 48560 19260 6840 testrtinstrcpususpserial 0 1 48340 19160 6760 testrtinstrcpususpserial 0 1 48340 19160 6760 testrtinstrcpuoffserial 1 0 260060 28060 214080 testrtinstrcpuoffserial 1 0 260060 28060 214080 testrtinstrcpuoffserial 1 1 55780 28100 9140 testrtinstrcpuoffserial 1 1 55780 28100 9140 testrtinstrcpuoffserial 1 2 55620 26860 9180 testrtinstrcpuoffserial 1 2 55620 26860 9180 testrtinstrcpuoffserial 1 3 55760 26440 9260 testrtinstrcpuoffserial 1 3 55760 26440 9260 testrtinstrcpuoffserial 0 0 251900 28880 204240 testrtinstrcpuoffserial 0 0 251900 28880 204240 testrtinstrcpuoffserial 0 1 252440 29560 204460 testrtinstrcpuoffserial 0 1 252440 29560 204460 testrtinstrpsciversionparallel 1 0 880 testrtinstrpsciversionparallel 1 0 880 testrtinstrpsciversionparallel 1 1 960 testrtinstrpsciversionparallel 1 1 960 testrtinstrpsciversionparallel 1 2 980 testrtinstrpsciversionparallel 1 2 980 testrtinstrpsciversionparallel 1 3 980 testrtinstrpsciversionparallel 1 3 980 testrtinstrpsciversionparallel 0 0 1040 testrtinstrpsciversionparallel 0 0 1040 testrtinstrpsciversionparallel 0 1 1180 testrtinstrpsciversionparallel 0 1 1180