Initial commit.
- qa-tools public release which includes:
- trace-based coverage tool
- quality metrics measurement and tracking setup
- associated in-source documentation.
Signed-off-by: Basil Eljuse <basil.eljuse@arm.com>
diff --git a/quality-metrics/docs/broker_component_user_guide.md b/quality-metrics/docs/broker_component_user_guide.md
new file mode 100644
index 0000000..58ddbdf
--- /dev/null
+++ b/quality-metrics/docs/broker_component_user_guide.md
@@ -0,0 +1,84 @@
+# Broker Component User Guide
+Broker component is Python Flask app, which handles the data being pushed by the data generator components of the quality metrics project. It implements APIs that allow the data generator scripts to POST metrics data in an agreed JSON format which gets pushed to InfluxDB backend database. For each received request with valid authorization token, it performs basic sanity check and pushes the data to InfluxDB. For details on how to visualize InfluxDB data using Grafana, please refer [visualisation user guide](./visualisation_user_guide.md).
+
+## Deploying broker component
+The broker component can be deployed in the infrastructure as a docker container with the docker files in the project.
+
+### Dependencies
+- docker
+- docker-compose
+- python3-dev
+- python3-pip
+- Ensure that ports 3000, 5000 and 8086 are enabled for incoming traffic.
+
+### Bringing-up Broker Component - Setup Instructions
+In order to bring-up broker component (for a first time setup), docker-compose.yml is used, which brings up 3 containers - Grafana, InfluxDB and Flask App. It is upto the user to adapt the docker_compose.yml file if only some of the components needs to be deployed.
+JWT Authorization Token is generated using following steps, which needs to be added in [tfa_variables.sh](../data_generator/tfa_variables.sh)
+1. Set value of HOST to host public IP address in file [constants.py](../broker-component/constants.py)
+1. [credentials.py](../broker-component/credentials.py) is provided for reference purpose only. Please change the value of SECRET_KEY and password for _tfa_metrics_ user.
+1. Ensure the current working directory is broker_component. Create [teamname]_credentials.json (as an example let us use 'tfa' as the teamname) and run following commands to generate Authorization Token:
+```bash
+$ cat tfa_credentials.json
+{
+ "username": "tfa_metrics",
+ "password": "[tfa_metrics password matching with password in credentials.py]"
+}
+
+$ docker network create metrics_network
+$ docker volume create influxdb-volume
+$ docker volume create grafana-volume
+$ docker-compose build
+$ docker-compose up -d
+$ curl -H "Content-Type: application/json" -X POST -d "$(cat tfa_credentials.json)" http://[Host Public IP]:5000/auth
+```
+
+## Testing the setup after deployment
+The following steps can help confirm if the deployment steps detailed in previous setup was indeed successful or not.
+
+### Verify that Grafana is up
+If URL *http://`[HOST Public IP]`:3000* is accessible, it confirms that Grafana is up.
+
+### Verify that InfluxDB is up
+If URL *http://`[HOST Public IP]`:8086/query* is accessible, it confirms that InfluxDB is up.
+
+### Create InfluxDB Database
+Database can be created by accessing InfluxDB container or by using InfluxDB API.
+
+#### Create database by accessing InfluxDB container
+```bash
+$ docker exec -it influxdb_container sh
+# influx
+> create database TFA_CodeChurn
+> create database TFA_Complexity
+> create database TFA_Defects
+> create database TFA_MisraDefects
+> create database TFA_ImageSize
+> create database TFA_RTINSTR
+> exit
+# exit
+```
+#### Create database using the InfluxDB API
+```bash
+$ curl -i -XPOST http://[HOST Public IP]:8086/query --data-urlencode "q=CREATE DATABASE TFA_CodeChurn"
+$ curl -i -XPOST http://[HOST Public IP]:8086/query --data-urlencode "q=CREATE DATABASE TFA_Complexity"
+$ curl -i -XPOST http://[HOST Public IP]:8086/query --data-urlencode "q=CREATE DATABASE TFA_Defects"
+$ curl -i -XPOST http://[HOST Public IP]:8086/query --data-urlencode "q=CREATE DATABASE TFA_MisraDefects"
+$ curl -i -XPOST http://[HOST Public IP]:8086/query --data-urlencode "q=CREATE DATABASE TFA_ImageSize"
+$ curl -i -XPOST http://[HOST Public IP]:8086/query --data-urlencode "q=CREATE DATABASE TFA_RTINSTR"
+$ curl -i -XPOST http://[HOST Public IP]:8086/query --data-urlencode "q=SHOW DATABASES"
+```
+
+### Pushing Data to InfluxDB
+Data can be pushed to InfluxDB by sending cURL POST request in the agreed-upon format and with correct authorization token.
+* The steps above mention that how authorization token can be generated.
+* Request is validated using [JSON schemas](../broker-component/metrics-schemas)
+In order to send push data, run following commands:
+```bash
+$ cd qa-tools/quality-metrics/data-generator/tfa_metrics
+$ ./tfa_quality_metrics.sh --tag [Release Tag]
+```
+
+For details, please refer [data generator user guide](./data_generator_user_guide.md).
+
+## License
+[BSD-3-Clause](../../license.md)
diff --git a/quality-metrics/docs/data_generator_user_guide.md b/quality-metrics/docs/data_generator_user_guide.md
new file mode 100644
index 0000000..fc81b54
--- /dev/null
+++ b/quality-metrics/docs/data_generator_user_guide.md
@@ -0,0 +1,194 @@
+# Data Generator User Guide
+Data generator scripts contain scripts to push JSON files containing metrics data to be stored in the InfluxDB database. These mainly push data for following TFA (trusted-firmware-a) metrics:
+1. Code churn, Code complexity and GitHub defects
+1. Image Size
+1. MISRA defects
+1. Runtime instrumentation
+
+For code churn, code complexity and defects metrics, data is generated by data generator scripts. For other metrics, the data generator scripts take as input a text file, that converts the information in the given file to JSON file that is posted to broker component. The text file is expected to be generated by TFA CI setup.
+
+## Dependencies
+- python3-dev
+- python3-pip
+
+## Generating Data
+Please refer [broker component user guide](./broker_component_user_guide.md) for changes essential for the setup. Once broker component is up and running, please make following changes to run data generator scipt.
+1. Please refer [broker component user guide](./broker_component_user_guide.md) for details on how to generate token, and add it in [tfa_variables.sh](../data-generator/tfa_metrics/tfa_variables.sh). "Bringing-up Broker Component - Setup Instructions" contains details on how to generate authentication token.
+1. Set the value of INFLUX_HOST to Influx Public Host IP in [tfa_variables.sh](../data-generator/tfa_metrics/tfa_variables.sh).
+1. Set the value of GitHub access token in [tfa_defects.py](../data-generator/tfa_metrics/tfa_defects.py)
+
+### Code churn, Code complexity and GitHub defects
+As mentioned in [broker component user guide](./broker_component_user_guide.md), code churn, complexity and defects data can be generated as follows for a given release tag:
+```bash
+$ cd qa-tools/quality-metrics/data-generator/tfa_metrics
+$ ./tfa_quality_metrics.sh --tag [Release Tag]
+```
+
+An example of "Release Tag" can be any of the [trusted-firmware-a repo's tags](https://git.trustedfirmware.org/TF-A/trusted-firmware-a.git/refs/tags)
+
+JSON files which are sent to broker component can be found in a timestamped folder in tfa_metrics folder. Data written to InfluxDB can be queried using following commands:
+```bash
+$ curl -G 'http://[HOST Public IP]:8086/query?db=TFA_CodeChurn' --data-urlencode 'q=show measurements'
+$ curl -G 'http://[HOST Public IP]:8086/query?db=TFA_CodeChurn' --data-urlencode 'q=select * from TFA_CodeChurn_Tracking'
+
+$ curl -G 'http://[HOST Public IP]:8086/query?db=TFA_Complexity' --data-urlencode 'q=show measurements'
+$ curl -G 'http://[HOST Public IP]:8086/query?db=TFA_Complexity' --data-urlencode 'q=select * from TFA_Complexity_Tracking'
+$ curl -G 'http://[HOST Public IP]:8086/query?db=TFA_Complexity' --data-urlencode 'q=select * from TFA_Complexity_Statistics'
+
+$ curl -G 'http://[HOST Public IP]:8086/query?db=TFA_Defects' --data-urlencode 'q=show measurements'
+$ curl -G 'http://[HOST Public IP]:8086/query?db=TFA_Defects' --data-urlencode 'q=select * from TFA_Defects_Tracking'
+$ curl -G 'http://[HOST Public IP]:8086/query?db=TFA_Defects' --data-urlencode 'q=select * from TFA_Defects_Statistics'
+```
+
+Using query "show measurements", all the measurements for corresponding DB are listed. After this, one can issue command "select * from measurement", where measurement can be any measurement from "show measurements" output.
+
+### Image Size
+```bash
+$ cd qa-tools/quality-metrics/data-generator/tfa_metrics
+$ ./tfa_quality_metrics.sh --metric_type image_size --image_size_file [Image Size Text File]
+$ curl -G 'http://[HOST Public IP]:8086/query?db=TFA_ImageSize' --data-urlencode 'q=show measurements'
+```
+
+"show measurements" would list the measurements available for TFA_ImageSize DB. To view the data image size measurements, issue command "select * from measurement", where measurement is the name of measurement for which data needs to be view data.
+
+#### Image Size Sample File
+```
+BuildConfig: zynqmp-tspd-tdram
+BinMode: debug
+CommitID: ed39d5e3c0709bab22821a1da3a62737c5d531de
+CommitTitle: Merge "Enabling DPU in dts file for TC0" into integration
+CommitDate: 2020-09-08T14:22:45+00:00
+/work/workspace/workspace/swqt-atf-image-size-metrics/scripts/metrics/atf_metrics/tf-topics/tools/renesas/rcar_layout_create/cert_header_sa6.elf:
+R 288
+
+/work/workspace/workspace/swqt-atf-image-size-metrics/scripts/metrics/atf_metrics/tf-topics/tools/renesas/rcar_layout_create/bootparam_sa0.elf:
+R 20
+
+/work/workspace/workspace/swqt-atf-image-size-metrics/scripts/metrics/atf_metrics/tf-topics/build/zynqmp/debug/bl31/bl31.elf:
+B 26676
+D 14193
+R 3121
+T 40916
+W 516
+
+/work/workspace/workspace/swqt-atf-image-size-metrics/scripts/metrics/atf_metrics/tf-topics/build/zynqmp/debug/bl32/bl32.elf:
+B 21512
+D 5
+R 1218
+T 11496
+W 196
+```
+
+### MISRA Defects
+```bash
+$ cd qa-tools/quality-metrics/data-generator/tfa_metrics
+$ ./tfa_quality_metrics.sh --metric_type coverity_misra --misra_defects_file [MISRA Defects Text File]
+$ curl -G 'http://[HOST Public IP]:8086/query?db=TFA_MisraDefects' --data-urlencode 'q=show measurements'
+```
+
+"show measurements" would list the measurements available for TFA_MisraDefects DB. To view the data MISRA defects measurements, issue command "select * from measurement", where measurement is the name of measurement for which data needs to be view data.
+
+#### MISRA Defects Sample File
+```
+BuildConfig: fvp-rst-bl31
+BinMode: debug
+CommitID: ed39d5e3c0709bab22821a1da3a62737c5d531de
+CommitTitle: Merge "Enabling DPU in dts file for TC0" into integration
+CommitDate: 2020-09-08T14:22:45+00:00
+TotalDefects: 667
+MandatoryDefects: 0
+RequiredDefects: 534
+AdvisoryDefects: 119
+```
+
+### Runtime Instrumentation
+```bash
+$ cd qa-tools/quality-metrics/data-generator/tfa_metrics
+$ ./tfa_quality_metrics.sh --metric_type runtime_instrumentation --rt_instr_file [RTINSTR Text File]
+$ curl -G 'http://[HOST Public IP]:8086/query?db=TFA_RTINSTR' --data-urlencode 'q=show measurements'
+```
+
+"show measurements" would list the measurements available for TFA_RTINSTR DB. To view the data runtime instrumentation measurements, issue command "select * from measurement", where measurement is the name of measurement for which data needs to be view data.
+
+#### Runtime Instrumentation Sample File
+```
+InstrumentationTarget: juno-tftf+aarch32-rt32.instr-r2
+CommitID: ed39d5e3c0709bab22821a1da3a62737c5d531de
+CommitTitle: Merge "Enabling DPU in dts file for TC0" into integration
+CommitDate: 2020-09-08T14:22:45+00:00
+testrtinstrsuspdeepparallel 1 0 128700 31280 9200
+testrtinstrsuspdeepparallel 1 0 128700 31280 9200
+testrtinstrsuspdeepparallel 1 1 283520 27260 9120
+testrtinstrsuspdeepparallel 1 1 283520 27260 9120
+testrtinstrsuspdeepparallel 1 2 541120 26860 214200
+testrtinstrsuspdeepparallel 1 2 541120 26860 214200
+testrtinstrsuspdeepparallel 1 3 206300 25180 9100
+testrtinstrsuspdeepparallel 1 3 206300 25180 9100
+testrtinstrsuspdeepparallel 0 0 51060 76960 6760
+testrtinstrsuspdeepparallel 0 0 51060 76960 6760
+testrtinstrsuspdeepparallel 0 1 361040 25560 203720
+testrtinstrsuspdeepparallel 0 1 361040 25560 203720
+testrtinstrsuspdeepserial 1 0 259740 28140 214060
+testrtinstrsuspdeepserial 1 0 259740 28140 214060
+testrtinstrsuspdeepserial 1 1 54480 23060 8920
+testrtinstrsuspdeepserial 1 1 54480 23060 8920
+testrtinstrsuspdeepserial 1 2 53860 23200 8920
+testrtinstrsuspdeepserial 1 2 53860 23200 8920
+testrtinstrsuspdeepserial 1 3 54280 23340 9120
+testrtinstrsuspdeepserial 1 3 54280 23340 9120
+testrtinstrsuspdeepserial 0 0 249020 25600 204180
+testrtinstrsuspdeepserial 0 0 249020 25600 204180
+testrtinstrsuspdeepserial 0 1 248800 25660 203940
+testrtinstrsuspdeepserial 0 1 248800 25660 203940
+testrtinstrcpususpparallel 1 0 209820 23500 10700
+testrtinstrcpususpparallel 1 0 209820 23500 10700
+testrtinstrcpususpparallel 1 1 287420 21920 11600
+testrtinstrcpususpparallel 1 1 287420 21920 11600
+testrtinstrcpususpparallel 1 2 364080 20980 11160
+testrtinstrcpususpparallel 1 2 364080 20980 11160
+testrtinstrcpususpparallel 1 3 441280 20720 10980
+testrtinstrcpususpparallel 1 3 441280 20720 10980
+testrtinstrcpususpparallel 0 0 52960 18920 9780
+testrtinstrcpususpparallel 0 0 52960 18920 9780
+testrtinstrcpususpparallel 0 1 130940 19020 9800
+testrtinstrcpususpparallel 0 1 130940 19020 9800
+testrtinstrcpususpserial 1 0 82460 20500 8960
+testrtinstrcpususpserial 1 0 82460 20500 8960
+testrtinstrcpususpserial 1 1 51320 20400 9040
+testrtinstrcpususpserial 1 1 51320 20400 9040
+testrtinstrcpususpserial 1 2 51180 19860 8980
+testrtinstrcpususpserial 1 2 51180 19860 8980
+testrtinstrcpususpserial 1 3 51220 20160 8940
+testrtinstrcpususpserial 1 3 51220 20160 8940
+testrtinstrcpususpserial 0 0 48560 19260 6840
+testrtinstrcpususpserial 0 0 48560 19260 6840
+testrtinstrcpususpserial 0 1 48340 19160 6760
+testrtinstrcpususpserial 0 1 48340 19160 6760
+testrtinstrcpuoffserial 1 0 260060 28060 214080
+testrtinstrcpuoffserial 1 0 260060 28060 214080
+testrtinstrcpuoffserial 1 1 55780 28100 9140
+testrtinstrcpuoffserial 1 1 55780 28100 9140
+testrtinstrcpuoffserial 1 2 55620 26860 9180
+testrtinstrcpuoffserial 1 2 55620 26860 9180
+testrtinstrcpuoffserial 1 3 55760 26440 9260
+testrtinstrcpuoffserial 1 3 55760 26440 9260
+testrtinstrcpuoffserial 0 0 251900 28880 204240
+testrtinstrcpuoffserial 0 0 251900 28880 204240
+testrtinstrcpuoffserial 0 1 252440 29560 204460
+testrtinstrcpuoffserial 0 1 252440 29560 204460
+testrtinstrpsciversionparallel 1 0 880
+testrtinstrpsciversionparallel 1 0 880
+testrtinstrpsciversionparallel 1 1 960
+testrtinstrpsciversionparallel 1 1 960
+testrtinstrpsciversionparallel 1 2 980
+testrtinstrpsciversionparallel 1 2 980
+testrtinstrpsciversionparallel 1 3 980
+testrtinstrpsciversionparallel 1 3 980
+testrtinstrpsciversionparallel 0 0 1040
+testrtinstrpsciversionparallel 0 0 1040
+testrtinstrpsciversionparallel 0 1 1180
+testrtinstrpsciversionparallel 0 1 1180
+```
+
+## License
+[BSD-3-Clause](../../license.md)
diff --git a/quality-metrics/docs/design_overview.md b/quality-metrics/docs/design_overview.md
new file mode 100644
index 0000000..77523a6
--- /dev/null
+++ b/quality-metrics/docs/design_overview.md
@@ -0,0 +1,73 @@
+# Multi-tier architecture
+The quality-metrics setup is designed based on a multi-tier architecture where we have different components at each tier. Current design has components at 3 tiers:
+1. front-end components which include a number of data generator scripts that compute the metric to be tracked and visualised. There is Grafana visualisation component to visualise the quality metrics data.
+1. middleware components which include a broker component that acts as an abstraction layer on top of the backend-components. This broker component performs basic data sanity checks and any data translation required.
+1. back-end components which include the InfluxDB timeseries database as the data sink.
+
+## Front-end: Data Generator scripts and Grafana visualisation tool
+Data generator scripts are metric specific scripts that are expected to be integrated as part of the regular CI setup. Metrics are generated based on a pre-determined frequency with which it is executed in the team's CI setup. Each data generator script is expected to compute the agreed metric and push the metric data along with the associated metadata to the broker component as a JSON data fragment which then gets pushed to the backend for visualisation.
+
+The data generator scripts are organised on a per-team basis.
+
+Grafana tool is used to visualise the quality metrics data on a per-team basis.
+
+### Defect Metric
+This metric tracks the open defects against a given project.
+
+### Code Churn Metric
+This metric tracks the code churn, that is number of lines added, modified and deleted, against given tag for a project.
+
+### Code Complexity Metric
+This metric tracks the code complexity against a given tag for a project. It reports modified McCabe score for the code complexity where a switch-statement is treated as a single decision point, thus reporting a value of 2. It uses pmccabe utility for calculating complexity score, and list all the functions having "Modified McCabe Cyclomatic Complexity" above the agreed upon threshold are pushed to database.
+
+### Other Metrics
+While the above mentioned metrics are computed by data generator scripts, there are some other metrics like image size, MISRA defects and run time instrumentation, which are not computed by data generator scripts. The role of scripts present in data generator folder for these metrics is to convert the input text file containing data (to be written to DB) into JSON file (which is sent to broker component). These input text files are expected to be generated as part of CI setup.
+
+## Middleware: Broker component
+The broker component provides a level of abstraction and decouples the data generator components from the backend components. This decoupling allows ease of future changes if a new database or visualisation component is to be added, without major changes needed at the front-end scripts.
+
+The broker component provides a simple token-based authentication scheme to validate the data sources that pushes the metrics data to the quality metrics setup. A token is issued on a per-team basis in this setup. The broker component implements a service queue for the data requests received from clients. The broker component always does a sanity check of the data pushed by a client. Only well-formed data against agreed data template will be processed by the broker component. The broker component can perform agreed data transformation also on some data pushed to it before committing them to the backend data base.
+
+## Back-end: InfluxDB database
+The backend consists of the data sink component which holds the quality metrics data. An individual data model is defined for each metric. The below sections capture the details of the individual measurements and the InfluxDB line protocol outlining the data model. Separate database is created for each metric for a team and all measurements associated with a metric is held on this database.
+
+### TF-A Defect Data Model
+This database holds the measurements which contains data for the open defects against TFA components raised in GitHub.
+
+#### Measurement: TFA_Defects_Tracking
+This measurement captures the open defect count against TFA in GitHub and allows to visualize the trend over time.
+
+Line Protocol Description: TFA_Defects_Tracking,Measured_Date=[timestamp in date format] Issue_Status=[Open from GitHub Issue states],Number_of_Defects=[integer value for the defect count]
+
+#### Measurement: TFA_Defects_Statistics
+This measurement is a holding database for the raw data to feed into the TFA_Defects_Tracking.
+
+Line Protocol Description: TFA_Defects_Statistics,Defect_ID=[defect identifier],Measured_Date=[timestamp in date format] Title=[Defect title],Issue_Status=[Open|Closed|... from GitHub Issue states],URL=[URL to the issue]
+
+### TF-A Code Churn Data Model
+This database holds the measurements that is used to provide the visualization for the trend of LoC changes over time against the TFA (trusted-firmware-a) code base.
+
+#### Measurement: TFA_CodeChurn_Tracking
+This measurement captures the LoC add/delete/modified count against the TFA versions and allows to visualize the trend over time.
+
+Line Protocol Description: TFA_CodeChurn_Tracking,Git_Tag_Date=[Git Tag Date],Target_Tag=[TFA version tag],Base_Tag=[base tag] Lines_of_Change=[LoC changed]
+
+
+### TF-A Complexity Data Model
+This database holds the measurements that is used to provide the visualization for the trend of complex functions over time against the TFA code.
+
+#### Measurement: TFA_Complexity_Tracking
+This measurement captures the function count which are above a given threshold against the TFA code base and allows to visualize the trend over time.
+
+Line Protocol Description: TFA_Complexity_Tracking,Git_Tag_Date=[Git Tag Date],Target_Tag=[TFA version tag] Threshold=[threshold value],Whitelisted=[no],Functions_Exceeding_Threshold_Not_Whitelisted=[number of functions exceeding complexity threshold which are not whitelisted]
+
+#### Measurement: TFA_Complexity_Statistics
+This measurement is a holding database for the raw data to feed into the TFA_Complexity_Tracking.
+
+Line Protocol Description: TFA_Complexity_Statistics,Git_Tag_Date=[Git Tag Date],Base_Tag=[base tag],Target_Tag=[TFA version tag],Location=[path in the code base for the function] Function_ID=[function identifier],Score=[mccabe score],Threshold=[threshold value],Whitelisted=[yes|no]
+
+Most data models can also be interpreted from [JSON schemas](../broker-component/metrics-schemas). "data" section contains the details about fields and tags.
+
+## License
+[BSD-3-Clause](../../license.md)
+
diff --git a/quality-metrics/docs/sample-dashboards/tfa_codechurn.json b/quality-metrics/docs/sample-dashboards/tfa_codechurn.json
new file mode 100644
index 0000000..ac9e0b7
--- /dev/null
+++ b/quality-metrics/docs/sample-dashboards/tfa_codechurn.json
@@ -0,0 +1,288 @@
+{
+ "annotations": {
+ "list": [
+ {
+ "builtIn": 1,
+ "datasource": "-- Grafana --",
+ "enable": true,
+ "hide": true,
+ "iconColor": "rgba(0, 211, 255, 1)",
+ "name": "Annotations & Alerts",
+ "type": "dashboard"
+ }
+ ]
+ },
+ "editable": true,
+ "gnetId": null,
+ "graphTooltip": 0,
+ "id": 3,
+ "links": [],
+ "panels": [
+ {
+ "aliasColors": {},
+ "bars": true,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": "TFA_CodeChurn",
+ "decimals": 2,
+ "description": "Tracking the lines of changes in TF-A source code per release tag",
+ "fieldConfig": {
+ "defaults": {
+ "custom": {}
+ },
+ "overrides": []
+ },
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 7,
+ "w": 24,
+ "x": 0,
+ "y": 0
+ },
+ "hiddenSeries": false,
+ "id": 5,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": false,
+ "total": false,
+ "values": false
+ },
+ "lines": false,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "percentage": false,
+ "pluginVersion": "7.1.3",
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "alias": "$tag_Target_Tag",
+ "dsType": "influxdb",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT max(Lines_of_Change) FROM \"TFA_CodeChurn_Tracking\" where $timeFilter group by Target_Tag, Git_Tag_Date order by time",
+ "rawQuery": true,
+ "refId": "A",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "TFA CodeChurn Tracking",
+ "tooltip": {
+ "shared": false,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "series",
+ "name": null,
+ "show": true,
+ "values": [
+ "max"
+ ]
+ },
+ "yaxes": [
+ {
+ "decimals": 0,
+ "format": "short",
+ "label": "Lines of Change",
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": false
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "columns": [],
+ "datasource": "TFA_CodeChurn",
+ "fieldConfig": {
+ "defaults": {
+ "custom": {}
+ },
+ "overrides": []
+ },
+ "fontSize": "100%",
+ "gridPos": {
+ "h": 10,
+ "w": 24,
+ "x": 0,
+ "y": 7
+ },
+ "id": 7,
+ "links": [],
+ "pageSize": 9,
+ "scroll": true,
+ "showHeader": true,
+ "sort": {
+ "col": 0,
+ "desc": true
+ },
+ "styles": [
+ {
+ "alias": "Time",
+ "align": "auto",
+ "dateFormat": "YYYY-MM-DD HH:mm:ss",
+ "pattern": "Time",
+ "type": "hidden"
+ },
+ {
+ "alias": "",
+ "align": "auto",
+ "colorMode": null,
+ "colors": [
+ "rgba(245, 54, 54, 0.9)",
+ "rgba(237, 129, 40, 0.89)",
+ "rgba(50, 172, 45, 0.97)"
+ ],
+ "decimals": 2,
+ "pattern": "/.*/",
+ "thresholds": [],
+ "type": "number",
+ "unit": "short"
+ }
+ ],
+ "targets": [
+ {
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "measurement": "TFA_CodeChurn_Tracking",
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT Base_Tag as \"Base Tag\", max(Lines_of_Change) as \"Lines of Change\" FROM \"TFA_CodeChurn_Tracking\" GROUP BY Target_Tag ",
+ "rawQuery": true,
+ "refId": "A",
+ "resultFormat": "table",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ }
+ ],
+ "title": "TF-A Code Churn Statistics",
+ "transform": "table",
+ "type": "table-old"
+ }
+ ],
+ "refresh": "1d",
+ "schemaVersion": 26,
+ "style": "dark",
+ "tags": [
+ "TFA_QUALITY_METRICS"
+ ],
+ "templating": {
+ "list": []
+ },
+ "time": {
+ "from": "now-5y",
+ "to": "now"
+ },
+ "timepicker": {
+ "refresh_intervals": [
+ "5s",
+ "10s",
+ "30s",
+ "1m",
+ "5m",
+ "15m",
+ "30m",
+ "1h",
+ "2h",
+ "1d"
+ ],
+ "time_options": [
+ "5m",
+ "15m",
+ "1h",
+ "6h",
+ "12h",
+ "24h",
+ "2d",
+ "7d",
+ "30d"
+ ]
+ },
+ "timezone": "",
+ "title": "TFA_Churn",
+ "uid": "tfa-churn",
+ "version": 4
+}
\ No newline at end of file
diff --git a/quality-metrics/docs/sample-dashboards/tfa_complexity.json b/quality-metrics/docs/sample-dashboards/tfa_complexity.json
new file mode 100644
index 0000000..87ee822
--- /dev/null
+++ b/quality-metrics/docs/sample-dashboards/tfa_complexity.json
@@ -0,0 +1,487 @@
+{
+ "annotations": {
+ "list": [
+ {
+ "builtIn": 1,
+ "datasource": "-- Grafana --",
+ "enable": true,
+ "hide": true,
+ "iconColor": "rgba(0, 211, 255, 1)",
+ "name": "Annotations & Alerts",
+ "type": "dashboard"
+ }
+ ]
+ },
+ "editable": true,
+ "gnetId": null,
+ "graphTooltip": 0,
+ "id": 5,
+ "iteration": 1600199355075,
+ "links": [],
+ "panels": [
+ {
+ "content": "<b><center>First table shows the details for the latest tag and the second table shows the details for the selected \"Target Tag\". </b>",
+ "datasource": null,
+ "fieldConfig": {
+ "defaults": {
+ "custom": {}
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 2,
+ "w": 24,
+ "x": 0,
+ "y": 0
+ },
+ "id": 6,
+ "links": [],
+ "mode": "html",
+ "options": {
+ "content": "<b><center>First table shows the details for the latest tag and the second table shows the details for the selected \"Target Tag\". </b>",
+ "mode": "html"
+ },
+ "pluginVersion": "7.1.0",
+ "title": "Please note that when \"yes\" is selected from the above drop-down, data for both whitelisted and non-whitelisted functions is shown. Currently there are 0 whitelisted functions for TF-A.",
+ "type": "text"
+ },
+ {
+ "aliasColors": {},
+ "bars": true,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": "TFA_Complexity",
+ "description": "Tracking the number of functions exceeding threshold",
+ "fieldConfig": {
+ "defaults": {
+ "custom": {}
+ },
+ "overrides": []
+ },
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 9,
+ "w": 24,
+ "x": 0,
+ "y": 2
+ },
+ "hiddenSeries": false,
+ "id": 3,
+ "legend": {
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "show": false,
+ "total": false,
+ "values": false
+ },
+ "lines": false,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "percentage": false,
+ "pluginVersion": "7.1.3",
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "alias": "$tag_Target_Tag",
+ "dsType": "influxdb",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT max(Functions_Exceeding_Threshold_Not_Whitelisted) FROM \"TFA_Complexity_Tracking\" where ((Whitelisted =~ /^$Whitelisted$/ OR Whitelisted =~ /^no$/) AND $timeFilter) group by Target_Tag, Git_Tag_Date order by time",
+ "rawQuery": true,
+ "refId": "A",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "TFA Complexity Tracking",
+ "tooltip": {
+ "shared": false,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "series",
+ "name": null,
+ "show": true,
+ "values": [
+ "max"
+ ]
+ },
+ "yaxes": [
+ {
+ "decimals": 0,
+ "format": "short",
+ "label": "Functions Exceeding Threshold",
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": false
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "columns": [],
+ "datasource": "TFA_Complexity",
+ "fieldConfig": {
+ "defaults": {
+ "custom": {}
+ },
+ "overrides": []
+ },
+ "fontSize": "100%",
+ "gridPos": {
+ "h": 9,
+ "w": 24,
+ "x": 0,
+ "y": 11
+ },
+ "id": 4,
+ "links": [],
+ "pageSize": 100,
+ "scroll": true,
+ "showHeader": true,
+ "sort": {
+ "col": 0,
+ "desc": true
+ },
+ "styles": [
+ {
+ "alias": "Time",
+ "align": "auto",
+ "dateFormat": "YYYY-MM-DD HH:mm:ss",
+ "pattern": "Time",
+ "type": "date"
+ },
+ {
+ "alias": "",
+ "align": "auto",
+ "colorMode": null,
+ "colors": [
+ "rgba(245, 54, 54, 0.9)",
+ "rgba(237, 129, 40, 0.89)",
+ "rgba(50, 172, 45, 0.97)"
+ ],
+ "decimals": 2,
+ "pattern": "/.*/",
+ "thresholds": [],
+ "type": "number",
+ "unit": "short"
+ }
+ ],
+ "targets": [
+ {
+ "dsType": "influxdb",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT Base_Tag, Function_ID, Location, Score, Threshold, Whitelisted FROM \"TFA_Complexity_Statistics\" where ((Whitelisted =~ /^$Whitelisted$/ OR Whitelisted =~ /^no$/) AND $timeFilter AND Target_Tag =~ /^$TargetTag1$/) GROUP BY Target_Tag",
+ "rawQuery": true,
+ "refId": "A",
+ "resultFormat": "table",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ }
+ ],
+ "title": "TFA Complexity Statistics for Target Tag $TargetTag1",
+ "transform": "table",
+ "type": "table-old"
+ },
+ {
+ "columns": [],
+ "datasource": "TFA_Complexity",
+ "fieldConfig": {
+ "defaults": {
+ "custom": {}
+ },
+ "overrides": []
+ },
+ "fontSize": "100%",
+ "gridPos": {
+ "h": 9,
+ "w": 24,
+ "x": 0,
+ "y": 20
+ },
+ "id": 7,
+ "links": [],
+ "pageSize": 100,
+ "scroll": true,
+ "showHeader": true,
+ "sort": {
+ "col": 0,
+ "desc": true
+ },
+ "styles": [
+ {
+ "alias": "Time",
+ "align": "auto",
+ "dateFormat": "YYYY-MM-DD HH:mm:ss",
+ "pattern": "Time",
+ "type": "date"
+ },
+ {
+ "alias": "",
+ "align": "auto",
+ "colorMode": null,
+ "colors": [
+ "rgba(245, 54, 54, 0.9)",
+ "rgba(237, 129, 40, 0.89)",
+ "rgba(50, 172, 45, 0.97)"
+ ],
+ "decimals": 2,
+ "pattern": "/.*/",
+ "thresholds": [],
+ "type": "number",
+ "unit": "short"
+ }
+ ],
+ "targets": [
+ {
+ "dsType": "influxdb",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT Base_Tag, Function_ID, Location, Score, Threshold, Whitelisted FROM \"TFA_Complexity_Statistics\" where ((Whitelisted =~ /^$Whitelisted$/ OR Whitelisted =~ /^no$/) AND Target_Tag =~ /^$TargetTag2$/) GROUP BY Target_Tag",
+ "rawQuery": true,
+ "refId": "A",
+ "resultFormat": "table",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ }
+ ],
+ "title": "TFA Complexity Statistics for Target Tag $TargetTag2",
+ "transform": "table",
+ "type": "table-old"
+ }
+ ],
+ "schemaVersion": 26,
+ "style": "dark",
+ "tags": [
+ "TFA_QUALITY_METRICS"
+ ],
+ "templating": {
+ "list": [
+ {
+ "allValue": null,
+ "current": {
+ "tags": [],
+ "text": "no",
+ "value": "no"
+ },
+ "hide": 0,
+ "includeAll": false,
+ "label": "Show data including \"whitelisted\" functions?",
+ "multi": false,
+ "name": "Whitelisted",
+ "options": [
+ {
+ "selected": false,
+ "text": "yes",
+ "value": "yes"
+ },
+ {
+ "selected": true,
+ "text": "no",
+ "value": "no"
+ }
+ ],
+ "query": "yes,no",
+ "skipUrlSync": false,
+ "type": "custom"
+ },
+ {
+ "allValue": null,
+ "current": {
+ "selected": false,
+ "text": "v2.3",
+ "value": "v2.3"
+ },
+ "datasource": "TFA_Complexity",
+ "definition": "SELECT LAST(Target_Tag) FROM (SELECT Target_Tag, Function_ID FROM TFA_Complexity_Statistics)",
+ "hide": 2,
+ "includeAll": false,
+ "label": "Target Tag 1",
+ "multi": false,
+ "name": "TargetTag1",
+ "options": [],
+ "query": "SELECT LAST(Target_Tag) FROM (SELECT Target_Tag, Function_ID FROM TFA_Complexity_Statistics)",
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "sort": 0,
+ "tagValuesQuery": "",
+ "tags": [],
+ "tagsQuery": "",
+ "type": "query",
+ "useTags": false
+ },
+ {
+ "allValue": null,
+ "current": {
+ "selected": false,
+ "text": "v2.0",
+ "value": "v2.0"
+ },
+ "datasource": "TFA_Complexity",
+ "definition": "SHOW TAG VALUES WITH KEY=\"Target_Tag\"",
+ "hide": 0,
+ "includeAll": false,
+ "label": "Target Tag",
+ "multi": false,
+ "name": "TargetTag2",
+ "options": [],
+ "query": "SHOW TAG VALUES WITH KEY=\"Target_Tag\"",
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "sort": 0,
+ "tagValuesQuery": "",
+ "tags": [],
+ "tagsQuery": "",
+ "type": "query",
+ "useTags": false
+ }
+ ]
+ },
+ "time": {
+ "from": "now-2y",
+ "to": "now"
+ },
+ "timepicker": {
+ "refresh_intervals": [
+ "5s",
+ "10s",
+ "30s",
+ "1m",
+ "5m",
+ "15m",
+ "30m",
+ "1h",
+ "2h",
+ "1d"
+ ],
+ "time_options": [
+ "5m",
+ "15m",
+ "1h",
+ "6h",
+ "12h",
+ "24h",
+ "2d",
+ "7d",
+ "30d"
+ ]
+ },
+ "timezone": "",
+ "title": "TFA_Complexity",
+ "uid": "tfa-complexity",
+ "version": 6
+}
\ No newline at end of file
diff --git a/quality-metrics/docs/sample-dashboards/tfa_defects.json b/quality-metrics/docs/sample-dashboards/tfa_defects.json
new file mode 100644
index 0000000..17b7f1e
--- /dev/null
+++ b/quality-metrics/docs/sample-dashboards/tfa_defects.json
@@ -0,0 +1,275 @@
+{
+ "annotations": {
+ "list": [
+ {
+ "builtIn": 1,
+ "datasource": "-- Grafana --",
+ "enable": true,
+ "hide": true,
+ "iconColor": "rgba(0, 211, 255, 1)",
+ "name": "Annotations & Alerts",
+ "type": "dashboard"
+ }
+ ]
+ },
+ "editable": true,
+ "gnetId": null,
+ "graphTooltip": 0,
+ "id": 6,
+ "links": [],
+ "panels": [
+ {
+ "aliasColors": {},
+ "bars": true,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": "TFA_Defects",
+ "decimals": 0,
+ "description": "Tracks the number of defects of TF-A per release tag",
+ "fieldConfig": {
+ "defaults": {
+ "custom": {}
+ },
+ "overrides": []
+ },
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 7,
+ "w": 24,
+ "x": 0,
+ "y": 0
+ },
+ "hiddenSeries": false,
+ "id": 1,
+ "legend": {
+ "alignAsTable": false,
+ "avg": false,
+ "current": false,
+ "max": false,
+ "min": false,
+ "rightSide": false,
+ "show": false,
+ "total": false,
+ "values": false
+ },
+ "lines": false,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "percentage": false,
+ "pluginVersion": "7.1.3",
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "alias": "Total Defects",
+ "dsType": "influxdb",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT last(Number_of_Defects) as Defects FROM \"TFA_Defects_Tracking\" where $timeFilter GROUP BY time(1w) fill(none)",
+ "rawQuery": true,
+ "refId": "A",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "TFA Defects Tracking",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "decimals": 0,
+ "format": "short",
+ "label": "Number of Defects",
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": false
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "datasource": "TFA_Defects",
+ "fieldConfig": {
+ "defaults": {
+ "custom": {
+ "align": null
+ },
+ "mappings": [],
+ "thresholds": {
+ "mode": "absolute",
+ "steps": [
+ {
+ "color": "green",
+ "value": null
+ },
+ {
+ "color": "red",
+ "value": 80
+ }
+ ]
+ }
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 8,
+ "w": 24,
+ "x": 0,
+ "y": 7
+ },
+ "id": 2,
+ "links": [],
+ "options": {
+ "showHeader": true
+ },
+ "pluginVersion": "7.1.3",
+ "targets": [
+ {
+ "alias": "",
+ "dsType": "influxdb",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT * FROM \"TFA_Defects_Statistics\" where $timeFilter",
+ "rawQuery": true,
+ "refId": "A",
+ "resultFormat": "table",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ }
+ ],
+ "title": "TF-A Defects Statistics",
+ "type": "table"
+ }
+ ],
+ "refresh": "1d",
+ "schemaVersion": 26,
+ "style": "dark",
+ "tags": [
+ "TFA_QUALITY_METRICS"
+ ],
+ "templating": {
+ "list": []
+ },
+ "time": {
+ "from": "now-90d",
+ "to": "now"
+ },
+ "timepicker": {
+ "refresh_intervals": [
+ "5s",
+ "10s",
+ "30s",
+ "1m",
+ "5m",
+ "15m",
+ "30m",
+ "1h",
+ "2h",
+ "1d"
+ ],
+ "time_options": [
+ "5m",
+ "15m",
+ "1h",
+ "6h",
+ "12h",
+ "24h",
+ "2d",
+ "7d",
+ "30d"
+ ]
+ },
+ "timezone": "",
+ "title": "TFA_Defects",
+ "uid": "tfa-defects",
+ "version": 6
+}
\ No newline at end of file
diff --git a/quality-metrics/docs/sample-dashboards/tfa_image_size.json b/quality-metrics/docs/sample-dashboards/tfa_image_size.json
new file mode 100644
index 0000000..8325e46
--- /dev/null
+++ b/quality-metrics/docs/sample-dashboards/tfa_image_size.json
@@ -0,0 +1,642 @@
+{
+ "annotations": {
+ "list": [
+ {
+ "builtIn": 1,
+ "datasource": "TFA_ImageSize",
+ "enable": false,
+ "hide": false,
+ "iconColor": "#e0f9d7",
+ "limit": 100,
+ "name": "View PR Details",
+ "query": "select PullRequestURL, Pull_Request_Title, CommitID, BL1_B, BL2_B, BL1U_B, BL2U_B, BL31_B, BL32_B from \"[[BuildConfig]]\"",
+ "showIn": 0,
+ "tagsColumn": "CommitID",
+ "textColumn": "Pull_Request_Title",
+ "type": "dashboard"
+ }
+ ]
+ },
+ "editable": true,
+ "gnetId": null,
+ "graphTooltip": 0,
+ "id": 9,
+ "iteration": 1600199357868,
+ "links": [],
+ "panels": [
+ {
+ "aliasColors": {},
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": "TFA_ImageSize",
+ "decimals": 3,
+ "description": "This shows the trend in image size of .ELF files for selected build config in stacked manner. The values are individually stacked, not cumulative.\n\nBuild Config and Bin Mode can be changed from the drop down at the top of dashboard.",
+ "fieldConfig": {
+ "defaults": {
+ "custom": {}
+ },
+ "overrides": []
+ },
+ "fill": 2,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 9,
+ "w": 24,
+ "x": 0,
+ "y": 0
+ },
+ "hiddenSeries": false,
+ "id": 10,
+ "legend": {
+ "avg": false,
+ "current": true,
+ "hideEmpty": true,
+ "hideZero": false,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": true
+ },
+ "lines": true,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "percentage": false,
+ "pluginVersion": "7.1.3",
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "alias": "$col",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT (\"BL1_B\" + \"BL1_D\" + \"BL1_R\" + \"BL1_T\" + \"BL1_W\") as bl1, (\"BL1U_B\" + \"BL1U_D\" + \"BL1U_R\" + \"BL1U_T\" + \"BL1U_W\") as bl1u, (\"BL2_B\" + \"BL2_D\" + \"BL2_R\" + \"BL2_T\" + \"BL2_W\") as bl2, (\"BL2U_B\" + \"BL2U_D\" + \"BL2U_R\" + \"BL2U_T\" + \"BL2U_W\") as bl2u, (\"BL31_B\" + \"BL31_D\" + \"BL31_R\" + \"BL31_T\" + \"BL31_W\") as bl31, (\"BL32_B\" + \"BL32_D\" + \"BL32_R\" + \"BL32_T\" + \"BL32_W\") as bl32 from \"$BuildConfig\" WHERE (\"BinMode\" =~ /^$BinMode$/ ) fill(0)",
+ "rawQuery": true,
+ "refId": "A",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "TF-A Image Size Time Series Graph for Selected Build Config ($BuildConfig) and Bin Mode ($BinMode)",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "decimals": 3,
+ "format": "decbytes",
+ "label": "Image Size",
+ "logBase": 1,
+ "max": null,
+ "min": "0",
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": "",
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": false
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "columns": [],
+ "datasource": "TFA_ImageSize",
+ "fieldConfig": {
+ "defaults": {
+ "custom": {}
+ },
+ "overrides": []
+ },
+ "fontSize": "100%",
+ "gridPos": {
+ "h": 9,
+ "w": 24,
+ "x": 0,
+ "y": 9
+ },
+ "id": 4,
+ "links": [],
+ "pageSize": null,
+ "scroll": true,
+ "showHeader": true,
+ "sort": {
+ "col": 0,
+ "desc": true
+ },
+ "styles": [
+ {
+ "alias": "Time",
+ "align": "auto",
+ "dateFormat": "YYYY-MM-DD HH:mm:ss",
+ "pattern": "Time",
+ "type": "date"
+ },
+ {
+ "alias": "",
+ "align": "auto",
+ "colorMode": null,
+ "colors": [
+ "rgba(245, 54, 54, 0.9)",
+ "rgba(237, 129, 40, 0.89)",
+ "rgba(50, 172, 45, 0.97)"
+ ],
+ "dateFormat": "YYYY-MM-DD HH:mm:ss",
+ "decimals": 3,
+ "pattern": "/^bl*/",
+ "thresholds": [],
+ "type": "number",
+ "unit": "decbytes"
+ },
+ {
+ "alias": "",
+ "align": "auto",
+ "colorMode": null,
+ "colors": [
+ "rgba(245, 54, 54, 0.9)",
+ "rgba(237, 129, 40, 0.89)",
+ "rgba(50, 172, 45, 0.97)"
+ ],
+ "dateFormat": "YYYY-MM-DD HH:mm:ss",
+ "decimals": 2,
+ "link": true,
+ "linkTargetBlank": true,
+ "linkTooltip": "",
+ "linkUrl": "${__cell:raw}",
+ "pattern": "Pull Request URL",
+ "thresholds": [],
+ "type": "number",
+ "unit": "short"
+ },
+ {
+ "alias": "",
+ "align": "auto",
+ "colorMode": null,
+ "colors": [
+ "rgba(245, 54, 54, 0.9)",
+ "rgba(237, 129, 40, 0.89)",
+ "rgba(50, 172, 45, 0.97)"
+ ],
+ "decimals": 2,
+ "pattern": "/.*/",
+ "thresholds": [],
+ "type": "number",
+ "unit": "short"
+ }
+ ],
+ "targets": [
+ {
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT CommitTitle as \"Commit Title\", \"BinMode\", \"CommitID\" as \"Commit ID\", (\"BL1_B\" + \"BL1_D\" + \"BL1_R\" + \"BL1_T\" + \"BL1_W\") as bl1, (\"BL1U_B\" + \"BL1U_D\" + \"BL1U_R\" + \"BL1U_T\" + \"BL1U_W\") as bl1u, (\"BL2_B\" + \"BL2_D\" + \"BL2_R\" + \"BL2_T\" + \"BL2_W\") as bl2, (\"BL2U_B\" + \"BL2U_D\" + \"BL2U_R\" + \"BL2U_T\" + \"BL2U_W\") as bl2u, (\"BL31_B\" + \"BL31_D\" + \"BL31_R\" + \"BL31_T\" + \"BL31_W\") as bl31, (\"BL32_B\" + \"BL32_D\" + \"BL32_R\" + \"BL32_T\" + \"BL32_W\") as bl32 from \"$BuildConfig\" WHERE (\"BinMode\" =~ /^$BinMode$/ ) fill(0)",
+ "rawQuery": true,
+ "refId": "A",
+ "resultFormat": "table",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ }
+ ],
+ "title": "TF-A Image Size Details Table for $BuildConfig",
+ "transform": "table",
+ "type": "table-old"
+ },
+ {
+ "aliasColors": {},
+ "bars": true,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": "TFA_ImageSize",
+ "decimals": 3,
+ "description": "",
+ "fieldConfig": {
+ "defaults": {
+ "custom": {}
+ },
+ "overrides": []
+ },
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 9,
+ "w": 24,
+ "x": 0,
+ "y": 18
+ },
+ "hiddenSeries": false,
+ "id": 8,
+ "legend": {
+ "alignAsTable": true,
+ "avg": false,
+ "current": true,
+ "hideEmpty": true,
+ "hideZero": true,
+ "max": false,
+ "min": false,
+ "rightSide": true,
+ "show": true,
+ "sideWidth": null,
+ "total": false,
+ "values": true
+ },
+ "lines": false,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "percentage": false,
+ "pluginVersion": "7.1.3",
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "alias": "$col",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT *\nFROM \"$BuildConfig\"\nWHERE (\"CommitID\" =~ /^$CommitID$/ AND \"BinMode\" =~ /^$BinMode$/ )\nGROUP BY \"BinMode\", \"PullRequestURL\", \"CommitID\"",
+ "rawQuery": true,
+ "refId": "A",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Image size memory details for selected Commit ID ($CommitID) and Bin Mode ($BinMode) for $BuildConfig",
+ "tooltip": {
+ "shared": false,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "series",
+ "name": null,
+ "show": true,
+ "values": [
+ "total"
+ ]
+ },
+ "yaxes": [
+ {
+ "decimals": 3,
+ "format": "decbytes",
+ "label": "",
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": "",
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "aliasColors": {},
+ "breakPoint": "50%",
+ "cacheTimeout": null,
+ "combine": {
+ "label": "Others",
+ "threshold": 0
+ },
+ "datasource": "TFA_ImageSize",
+ "decimals": 3,
+ "fieldConfig": {
+ "defaults": {
+ "custom": {}
+ },
+ "overrides": []
+ },
+ "fontSize": "80%",
+ "format": "decbytes",
+ "gridPos": {
+ "h": 9,
+ "w": 19,
+ "x": 0,
+ "y": 27
+ },
+ "id": 6,
+ "interval": null,
+ "legend": {
+ "header": "Image Size",
+ "show": true,
+ "sideWidth": 300,
+ "values": true
+ },
+ "legendType": "Right side",
+ "links": [],
+ "maxDataPoints": 3,
+ "nullPointMode": "connected",
+ "pieType": "pie",
+ "strokeWidth": "2",
+ "targets": [
+ {
+ "alias": "$col",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT (\"BL1_B\" + \"BL1_D\" + \"BL1_R\" + \"BL1_T\" + \"BL1_W\") as bl1, (\"BL1U_B\" + \"BL1U_D\" + \"BL1U_R\" + \"BL1U_T\" + \"BL1U_W\") as bl1u, (\"BL2_B\" + \"BL2_D\" + \"BL2_R\" + \"BL2_T\" + \"BL2_W\") as bl2, (\"BL2U_B\" + \"BL2U_D\" + \"BL2U_R\" + \"BL2U_T\" + \"BL2U_W\") as bl2u, (\"BL31_B\" + \"BL31_D\" + \"BL31_R\" + \"BL31_T\" + \"BL31_W\") as bl31, (\"BL32_B\" + \"BL32_D\" + \"BL32_R\" + \"BL32_T\" + \"BL32_W\") as bl32 \nFROM \"$BuildConfig\"\nWHERE (\"CommitID\" =~ /^$CommitID$/ AND (\"BinMode\" =~ /^$BinMode$/ ))\nGROUP BY \"CommitID\", \"PullRequestURL\", \"BinMode\" fill(0)\n",
+ "rawQuery": true,
+ "refId": "A",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ }
+ ],
+ "title": "Image size pie chart for selected Commit ID ($CommitID) and Bin Mode ($BinMode) for $BuildConfig",
+ "type": "grafana-piechart-panel",
+ "valueName": "current"
+ },
+ {
+ "content": "<li>\"BlX_B\": Size of uninitialized data section</li>\n<li>\"BlX_D\": Size of initialized data section</li>\n<li>\"BlX_R\": Size of read only data section</li>\n<li>\"BlX_T\": Size of text (code) section</li>\n<li>\"BlX_V\": Size of weak object</li>\n<li>\"BlX_W\": Size of weak symbol</li>\n\n<br>Build Config, Commit ID and Bin Mode can<br>\nbe changed from the drop down menu at <br>the top of\npage.",
+ "datasource": null,
+ "fieldConfig": {
+ "defaults": {
+ "custom": {}
+ },
+ "overrides": []
+ },
+ "gridPos": {
+ "h": 8,
+ "w": 5,
+ "x": 19,
+ "y": 27
+ },
+ "id": 12,
+ "links": [],
+ "mode": "html",
+ "options": {
+ "content": "<li>\"BlX_B\": Size of uninitialized data section</li>\n<li>\"BlX_D\": Size of initialized data section</li>\n<li>\"BlX_R\": Size of read only data section</li>\n<li>\"BlX_T\": Size of text (code) section</li>\n<li>\"BlX_V\": Size of weak object</li>\n<li>\"BlX_W\": Size of weak symbol</li>\n\n<br>Build Config, Commit ID and Bin Mode can<br>\nbe changed from the drop down menu at <br>the top of\npage.",
+ "mode": "html"
+ },
+ "pluginVersion": "7.1.0",
+ "title": "Memory Section Details:",
+ "transparent": true,
+ "type": "text"
+ }
+ ],
+ "refresh": "1d",
+ "schemaVersion": 26,
+ "style": "dark",
+ "tags": [
+ "TFA_QUALITY_METRICS"
+ ],
+ "templating": {
+ "list": [
+ {
+ "allValue": null,
+ "current": {
+ "selected": false,
+ "text": "poplar-default",
+ "value": "poplar-default"
+ },
+ "datasource": "TFA_ImageSize",
+ "definition": "",
+ "hide": 0,
+ "includeAll": false,
+ "label": "Build Config",
+ "multi": false,
+ "name": "BuildConfig",
+ "options": [],
+ "query": "show measurements",
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "sort": 1,
+ "tagValuesQuery": "",
+ "tags": [],
+ "tagsQuery": "",
+ "type": "query",
+ "useTags": false
+ },
+ {
+ "allValue": null,
+ "current": {
+ "selected": false,
+ "text": "9b2bf15016",
+ "value": "9b2bf15016"
+ },
+ "datasource": "TFA_ImageSize",
+ "definition": "",
+ "hide": 0,
+ "includeAll": false,
+ "label": "Commit ID",
+ "multi": false,
+ "name": "CommitID",
+ "options": [],
+ "query": "SHOW TAG VALUES WITH KEY = \"CommitID\"",
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "sort": 0,
+ "tagValuesQuery": "",
+ "tags": [],
+ "tagsQuery": "",
+ "type": "query",
+ "useTags": false
+ },
+ {
+ "allValue": null,
+ "current": {
+ "selected": false,
+ "text": "Debug",
+ "value": "Debug"
+ },
+ "datasource": "TFA_ImageSize",
+ "definition": "",
+ "hide": 0,
+ "includeAll": false,
+ "label": "Bin Mode",
+ "multi": false,
+ "name": "BinMode",
+ "options": [],
+ "query": "SHOW TAG VALUES WITH KEY = \"BinMode\"",
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "sort": 0,
+ "tagValuesQuery": "",
+ "tags": [],
+ "tagsQuery": "",
+ "type": "query",
+ "useTags": false
+ }
+ ]
+ },
+ "time": {
+ "from": "now-30d",
+ "to": "now"
+ },
+ "timepicker": {
+ "refresh_intervals": [
+ "1d"
+ ],
+ "time_options": [
+ "5m",
+ "15m",
+ "1h",
+ "6h",
+ "12h",
+ "24h",
+ "2d",
+ "7d",
+ "30d"
+ ]
+ },
+ "timezone": "",
+ "title": "TFA_Image_Size",
+ "uid": "GkzYOKFiz",
+ "version": 6
+}
\ No newline at end of file
diff --git a/quality-metrics/docs/sample-dashboards/tfa_misra_defects.json b/quality-metrics/docs/sample-dashboards/tfa_misra_defects.json
new file mode 100644
index 0000000..303e802
--- /dev/null
+++ b/quality-metrics/docs/sample-dashboards/tfa_misra_defects.json
@@ -0,0 +1,897 @@
+{
+ "annotations": {
+ "list": [
+ {
+ "builtIn": 1,
+ "datasource": "TFA_MisraDefects",
+ "enable": false,
+ "hide": false,
+ "iconColor": "#e0f9d7",
+ "limit": 100,
+ "name": "View PR Details",
+ "query": "select PullRequestURL, Pull_Request_Title, CommitID, TotalDefects from \"[[BuildConfig]]\"",
+ "showIn": 0,
+ "tagsColumn": "CommitID",
+ "textColumn": "Pull_Request_Title",
+ "type": "dashboard"
+ }
+ ]
+ },
+ "editable": true,
+ "gnetId": null,
+ "graphTooltip": 0,
+ "id": 8,
+ "iteration": 1600199358960,
+ "links": [],
+ "panels": [
+ {
+ "aliasColors": {
+ "MandatoryDefects": "#bf1b00",
+ "RequiredDefects": "#eab839"
+ },
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": "TFA_MisraDefects",
+ "fieldConfig": {
+ "defaults": {
+ "custom": {}
+ },
+ "overrides": []
+ },
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 9,
+ "w": 24,
+ "x": 0,
+ "y": 0
+ },
+ "hiddenSeries": false,
+ "id": 2,
+ "legend": {
+ "alignAsTable": false,
+ "avg": false,
+ "current": true,
+ "max": false,
+ "min": false,
+ "show": true,
+ "total": false,
+ "values": true
+ },
+ "lines": true,
+ "linewidth": 2,
+ "links": [],
+ "nullPointMode": "null as zero",
+ "percentage": false,
+ "pluginVersion": "7.1.3",
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "alias": "$col",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT TotalDefects FROM \"$BuildConfig\" WHERE (\"BinMode\" =~ /^Debug$/ )",
+ "rawQuery": true,
+ "refId": "A",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ },
+ {
+ "alias": "$col",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT MandatoryDefects FROM \"$BuildConfig\" WHERE (\"BinMode\" =~ /^Debug$/ )",
+ "rawQuery": true,
+ "refId": "B",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ },
+ {
+ "alias": "$col",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT RequiredDefects FROM \"$BuildConfig\" WHERE (\"BinMode\" =~ /^Debug$/ )",
+ "rawQuery": true,
+ "refId": "C",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ },
+ {
+ "alias": "$col",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT AdvisoryDefects FROM \"$BuildConfig\" WHERE (\"BinMode\" =~ /^Debug$/ )",
+ "rawQuery": true,
+ "refId": "D",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "TF-A Misra Defects Time Series Graph for Selected Build Config ($BuildConfig) and Bin Mode Debug",
+ "tooltip": {
+ "shared": true,
+ "sort": 1,
+ "value_type": "individual"
+ },
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": false
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "columns": [],
+ "datasource": "TFA_MisraDefects",
+ "fieldConfig": {
+ "defaults": {
+ "custom": {}
+ },
+ "overrides": []
+ },
+ "fontSize": "100%",
+ "gridPos": {
+ "h": 9,
+ "w": 24,
+ "x": 0,
+ "y": 9
+ },
+ "id": 4,
+ "links": [],
+ "pageSize": null,
+ "scroll": true,
+ "showHeader": true,
+ "sort": {
+ "col": 0,
+ "desc": true
+ },
+ "styles": [
+ {
+ "alias": "Time",
+ "align": "auto",
+ "dateFormat": "YYYY-MM-DD HH:mm:ss",
+ "pattern": "Time",
+ "type": "date"
+ },
+ {
+ "alias": "",
+ "align": "auto",
+ "colorMode": null,
+ "colors": [
+ "rgba(245, 54, 54, 0.9)",
+ "rgba(237, 129, 40, 0.89)",
+ "rgba(50, 172, 45, 0.97)"
+ ],
+ "dateFormat": "YYYY-MM-DD HH:mm:ss",
+ "decimals": 2,
+ "link": true,
+ "linkTargetBlank": true,
+ "linkTooltip": "",
+ "linkUrl": "${__cell:raw} ",
+ "pattern": "Pull Request URL",
+ "thresholds": [],
+ "type": "number",
+ "unit": "short"
+ },
+ {
+ "alias": "",
+ "align": "auto",
+ "colorMode": null,
+ "colors": [
+ "rgba(245, 54, 54, 0.9)",
+ "rgba(237, 129, 40, 0.89)",
+ "rgba(50, 172, 45, 0.97)"
+ ],
+ "decimals": 3,
+ "pattern": "/.*/",
+ "thresholds": [],
+ "type": "number",
+ "unit": "short"
+ }
+ ],
+ "targets": [
+ {
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT CommitTitle as \"Commit Title\", \"BinMode\", \"CommitID\" as \"Commit ID\", \"MandatoryDefects\" as \"Mandatory Defects\", \"RequiredDefects\" as \"Required Defects\", \"AdvisoryDefects\" as \"Advisory Defects\", \"TotalDefects\" as \"Total Defects\" from \"$BuildConfig\" WHERE (\"BinMode\" =~ /^Debug$/ )",
+ "rawQuery": true,
+ "refId": "A",
+ "resultFormat": "table",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ }
+ ],
+ "title": "TF-A Misra C Defects Details Table for $BuildConfig",
+ "transform": "table",
+ "type": "table-old"
+ },
+ {
+ "columns": [],
+ "datasource": "TFA_MisraDefects",
+ "fieldConfig": {
+ "defaults": {
+ "custom": {}
+ },
+ "overrides": []
+ },
+ "fontSize": "100%",
+ "gridPos": {
+ "h": 5,
+ "w": 24,
+ "x": 0,
+ "y": 18
+ },
+ "id": 6,
+ "links": [],
+ "pageSize": null,
+ "scroll": true,
+ "showHeader": true,
+ "sort": {
+ "col": 0,
+ "desc": true
+ },
+ "styles": [
+ {
+ "alias": "Time",
+ "align": "auto",
+ "dateFormat": "YYYY-MM-DD HH:mm:ss",
+ "pattern": "Time",
+ "type": "date"
+ },
+ {
+ "alias": "",
+ "align": "auto",
+ "colorMode": null,
+ "colors": [
+ "rgba(245, 54, 54, 0.9)",
+ "rgba(237, 129, 40, 0.89)",
+ "rgba(50, 172, 45, 0.97)"
+ ],
+ "dateFormat": "YYYY-MM-DD HH:mm:ss",
+ "decimals": 2,
+ "link": true,
+ "linkTargetBlank": true,
+ "linkTooltip": "",
+ "linkUrl": "$__cell",
+ "pattern": "Pull Request URL",
+ "thresholds": [],
+ "type": "number",
+ "unit": "short"
+ },
+ {
+ "alias": "",
+ "align": "auto",
+ "colorMode": null,
+ "colors": [
+ "rgba(245, 54, 54, 0.9)",
+ "rgba(237, 129, 40, 0.89)",
+ "rgba(50, 172, 45, 0.97)"
+ ],
+ "decimals": 3,
+ "pattern": "/.*/",
+ "thresholds": [],
+ "type": "number",
+ "unit": "short"
+ }
+ ],
+ "targets": [
+ {
+ "alias": "$col",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT \"CommitID\" as \"Commit ID\", CommitTitle as \"Commit Title\", \"BinMode\", \"MandatoryDefects\" as \"Mandatory Defects\", \"RequiredDefects\" as \"Required Defects\", \"AdvisoryDefects\" as \"Advisory Defects\", \"TotalDefects\" as \"Total Defects\" from \"$BuildConfig\" WHERE (\"BinMode\" =~ /^Debug$/ AND \"CommitID\" =~ /^$CommitID1$/ )",
+ "rawQuery": true,
+ "refId": "A",
+ "resultFormat": "table",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ },
+ {
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT \"CommitID\" as \"Commit ID\", CommitTitle as \"Commit Title\", \"BinMode\", \"MandatoryDefects\" as \"Mandatory Defects\", \"RequiredDefects\" as \"Required Defects\", \"AdvisoryDefects\" as \"Advisory Defects\", \"TotalDefects\" as \"Total Defects\" from \"$BuildConfig\" WHERE (\"BinMode\" =~ /^Debug$/ AND \"CommitID\" =~ /^$CommitID2$/ )",
+ "rawQuery": true,
+ "refId": "B",
+ "resultFormat": "table",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ }
+ ],
+ "title": "Comparison table between $CommitID1 and $CommitID2 for BinMode \"Debug\"",
+ "transform": "table",
+ "transparent": true,
+ "type": "table-old"
+ },
+ {
+ "aliasColors": {
+ "Advisory Defects": "#ef843c",
+ "Mandatory Defects": "#bf1b00",
+ "Total Defects": "#629e51"
+ },
+ "bars": true,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": "TFA_MisraDefects",
+ "fieldConfig": {
+ "defaults": {
+ "custom": {}
+ },
+ "overrides": []
+ },
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 9,
+ "w": 24,
+ "x": 0,
+ "y": 23
+ },
+ "hiddenSeries": false,
+ "id": 8,
+ "legend": {
+ "alignAsTable": true,
+ "avg": false,
+ "current": true,
+ "max": false,
+ "min": false,
+ "rightSide": true,
+ "show": true,
+ "total": false,
+ "values": true
+ },
+ "lines": false,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "percentage": false,
+ "pluginVersion": "7.1.3",
+ "pointradius": 5,
+ "points": false,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "alias": "Mandatory Defects",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "hide": false,
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT DIFFERENCE(\"MandatoryDefects\") FROM \"$BuildConfig\" WHERE ((\"CommitID\" =~ /^$CommitID1$/ AND (\"BinMode\" =~ /^Debug$/ )) OR (\"CommitID\" =~ /^$CommitID2$/ AND (\"BinMode\" =~ /^Debug$/ )))",
+ "rawQuery": true,
+ "refId": "A",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ },
+ {
+ "alias": "Required Defects",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "hide": false,
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT DIFFERENCE(\"RequiredDefects\") FROM \"$BuildConfig\" WHERE ((\"CommitID\" =~ /^$CommitID1$/ AND (\"BinMode\" =~ /^Debug$/ )) OR (\"CommitID\" =~ /^$CommitID2$/ AND (\"BinMode\" =~ /^Debug$/ )))",
+ "rawQuery": true,
+ "refId": "B",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ },
+ {
+ "alias": "Advisory Defects",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "hide": false,
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT DIFFERENCE(\"AdvisoryDefects\") FROM \"$BuildConfig\" WHERE ((\"CommitID\" =~ /^$CommitID1$/ AND (\"BinMode\" =~ /^Debug$/ )) OR (\"CommitID\" =~ /^$CommitID2$/ AND (\"BinMode\" =~ /^Debug$/ )))",
+ "rawQuery": true,
+ "refId": "C",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ },
+ {
+ "alias": "Total Defects",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "hide": false,
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT DIFFERENCE(\"TotalDefects\") FROM \"$BuildConfig\" WHERE ((\"CommitID\" =~ /^$CommitID1$/ AND (\"BinMode\" =~ /^Debug$/ )) OR (\"CommitID\" =~ /^$CommitID2$/ AND (\"BinMode\" =~ /^Debug$/ )))",
+ "rawQuery": true,
+ "refId": "D",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "Comparison Chart between [$CommitID1,Debug] and [$CommitID2,Debug] Misra C Defects for \"$BuildConfig\"",
+ "tooltip": {
+ "shared": false,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "transparent": true,
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "series",
+ "name": null,
+ "show": true,
+ "values": [
+ "total"
+ ]
+ },
+ "yaxes": [
+ {
+ "format": "short",
+ "label": "Difference in Misra C Defects",
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": true
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ }
+ ],
+ "schemaVersion": 26,
+ "style": "dark",
+ "tags": [
+ "TFA_QUALITY_METRICS"
+ ],
+ "templating": {
+ "list": [
+ {
+ "allValue": null,
+ "current": {
+ "selected": false,
+ "text": "fvp-rst-bl31",
+ "value": "fvp-rst-bl31"
+ },
+ "datasource": "TFA_MisraDefects",
+ "definition": "",
+ "hide": 0,
+ "includeAll": false,
+ "label": "Build Config",
+ "multi": false,
+ "name": "BuildConfig",
+ "options": [],
+ "query": "show measurements",
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "sort": 0,
+ "tagValuesQuery": "",
+ "tags": [],
+ "tagsQuery": "",
+ "type": "query",
+ "useTags": false
+ },
+ {
+ "allValue": null,
+ "current": {
+ "selected": false,
+ "text": "9b2bf15016",
+ "value": "9b2bf15016"
+ },
+ "datasource": "TFA_MisraDefects",
+ "definition": "",
+ "hide": 0,
+ "includeAll": false,
+ "label": "Commit ID 1",
+ "multi": false,
+ "name": "CommitID1",
+ "options": [],
+ "query": "SHOW TAG VALUES WITH KEY = \"CommitID\" ",
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "sort": 0,
+ "tagValuesQuery": "",
+ "tags": [],
+ "tagsQuery": "",
+ "type": "query",
+ "useTags": false
+ },
+ {
+ "allValue": null,
+ "current": {
+ "selected": false,
+ "text": "a41ca4c344",
+ "value": "a41ca4c344"
+ },
+ "datasource": "TFA_MisraDefects",
+ "definition": "",
+ "hide": 0,
+ "includeAll": false,
+ "label": "Commit ID 2",
+ "multi": false,
+ "name": "CommitID2",
+ "options": [],
+ "query": "SHOW TAG VALUES WITH KEY = \"CommitID\" ",
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "sort": 0,
+ "tagValuesQuery": "",
+ "tags": [],
+ "tagsQuery": "",
+ "type": "query",
+ "useTags": false
+ }
+ ]
+ },
+ "time": {
+ "from": "now-30d",
+ "to": "now"
+ },
+ "timepicker": {
+ "refresh_intervals": [
+ "1d"
+ ],
+ "time_options": [
+ "5m",
+ "15m",
+ "1h",
+ "6h",
+ "12h",
+ "24h",
+ "2d",
+ "7d",
+ "30d"
+ ]
+ },
+ "timezone": "",
+ "title": "TFA_Misra_Defects",
+ "uid": "41hRgW-mz",
+ "version": 10
+}
\ No newline at end of file
diff --git a/quality-metrics/docs/sample-dashboards/tfa_runtime_perf_details.json b/quality-metrics/docs/sample-dashboards/tfa_runtime_perf_details.json
new file mode 100644
index 0000000..bbe140b
--- /dev/null
+++ b/quality-metrics/docs/sample-dashboards/tfa_runtime_perf_details.json
@@ -0,0 +1,1089 @@
+{
+ "annotations": {
+ "list": [
+ {
+ "builtIn": 1,
+ "datasource": "TFA_RunTime_Perf",
+ "enable": false,
+ "hide": false,
+ "iconColor": "#fce2de",
+ "limit": 100,
+ "name": "View Commit Details",
+ "query": "select CommitTitle, CommitID, TC_Name, CPU_Core, CacheFlush, Latency_EL3Entry_EL3Exit from \"[[Instrumentation]]\" WHERE ($timeFilter AND \"TC_Name\" =~ /^$TestCase$/ AND \"CPU_Core\" =~ /^0/ AND \"Cluster_ID\" =~ /^0/) LIMIT 1000",
+ "showIn": 0,
+ "tagsColumn": "CommitID",
+ "textColumn": "CommitTitle",
+ "type": "dashboard"
+ }
+ ]
+ },
+ "editable": true,
+ "gnetId": null,
+ "graphTooltip": 0,
+ "id": 10,
+ "iteration": 1600199360656,
+ "links": [],
+ "panels": [
+ {
+ "aliasColors": {
+ "Cluster ID: 0, CPU Core: 0": "#eab839",
+ "Cluster ID: 0, CPU Core: 1": "#eab839",
+ "Cluster ID: 1, CPU Core: 0": "#70dbed",
+ "Cluster ID: 1, CPU Core: 1": "#70dbed",
+ "Cluster ID: 1, CPU Core: 2": "#70dbed",
+ "Cluster ID: 1, CPU Core: 3": "#70dbed",
+ "Latency_CPUWakeup_EL3Exit for CPU Core 0": "#e24d42"
+ },
+ "bars": false,
+ "dashLength": 10,
+ "dashes": false,
+ "datasource": "TFA_RunTime_Perf",
+ "decimals": 3,
+ "fieldConfig": {
+ "defaults": {
+ "custom": {}
+ },
+ "overrides": []
+ },
+ "fill": 1,
+ "fillGradient": 0,
+ "gridPos": {
+ "h": 9,
+ "w": 24,
+ "x": 0,
+ "y": 0
+ },
+ "hiddenSeries": false,
+ "id": 2,
+ "legend": {
+ "alignAsTable": true,
+ "avg": false,
+ "current": true,
+ "hideEmpty": true,
+ "hideZero": false,
+ "max": false,
+ "min": false,
+ "rightSide": true,
+ "show": true,
+ "total": false,
+ "values": true
+ },
+ "lines": false,
+ "linewidth": 1,
+ "links": [],
+ "nullPointMode": "null",
+ "percentage": false,
+ "pluginVersion": "7.1.3",
+ "pointradius": 3,
+ "points": true,
+ "renderer": "flot",
+ "seriesOverrides": [],
+ "spaceLength": 10,
+ "stack": false,
+ "steppedLine": false,
+ "targets": [
+ {
+ "alias": "Cluster ID: 0, CPU Core: 0",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT \"$Latency\" FROM \"$Instrumentation\" WHERE (\"TC_Name\" =~ /^$TestCase$/ AND \"CPU_Core\" =~ /^0/ AND \"Cluster_ID\" =~ /^0/)",
+ "rawQuery": true,
+ "refId": "A",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ },
+ {
+ "alias": "Cluster ID: 0, CPU Core: 1",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT \"$Latency\" FROM \"$Instrumentation\" WHERE (\"TC_Name\" =~ /^$TestCase$/ AND \"CPU_Core\" =~ /^1/ AND \"Cluster_ID\" =~ /^0/)",
+ "rawQuery": true,
+ "refId": "B",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ },
+ {
+ "alias": "Cluster ID: 0, CPU Core: 2",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT \"$Latency\" FROM \"$Instrumentation\" WHERE (\"TC_Name\" =~ /^$TestCase$/ AND \"CPU_Core\" =~ /^2/ AND \"Cluster_ID\" =~ /^0/)",
+ "rawQuery": true,
+ "refId": "C",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ },
+ {
+ "alias": "Cluster ID: 0, CPU Core: 3",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT \"$Latency\" FROM \"$Instrumentation\" WHERE (\"TC_Name\" =~ /^$TestCase$/ AND \"CPU_Core\" =~ /^3/ AND \"Cluster_ID\" =~ /^0/)",
+ "rawQuery": true,
+ "refId": "D",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ },
+ {
+ "alias": "Cluster ID: 1, CPU Core: 0",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT \"$Latency\" FROM \"$Instrumentation\" WHERE (\"TC_Name\" =~ /^$TestCase$/ AND \"CPU_Core\" =~ /^0/ AND \"Cluster_ID\" =~ /^1/)",
+ "rawQuery": true,
+ "refId": "E",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ },
+ {
+ "alias": "Cluster ID: 1, CPU Core: 1",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT \"$Latency\" FROM \"$Instrumentation\" WHERE (\"TC_Name\" =~ /^$TestCase$/ AND \"CPU_Core\" =~ /^1/ AND \"Cluster_ID\" =~ /^1/)",
+ "rawQuery": true,
+ "refId": "F",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ },
+ {
+ "alias": "Cluster ID: 1, CPU Core: 2",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT \"$Latency\" FROM \"$Instrumentation\" WHERE (\"TC_Name\" =~ /^$TestCase$/ AND \"CPU_Core\" =~ /^2/ AND \"Cluster_ID\" =~ /^1/)",
+ "rawQuery": true,
+ "refId": "G",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ },
+ {
+ "alias": "Cluster ID: 1, CPU Core: 3",
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT \"$Latency\" FROM \"$Instrumentation\" WHERE (\"TC_Name\" =~ /^$TestCase$/ AND \"CPU_Core\" =~ /^3/ AND \"Cluster_ID\" =~ /^1/)",
+ "rawQuery": true,
+ "refId": "H",
+ "resultFormat": "time_series",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ }
+ ],
+ "thresholds": [],
+ "timeFrom": null,
+ "timeRegions": [],
+ "timeShift": null,
+ "title": "[TC: $TestCase] $Latency Graph for $Instrumentation",
+ "tooltip": {
+ "shared": true,
+ "sort": 0,
+ "value_type": "individual"
+ },
+ "transparent": true,
+ "type": "graph",
+ "xaxis": {
+ "buckets": null,
+ "mode": "time",
+ "name": null,
+ "show": true,
+ "values": []
+ },
+ "yaxes": [
+ {
+ "decimals": 3,
+ "format": "µs",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": "0",
+ "show": true
+ },
+ {
+ "format": "short",
+ "label": null,
+ "logBase": 1,
+ "max": null,
+ "min": null,
+ "show": false
+ }
+ ],
+ "yaxis": {
+ "align": false,
+ "alignLevel": null
+ }
+ },
+ {
+ "columns": [],
+ "datasource": "TFA_RunTime_Perf",
+ "fieldConfig": {
+ "defaults": {
+ "custom": {}
+ },
+ "overrides": []
+ },
+ "fontSize": "100%",
+ "gridPos": {
+ "h": 9,
+ "w": 24,
+ "x": 0,
+ "y": 9
+ },
+ "id": 7,
+ "links": [],
+ "pageSize": null,
+ "scroll": true,
+ "showHeader": true,
+ "sort": {
+ "col": 0,
+ "desc": true
+ },
+ "styles": [
+ {
+ "alias": "Time",
+ "align": "auto",
+ "dateFormat": "YYYY-MM-DD HH:mm:ss",
+ "pattern": "Time",
+ "type": "hidden"
+ },
+ {
+ "alias": "",
+ "align": "auto",
+ "colorMode": null,
+ "colors": [
+ "rgba(245, 54, 54, 0.9)",
+ "rgba(237, 129, 40, 0.89)",
+ "rgba(50, 172, 45, 0.97)"
+ ],
+ "dateFormat": "YYYY-MM-DD HH:mm:ss",
+ "decimals": 3,
+ "pattern": "/^Max*/",
+ "thresholds": [],
+ "type": "number",
+ "unit": "ns"
+ },
+ {
+ "alias": "",
+ "align": "auto",
+ "colorMode": null,
+ "colors": [
+ "rgba(245, 54, 54, 0.9)",
+ "rgba(237, 129, 40, 0.89)",
+ "rgba(50, 172, 45, 0.97)"
+ ],
+ "dateFormat": "YYYY-MM-DD HH:mm:ss",
+ "decimals": 2,
+ "link": true,
+ "linkTargetBlank": true,
+ "linkTooltip": "",
+ "linkUrl": "${__cell:raw}",
+ "pattern": "Pull_Request_URL",
+ "thresholds": [],
+ "type": "number",
+ "unit": "short"
+ },
+ {
+ "alias": "",
+ "align": "auto",
+ "colorMode": null,
+ "colors": [
+ "rgba(245, 54, 54, 0.9)",
+ "rgba(237, 129, 40, 0.89)",
+ "rgba(50, 172, 45, 0.97)"
+ ],
+ "decimals": 2,
+ "pattern": "/.*/",
+ "thresholds": [],
+ "type": "number",
+ "unit": "short"
+ }
+ ],
+ "targets": [
+ {
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT CommitID, CommitTitle as \"Commit Title\", Cluster_ID as \"Cluster ID\", MAX($Latency) AS \"Max $Latency\"\nFROM \"[[Instrumentation]]\" WHERE (TC_Name =~ /^$TestCase$/ AND Cluster_ID =~ /^0/) GROUP BY CommitID",
+ "rawQuery": true,
+ "refId": "A",
+ "resultFormat": "table",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ },
+ {
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT CommitID, CommitTitle as \"Commit Title\", Cluster_ID as \"Cluster ID\", MAX($Latency) AS \"Max $Latency\"\nFROM \"[[Instrumentation]]\" WHERE (TC_Name =~ /^$TestCase$/ AND Cluster_ID =~ /^1/) GROUP BY CommitID",
+ "rawQuery": true,
+ "refId": "B",
+ "resultFormat": "table",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ }
+ ],
+ "title": "Details for selected instrumentation and test case ([[Instrumentation]], [[TestCase]])",
+ "transform": "table",
+ "type": "table-old"
+ },
+ {
+ "columns": [],
+ "datasource": "TFA_RunTime_Perf",
+ "fieldConfig": {
+ "defaults": {
+ "custom": {}
+ },
+ "overrides": []
+ },
+ "fontSize": "110%",
+ "gridPos": {
+ "h": 3.7,
+ "w": 12,
+ "x": 0,
+ "y": 18
+ },
+ "id": 4,
+ "links": [],
+ "pageSize": null,
+ "scroll": true,
+ "showHeader": true,
+ "sort": {
+ "col": null,
+ "desc": false
+ },
+ "styles": [
+ {
+ "alias": "Time",
+ "align": "auto",
+ "dateFormat": "YYYY-MM-DD HH:mm:ss",
+ "pattern": "Time",
+ "type": "hidden"
+ },
+ {
+ "alias": "",
+ "align": "auto",
+ "colorMode": null,
+ "colors": [
+ "rgba(245, 54, 54, 0.9)",
+ "rgba(237, 129, 40, 0.89)",
+ "rgba(50, 172, 45, 0.97)"
+ ],
+ "dateFormat": "YYYY-MM-DD HH:mm:ss",
+ "decimals": 3,
+ "pattern": "/^MAX*/",
+ "thresholds": [],
+ "type": "number",
+ "unit": "ns"
+ },
+ {
+ "alias": "",
+ "align": "auto",
+ "colorMode": null,
+ "colors": [
+ "rgba(245, 54, 54, 0.9)",
+ "rgba(237, 129, 40, 0.89)",
+ "rgba(50, 172, 45, 0.97)"
+ ],
+ "decimals": 2,
+ "pattern": "/.*/",
+ "thresholds": [],
+ "type": "number",
+ "unit": "short"
+ }
+ ],
+ "targets": [
+ {
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT Cluster_ID, MAX(\"$Latency\") AS \"MAX($Latency)\" FROM \"[[Instrumentation]]\" WHERE (CommitID =~ /^$CommitID1$/ AND Cluster_ID =~ /^0/ AND \"TC_Name\" =~ /^$TestCase$/)",
+ "rawQuery": true,
+ "refId": "A",
+ "resultFormat": "table",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ },
+ {
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT Cluster_ID, MAX(\"$Latency\") AS \"MAX($Latency)\" FROM \"[[Instrumentation]]\" WHERE (CommitID =~ /^$CommitID1$/ AND Cluster_ID =~ /^1/ AND \"TC_Name\" =~ /^$TestCase$/)",
+ "rawQuery": true,
+ "refId": "B",
+ "resultFormat": "table",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ }
+ ],
+ "title": "Max $Latency for Cluster IDs 0 and 1 for ($TestCase, $CommitID1)",
+ "transform": "table",
+ "transparent": true,
+ "type": "table-old"
+ },
+ {
+ "columns": [],
+ "datasource": "TFA_RunTime_Perf",
+ "fieldConfig": {
+ "defaults": {
+ "custom": {}
+ },
+ "overrides": []
+ },
+ "fontSize": "110%",
+ "gridPos": {
+ "h": 3.7,
+ "w": 12,
+ "x": 12,
+ "y": 18
+ },
+ "id": 5,
+ "links": [],
+ "pageSize": null,
+ "scroll": true,
+ "showHeader": true,
+ "sort": {
+ "col": null,
+ "desc": false
+ },
+ "styles": [
+ {
+ "alias": "Time",
+ "align": "auto",
+ "dateFormat": "YYYY-MM-DD HH:mm:ss",
+ "pattern": "Time",
+ "type": "hidden"
+ },
+ {
+ "alias": "",
+ "align": "auto",
+ "colorMode": null,
+ "colors": [
+ "rgba(245, 54, 54, 0.9)",
+ "rgba(237, 129, 40, 0.89)",
+ "rgba(50, 172, 45, 0.97)"
+ ],
+ "dateFormat": "YYYY-MM-DD HH:mm:ss",
+ "decimals": 3,
+ "pattern": "/^MAX*/",
+ "thresholds": [],
+ "type": "number",
+ "unit": "ns"
+ },
+ {
+ "alias": "",
+ "align": "auto",
+ "colorMode": null,
+ "colors": [
+ "rgba(245, 54, 54, 0.9)",
+ "rgba(237, 129, 40, 0.89)",
+ "rgba(50, 172, 45, 0.97)"
+ ],
+ "decimals": 2,
+ "pattern": "/.*/",
+ "thresholds": [],
+ "type": "number",
+ "unit": "short"
+ }
+ ],
+ "targets": [
+ {
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT Cluster_ID, MAX(\"$Latency\") AS \"MAX($Latency)\" FROM \"[[Instrumentation]]\" WHERE (CommitID =~ /^$CommitID2$/ AND Cluster_ID =~ /^0/ AND \"TC_Name\" =~ /^$TestCase$/)",
+ "rawQuery": true,
+ "refId": "A",
+ "resultFormat": "table",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ },
+ {
+ "groupBy": [
+ {
+ "params": [
+ "$__interval"
+ ],
+ "type": "time"
+ },
+ {
+ "params": [
+ "null"
+ ],
+ "type": "fill"
+ }
+ ],
+ "orderByTime": "ASC",
+ "policy": "default",
+ "query": "SELECT Cluster_ID, MAX(\"$Latency\") AS \"MAX($Latency)\" FROM \"[[Instrumentation]]\" WHERE (CommitID =~ /^$CommitID2$/ AND Cluster_ID =~ /^1/ AND \"TC_Name\" =~ /^$TestCase$/)",
+ "rawQuery": true,
+ "refId": "B",
+ "resultFormat": "table",
+ "select": [
+ [
+ {
+ "params": [
+ "value"
+ ],
+ "type": "field"
+ },
+ {
+ "params": [],
+ "type": "mean"
+ }
+ ]
+ ],
+ "tags": []
+ }
+ ],
+ "title": "Max $Latency for Cluster IDs 0 and 1 for ($TestCase, $CommitID2)",
+ "transform": "table",
+ "transparent": true,
+ "type": "table-old"
+ }
+ ],
+ "refresh": false,
+ "schemaVersion": 26,
+ "style": "dark",
+ "tags": [
+ "TFA_QUALITY_METRICS"
+ ],
+ "templating": {
+ "list": [
+ {
+ "allValue": null,
+ "current": {
+ "selected": false,
+ "text": "juno-tftf+aarch32-rt32.instr-r2",
+ "value": "juno-tftf+aarch32-rt32.instr-r2"
+ },
+ "datasource": "TFA_RunTime_Perf",
+ "definition": "show measurements",
+ "hide": 0,
+ "includeAll": false,
+ "label": "Instrumentation",
+ "multi": false,
+ "name": "Instrumentation",
+ "options": [],
+ "query": "show measurements",
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "sort": 0,
+ "tagValuesQuery": "",
+ "tags": [],
+ "tagsQuery": "",
+ "type": "query",
+ "useTags": false
+ },
+ {
+ "allValue": null,
+ "current": {
+ "tags": [],
+ "text": "testrtinstrpsciversionparallel",
+ "value": "testrtinstrpsciversionparallel"
+ },
+ "hide": 0,
+ "includeAll": false,
+ "label": "Test Case",
+ "multi": false,
+ "name": "TestCase",
+ "options": [
+ {
+ "selected": false,
+ "text": "testrtinstrcpuoffserial",
+ "value": "testrtinstrcpuoffserial"
+ },
+ {
+ "selected": false,
+ "text": "testrtinstrcpususpparallel",
+ "value": "testrtinstrcpususpparallel"
+ },
+ {
+ "selected": false,
+ "text": "testrtinstrcpususpserial",
+ "value": "testrtinstrcpususpserial"
+ },
+ {
+ "selected": true,
+ "text": "testrtinstrpsciversionparallel",
+ "value": "testrtinstrpsciversionparallel"
+ },
+ {
+ "selected": false,
+ "text": "testrtinstrsuspdeepparallel",
+ "value": "testrtinstrsuspdeepparallel"
+ },
+ {
+ "selected": false,
+ "text": "testrtinstrsuspdeepserial",
+ "value": "testrtinstrsuspdeepserial"
+ }
+ ],
+ "query": "testrtinstrcpuoffserial,testrtinstrcpususpparallel,testrtinstrcpususpserial,testrtinstrpsciversionparallel,testrtinstrsuspdeepparallel,testrtinstrsuspdeepserial",
+ "skipUrlSync": false,
+ "type": "custom"
+ },
+ {
+ "allValue": null,
+ "current": {
+ "tags": [],
+ "text": "Latency_EL3Entry_EL3Exit",
+ "value": "Latency_EL3Entry_EL3Exit"
+ },
+ "hide": 0,
+ "includeAll": false,
+ "label": "Latency",
+ "multi": false,
+ "name": "Latency",
+ "options": [
+ {
+ "selected": false,
+ "text": "CacheFlush",
+ "value": "CacheFlush"
+ },
+ {
+ "selected": false,
+ "text": "Latency_CPUWakeup_EL3Exit",
+ "value": "Latency_CPUWakeup_EL3Exit"
+ },
+ {
+ "selected": false,
+ "text": "Latency_EL3Entry_CPUPowerDown",
+ "value": "Latency_EL3Entry_CPUPowerDown"
+ },
+ {
+ "selected": true,
+ "text": "Latency_EL3Entry_EL3Exit",
+ "value": "Latency_EL3Entry_EL3Exit"
+ }
+ ],
+ "query": "CacheFlush,Latency_CPUWakeup_EL3Exit,Latency_EL3Entry_CPUPowerDown,Latency_EL3Entry_EL3Exit",
+ "skipUrlSync": false,
+ "type": "custom"
+ },
+ {
+ "allValue": null,
+ "current": {
+ "selected": false,
+ "text": "e98d934aee",
+ "value": "e98d934aee"
+ },
+ "datasource": "TFA_RunTime_Perf",
+ "definition": "SHOW TAG VALUES WITH KEY=CommitID",
+ "hide": 0,
+ "includeAll": false,
+ "label": "Commit ID 1",
+ "multi": false,
+ "name": "CommitID1",
+ "options": [],
+ "query": "SHOW TAG VALUES WITH KEY=CommitID",
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "sort": 0,
+ "tagValuesQuery": "",
+ "tags": [],
+ "tagsQuery": "",
+ "type": "query",
+ "useTags": false
+ },
+ {
+ "allValue": null,
+ "current": {
+ "selected": false,
+ "text": "e98d934aee",
+ "value": "e98d934aee"
+ },
+ "datasource": "TFA_RunTime_Perf",
+ "definition": "SHOW TAG VALUES WITH KEY=CommitID",
+ "hide": 0,
+ "includeAll": false,
+ "label": "Commit ID 2",
+ "multi": false,
+ "name": "CommitID2",
+ "options": [],
+ "query": "SHOW TAG VALUES WITH KEY=CommitID",
+ "refresh": 1,
+ "regex": "",
+ "skipUrlSync": false,
+ "sort": 0,
+ "tagValuesQuery": "",
+ "tags": [],
+ "tagsQuery": "",
+ "type": "query",
+ "useTags": false
+ }
+ ]
+ },
+ "time": {
+ "from": "now-30d",
+ "to": "now"
+ },
+ "timepicker": {
+ "refresh_intervals": [
+ "1d"
+ ],
+ "time_options": [
+ "5m",
+ "15m",
+ "1h",
+ "6h",
+ "12h",
+ "24h",
+ "2d",
+ "7d",
+ "30d"
+ ]
+ },
+ "timezone": "",
+ "title": "TFA_RunTime_Perf_Details",
+ "uid": "qqVv390mz",
+ "version": 10
+}
\ No newline at end of file
diff --git a/quality-metrics/docs/visualisation_user_guide.md b/quality-metrics/docs/visualisation_user_guide.md
new file mode 100644
index 0000000..6233e53
--- /dev/null
+++ b/quality-metrics/docs/visualisation_user_guide.md
@@ -0,0 +1,19 @@
+# Visualisation User Guide
+Once broker component is up and running, and it is verified that data is getting pushed to InfluxDB, it can be visualised using Grafana. Following are the steps to create Grafana dashboard:
+
+1. Go to `http://[Host public IP address]:3000` and sign in using the appropriate credentials (Grafana has the following default credentials: admin, admin).
+1. Create data sources for each of the database. Set the value of data source name, database and URL to "`http://[Host public IP address]:8086`". Click on "Save & Test".
+1. In order to create dashboard, click on '+' sign on the left side bar. Select "Import" and paste the content of JSON file from [sample dashboards](./sample-dashboards) folder for the dashboard that needs to be created. In [sample dashboards](./sample-dashboards), TFA dashboards JSON files have been provided for reference. User can also create custom dashboard or modify sample ones. For details on creating dashboard, please refer [Grafana Labs](https://grafana.com/docs/grafana/latest/getting-started/getting-started/).
+
+Following table captures the details of datasource for the dashboards provided in [sample dashboards](./sample-dashboards) folder:
+
+| S. No. | Data Source Name | Database |
+| ------------- | ------------- | ------------- |
+| 1 | TFA_CodeChurn | TFA_CodeChurn |
+| 2 | TFA_Complexity | TFA_Complexity |
+| 3 | TFA_Defects | TFA_Defects |
+| 4 | TFA_ImageSize | TFA_ImageSize |
+| 5 | TFA_MisraDefects | TFA_MisraDefects |
+| 6 | TFA_RunTime_Perf | TFA_RTINSTR |
+
+Please note that URL remains same for all the data sources, i.e., `http://[Host public IP address]:8086`