From d7fdb1bf70f3d5d14d99f1e18ab5f0db4a057abb Mon Sep 17 00:00:00 2001 From: GetmanetsIrina Date: Tue, 7 Nov 2017 12:44:22 +0200 Subject: [PATCH 1/4] Proposal "Test Reports Folder structure" --- ...hange_ATF_test_reports_folder_structure.md | 55 +++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 proposals/NNNN-Change_ATF_test_reports_folder_structure.md diff --git a/proposals/NNNN-Change_ATF_test_reports_folder_structure.md b/proposals/NNNN-Change_ATF_test_reports_folder_structure.md new file mode 100644 index 0000000000..8e7ea69888 --- /dev/null +++ b/proposals/NNNN-Change_ATF_test_reports_folder_structure.md @@ -0,0 +1,55 @@ +# Change ATF test reports folder structure + +* Proposal: [SDL-NNNN](NNNN-Change_ATF_test_reports_folder_structure.md) +* Author: [Irina Getmanets](https://github.com/GetmanetsIrina) +* Status: **Awaiting review** +* Impacted Platforms: ATF + +## Introduction + +For now ATF reports folder has inconvenient nested structure. ATF creates structure of folders from relative path to script, e.g. runned script fom test_scripts/API/Navidation/Subscriptions/. +With current implementation will be created the next structure +

+TestingReports
+  ->ATF_timestamp
+    ->test_scripts
+      ->API
+      	->Navigation
+	  ->Subscriptions
+  ->SDL_timestamp
+    ->test_scripts
+      ->API
+      	->Navigation
+	  ->Subscriptions
+  ->Reports_timestamp
+    ->test_scripts
+      ->API
+      	->Navigation
+	  ->Subscriptions
+
+ +## Motivation + +Create more convenient reports structure, less nested. + +## Proposed solution + +The solution is to create new structure: +
TestingReports
+	-> ScriptName_timestamp
+		-> ATF
+		-> SDL
+Subfolder ATF will contain ATF logs and report. +Subfolder SDL will contain SDL logs. + +## Potential downsides + +n/a + +## Impact on existing code + +Impact on ATF reporting functionality. + +## Alternatives considered + +n/a From 14e20e522aa7592aebf356eec293d24797f7802e Mon Sep 17 00:00:00 2001 From: "Getmanets Irina (GitHub)" Date: Mon, 13 Nov 2017 08:59:02 +0200 Subject: [PATCH 2/4] Update according to review comment. --- ...hange_ATF_test_reports_folder_structure.md | 40 +++++++------------ 1 file changed, 15 insertions(+), 25 deletions(-) diff --git a/proposals/NNNN-Change_ATF_test_reports_folder_structure.md b/proposals/NNNN-Change_ATF_test_reports_folder_structure.md index 8e7ea69888..1dcbf7d1cd 100644 --- a/proposals/NNNN-Change_ATF_test_reports_folder_structure.md +++ b/proposals/NNNN-Change_ATF_test_reports_folder_structure.md @@ -7,38 +7,28 @@ ## Introduction -For now ATF reports folder has inconvenient nested structure. ATF creates structure of folders from relative path to script, e.g. runned script fom test_scripts/API/Navidation/Subscriptions/. -With current implementation will be created the next structure -

-TestingReports
-  ->ATF_timestamp
-    ->test_scripts
-      ->API
-      	->Navigation
-	  ->Subscriptions
-  ->SDL_timestamp
-    ->test_scripts
-      ->API
-      	->Navigation
-	  ->Subscriptions
-  ->Reports_timestamp
-    ->test_scripts
-      ->API
-      	->Navigation
-	  ->Subscriptions
-
+Main output of ATF is test reports. +ATF provide such reports: +* Console logs +* SDL logs +* Transport logs +* Detailed report + +This proposal is about creating clear and useful structure of test scripts reports. ## Motivation -Create more convenient reports structure, less nested. +Create more convenient reports structure, less nested. Multiple running of test script should not override old reports. Structure of reports should be clear for easy searching certain report. ## Proposed solution The solution is to create new structure: -
TestingReports
-	-> ScriptName_timestamp
+```
+TestingReports
+	-> ScriptName_YYYYMMDDHHMMSS
 		-> ATF
-		-> SDL
+ -> SDL +``` Subfolder ATF will contain ATF logs and report. Subfolder SDL will contain SDL logs. @@ -52,4 +42,4 @@ Impact on ATF reporting functionality. ## Alternatives considered -n/a +Leave as is. From 418ed2b404a10233f52826eb720052896cf44518 Mon Sep 17 00:00:00 2001 From: "Getmanets Irina (GitHub)" Date: Tue, 28 Nov 2017 11:11:29 +0200 Subject: [PATCH 3/4] Update NNNN-Change_ATF_test_reports_folder_structure.md --- .../NNNN-Change_ATF_test_reports_folder_structure.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/NNNN-Change_ATF_test_reports_folder_structure.md b/proposals/NNNN-Change_ATF_test_reports_folder_structure.md index 1dcbf7d1cd..0be2f8893e 100644 --- a/proposals/NNNN-Change_ATF_test_reports_folder_structure.md +++ b/proposals/NNNN-Change_ATF_test_reports_folder_structure.md @@ -9,10 +9,10 @@ Main output of ATF is test reports. ATF provide such reports: -* Console logs -* SDL logs -* Transport logs -* Detailed report +* Console logs - name of test cases with execution status FAILED/PASSED. +* SDL logs - SDL log received by ATF via telnet logger. +* Transport logs - all received and sent data from/to SDL. Transport log can have default view - messages on protocol level (except binaries) and full view - log will be expanded with packages: streamings, service messages, heartbeat, json files. +* Detailed report - all expected and sent massages related to test cases with expected and actual result.Also contains info about test cases: name, time, sequence, status, duration. This proposal is about creating clear and useful structure of test scripts reports. From 397541b6fb295b86ba1b3bf317b0a5035ff009cd Mon Sep 17 00:00:00 2001 From: "Getmanets Irina (GitHub)" Date: Mon, 11 Dec 2017 09:32:33 +0200 Subject: [PATCH 4/4] Updated Motivation section. Updated Motivation section according to review comment. --- proposals/NNNN-Change_ATF_test_reports_folder_structure.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/NNNN-Change_ATF_test_reports_folder_structure.md b/proposals/NNNN-Change_ATF_test_reports_folder_structure.md index 0be2f8893e..4b5fd8ece5 100644 --- a/proposals/NNNN-Change_ATF_test_reports_folder_structure.md +++ b/proposals/NNNN-Change_ATF_test_reports_folder_structure.md @@ -18,7 +18,7 @@ This proposal is about creating clear and useful structure of test scripts repor ## Motivation -Create more convenient reports structure, less nested. Multiple running of test script should not override old reports. Structure of reports should be clear for easy searching certain report. +Create more convenient reports structure, less nested. Running multiple test scripts should not override old reports. Structure of reports should be clear, so you can easily search for a certain report. ## Proposed solution