From 812f73cf3827607ea8b8eb516d0673753f5438ce Mon Sep 17 00:00:00 2001 From: iNileshW <33792419+iNileshW@users.noreply.github.com> Date: Wed, 14 Jan 2026 12:09:46 +0000 Subject: [PATCH 1/7] Create Test Execution Strategy document Added a comprehensive Test Execution Strategy document covering objectives, phases, and prioritization. --- quality-assurance/test-execution | 67 ++++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) create mode 100644 quality-assurance/test-execution diff --git a/quality-assurance/test-execution b/quality-assurance/test-execution new file mode 100644 index 000000000..a8c09de76 --- /dev/null +++ b/quality-assurance/test-execution @@ -0,0 +1,67 @@ +A Test Execution Strategy defines how testing will be carried out—covering environments, prioritization, entry/exit criteria, responsibilities, tooling, reporting, and risk handling. It ensures consistency, predictability, and quality across all test cycles. +Below is a comprehensive strategy for web, legacy, cloud‑migrated, or API-based systems. + +1. Objectives of Test Execution + +- Validate functional and non-functional requirements. +- Ensure system stability across environments. +- Detect defects early through automation and continuous integration. +- Verify readiness for release. + + +2. Test Execution Phases +🔹 2.1 Preparation Phase + +- Review requirements, user stories, acceptance criteria. +- Finalize test scenarios, test cases, automation scripts. +- Document all the test cases in Azure DevOps Test Plans section with proper tagging (e.g. Smoke Tests, Regression Tests, etc.) +- Prepare test data (static, dynamic, synthetic, masked). +- Configure environments (Dev, QA, Staging, Pre-Prod). +- Validate CI/CD pipelines for automated tests (e.g., GitHub Actions, Azure DevOps). + + +🔹 2.2 Execution Phase +This involves execution of manual, automated, API, and performance tests. + Automation First Approach + - Run automation smoke suite on each deployment. + - Run regression suite nightly / on pull requests. + - Use API + UI hybrid strategy (fast feedback + UI validation). + + Manual Testing + - Run Manual Tests using Azure DevOps Test Execution Tool + - Add results for each test case + - Add & Attach Test result screenshot for test steps that prove the testcase has passed to each testcase for clear traceability + + + +🔹 2.3 Defect Management +- Log defects in Azure DevOps / Jira. +- Prioritize based on severity + business impact. +- Conduct daily triage with developers. +- Track defect aging, reopen rate, resolution time. + +🔹 2.4 Exit Phase +Release decisions based on: + - Completion of high/medium priority test cases. + - Acceptance criteria met. + - No critical/blocker defects. + - Performance thresholds met. + - Sign-off from QA + stakeholders. + +3. Test Prioritization Strategy +🟢 High Priority + - Core business flows (e.g., login, payment, workflows). + - API endpoints used by multiple services. + - Newly developed or changed features. + - High-risk modules (legacy or customer complaints). + +🟡 Medium Priority + - Secondary workflows. + - Negative test scenarios. + - Permissions, roles, access control. + +🔵 Low Priority + - Edge-case validations. + - Optional UI behaviours (alignment, colors, minor visuals). + + From 6bd7c06ed83cb41a61f0829b12a913e4e2279665 Mon Sep 17 00:00:00 2001 From: iNileshW <33792419+iNileshW@users.noreply.github.com> Date: Wed, 14 Jan 2026 12:57:40 +0000 Subject: [PATCH 2/7] Added formatting Updated the Test Execution document to enhance structure and clarity, including objectives, phases, and prioritization strategies. --- .../{test-execution => test-execution.md} | 45 ++++++++++--------- 1 file changed, 23 insertions(+), 22 deletions(-) rename quality-assurance/{test-execution => test-execution.md} (71%) diff --git a/quality-assurance/test-execution b/quality-assurance/test-execution.md similarity index 71% rename from quality-assurance/test-execution rename to quality-assurance/test-execution.md index a8c09de76..901cef755 100644 --- a/quality-assurance/test-execution +++ b/quality-assurance/test-execution.md @@ -1,7 +1,9 @@ +# Test Execution + A Test Execution Strategy defines how testing will be carried out—covering environments, prioritization, entry/exit criteria, responsibilities, tooling, reporting, and risk handling. It ensures consistency, predictability, and quality across all test cycles. Below is a comprehensive strategy for web, legacy, cloud‑migrated, or API-based systems. -1. Objectives of Test Execution +## 1. Objectives of Test Execution - Validate functional and non-functional requirements. - Ensure system stability across environments. @@ -9,8 +11,8 @@ Below is a comprehensive strategy for web, legacy, cloud‑migrated, or API-base - Verify readiness for release. -2. Test Execution Phases -🔹 2.1 Preparation Phase +## 2. Test Execution Phases +### 🔹 2.1 Preparation Phase - Review requirements, user stories, acceptance criteria. - Finalize test scenarios, test cases, automation scripts. @@ -20,27 +22,26 @@ Below is a comprehensive strategy for web, legacy, cloud‑migrated, or API-base - Validate CI/CD pipelines for automated tests (e.g., GitHub Actions, Azure DevOps). -🔹 2.2 Execution Phase +### 🔹 2.2 Execution Phase This involves execution of manual, automated, API, and performance tests. - Automation First Approach - - Run automation smoke suite on each deployment. - - Run regression suite nightly / on pull requests. - - Use API + UI hybrid strategy (fast feedback + UI validation). - - Manual Testing - - Run Manual Tests using Azure DevOps Test Execution Tool - - Add results for each test case - - Add & Attach Test result screenshot for test steps that prove the testcase has passed to each testcase for clear traceability - - - -🔹 2.3 Defect Management + ### Automation First Approach + - Run automation smoke suite on each deployment. + - Run regression suite nightly / on pull requests. + - Use API + UI hybrid strategy (fast feedback + UI validation). + + ### Manual Testing + - Run Manual Tests using Azure DevOps Test Execution Tool + - Add results for each test case + - Add & Attach Test result screenshot for test steps that prove the testcase has passed to each testcase for clear traceability + + +### 🔹 2.3 Defect Management - Log defects in Azure DevOps / Jira. - Prioritize based on severity + business impact. - Conduct daily triage with developers. - Track defect aging, reopen rate, resolution time. -🔹 2.4 Exit Phase +### 🔹 2.4 Exit Phase Release decisions based on: - Completion of high/medium priority test cases. - Acceptance criteria met. @@ -48,19 +49,19 @@ Release decisions based on: - Performance thresholds met. - Sign-off from QA + stakeholders. -3. Test Prioritization Strategy -🟢 High Priority +## 3. Test Prioritization Strategy +### 🟢 High Priority - Core business flows (e.g., login, payment, workflows). - API endpoints used by multiple services. - Newly developed or changed features. - High-risk modules (legacy or customer complaints). -🟡 Medium Priority +### 🟡 Medium Priority - Secondary workflows. - Negative test scenarios. - Permissions, roles, access control. -🔵 Low Priority +### 🔵 Low Priority - Edge-case validations. - Optional UI behaviours (alignment, colors, minor visuals). From dc684bc4d29d075b6875cb2ced641ca78cc1fc84 Mon Sep 17 00:00:00 2001 From: iNileshW <33792419+iNileshW@users.noreply.github.com> Date: Wed, 21 Jan 2026 16:39:28 +0000 Subject: [PATCH 3/7] Update defect management logging instructions Removed reference to Jira for defect logging. --- quality-assurance/test-execution.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/quality-assurance/test-execution.md b/quality-assurance/test-execution.md index 901cef755..7d681cd97 100644 --- a/quality-assurance/test-execution.md +++ b/quality-assurance/test-execution.md @@ -36,7 +36,7 @@ This involves execution of manual, automated, API, and performance tests. ### 🔹 2.3 Defect Management -- Log defects in Azure DevOps / Jira. +- Log defects in Azure DevOps. - Prioritize based on severity + business impact. - Conduct daily triage with developers. - Track defect aging, reopen rate, resolution time. From c63fb0b1c54242e332c0ca0fe2f354f99f9628e2 Mon Sep 17 00:00:00 2001 From: iNileshW <33792419+iNileshW@users.noreply.github.com> Date: Thu, 7 May 2026 17:17:13 +0100 Subject: [PATCH 4/7] Update API contract section in test strategy Removed mention of PACT technology in API contract section. --- quality-assurance/test-strategy.md | 1 - 1 file changed, 1 deletion(-) diff --git a/quality-assurance/test-strategy.md b/quality-assurance/test-strategy.md index 7a9f727d0..907a14366 100644 --- a/quality-assurance/test-strategy.md +++ b/quality-assurance/test-strategy.md @@ -55,7 +55,6 @@ This section outlines the test types that need to be considered when preparing a * Contract testing should be considered when developing an API that will communicate with another UKHO API. * When interacting with the Data Platform this is mandatory. -* [PACT](https://docs.pact.io/) is the technology to use. * Use the [UKHO Data Platform PACT documentation](https://docs.data.ukho.gov.uk/testing/pact/introduction/) to learn how to do this. ### Cross Browser From eed0ae10f6c3ca7c86a44bd254dd3d523b1eca80 Mon Sep 17 00:00:00 2001 From: iNileshW <33792419+iNileshW@users.noreply.github.com> Date: Thu, 7 May 2026 17:23:54 +0100 Subject: [PATCH 5/7] Removed broken link for safety standard reference --- quality-assurance/test-strategy.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/quality-assurance/test-strategy.md b/quality-assurance/test-strategy.md index 907a14366..182fe4287 100644 --- a/quality-assurance/test-strategy.md +++ b/quality-assurance/test-strategy.md @@ -118,7 +118,7 @@ Tests should be added at the correct level of the test pyramid which directly in ### Safety * Safety assurance is part of our core software engineering processes (ensuring our products are safe for the end user). -* We are working to adopt [BS EN 61508-3:2010](https://fdocuments.in/document/iec-61508-6.html) as our safety standard. +* We are working to adopt [BS EN 61508-3:2010] as our safety standard. * Safety is the responsibility of the whole team and is championed by the Test Engineer. * More details can be found on on the [safety assurance](safety-assurance-guidance.md) page. From 125c88a1dd10125d1f4459933302947d27d5ddb2 Mon Sep 17 00:00:00 2001 From: iNileshW <33792419+iNileshW@users.noreply.github.com> Date: Thu, 7 May 2026 17:25:24 +0100 Subject: [PATCH 6/7] Delete quality-assurance/test-execution.md --- quality-assurance/test-execution.md | 68 ----------------------------- 1 file changed, 68 deletions(-) delete mode 100644 quality-assurance/test-execution.md diff --git a/quality-assurance/test-execution.md b/quality-assurance/test-execution.md deleted file mode 100644 index 7d681cd97..000000000 --- a/quality-assurance/test-execution.md +++ /dev/null @@ -1,68 +0,0 @@ -# Test Execution - -A Test Execution Strategy defines how testing will be carried out—covering environments, prioritization, entry/exit criteria, responsibilities, tooling, reporting, and risk handling. It ensures consistency, predictability, and quality across all test cycles. -Below is a comprehensive strategy for web, legacy, cloud‑migrated, or API-based systems. - -## 1. Objectives of Test Execution - -- Validate functional and non-functional requirements. -- Ensure system stability across environments. -- Detect defects early through automation and continuous integration. -- Verify readiness for release. - - -## 2. Test Execution Phases -### 🔹 2.1 Preparation Phase - -- Review requirements, user stories, acceptance criteria. -- Finalize test scenarios, test cases, automation scripts. -- Document all the test cases in Azure DevOps Test Plans section with proper tagging (e.g. Smoke Tests, Regression Tests, etc.) -- Prepare test data (static, dynamic, synthetic, masked). -- Configure environments (Dev, QA, Staging, Pre-Prod). -- Validate CI/CD pipelines for automated tests (e.g., GitHub Actions, Azure DevOps). - - -### 🔹 2.2 Execution Phase -This involves execution of manual, automated, API, and performance tests. - ### Automation First Approach - - Run automation smoke suite on each deployment. - - Run regression suite nightly / on pull requests. - - Use API + UI hybrid strategy (fast feedback + UI validation). - - ### Manual Testing - - Run Manual Tests using Azure DevOps Test Execution Tool - - Add results for each test case - - Add & Attach Test result screenshot for test steps that prove the testcase has passed to each testcase for clear traceability - - -### 🔹 2.3 Defect Management -- Log defects in Azure DevOps. -- Prioritize based on severity + business impact. -- Conduct daily triage with developers. -- Track defect aging, reopen rate, resolution time. - -### 🔹 2.4 Exit Phase -Release decisions based on: - - Completion of high/medium priority test cases. - - Acceptance criteria met. - - No critical/blocker defects. - - Performance thresholds met. - - Sign-off from QA + stakeholders. - -## 3. Test Prioritization Strategy -### 🟢 High Priority - - Core business flows (e.g., login, payment, workflows). - - API endpoints used by multiple services. - - Newly developed or changed features. - - High-risk modules (legacy or customer complaints). - -### 🟡 Medium Priority - - Secondary workflows. - - Negative test scenarios. - - Permissions, roles, access control. - -### 🔵 Low Priority - - Edge-case validations. - - Optional UI behaviours (alignment, colors, minor visuals). - - From af0d24c487386e6a3ae65b8b8ec953a4299a8490 Mon Sep 17 00:00:00 2001 From: iNileshW <33792419+iNileshW@users.noreply.github.com> Date: Mon, 11 May 2026 11:32:14 +0100 Subject: [PATCH 7/7] Removed reducdant PACT Link & BS EN Standard PACT link referred to broken link document BS EN Link removed as redundant & not working --- quality-assurance/test-strategy.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/quality-assurance/test-strategy.md b/quality-assurance/test-strategy.md index 182fe4287..5aa2a1683 100644 --- a/quality-assurance/test-strategy.md +++ b/quality-assurance/test-strategy.md @@ -55,7 +55,8 @@ This section outlines the test types that need to be considered when preparing a * Contract testing should be considered when developing an API that will communicate with another UKHO API. * When interacting with the Data Platform this is mandatory. -* Use the [UKHO Data Platform PACT documentation](https://docs.data.ukho.gov.uk/testing/pact/introduction/) to learn how to do this. +* [PACT](https://docs.pact.io/) is the technology to use. + ### Cross Browser @@ -118,7 +119,6 @@ Tests should be added at the correct level of the test pyramid which directly in ### Safety * Safety assurance is part of our core software engineering processes (ensuring our products are safe for the end user). -* We are working to adopt [BS EN 61508-3:2010] as our safety standard. * Safety is the responsibility of the whole team and is championed by the Test Engineer. * More details can be found on on the [safety assurance](safety-assurance-guidance.md) page.