How to Execute Test Cases in Azure DevOps

In this comprehensive guide, I will walk you through exactly how to execute test cases in Azure DevOps. I will cover everything from setting up your initial test plans to running manual tests and even touching upon the workflow for automated execution. My goal is to help you master this environment.

How to Execute Test Cases in Azure DevOps

The Hierarchy of Artifacts

In Azure DevOps, testing artifacts are structured logically to support traceability. Here is how:

1. Test Plan: The root container. This usually corresponds to a Sprint (e.g., “Sprint 24 Plan”) or a specific Release (e.g., “Q3 Release Candidate”).
2. Test Suite: These are folders within the plan. They can be static (custom grouping), requirement-based (linked to User Stories), or query-based (dynamic lists based on criteria).
3. Test Case: The actual script containing steps, expected results, and test data.

Prerequisites: Setting the Stage for Execution

You cannot execute what doesn’t exist. Before clicking “Run,” I always ensure my test repository is pristine. Here is the checklist for my projects:

• Access Levels: Ensure your team members have “Basic + Test Plans” access level or a Visual Studio Enterprise subscription. Basic access alone will not let you execute tests in the Test Plans module.
• Test Plan Creation: I always name my test plans clearly. For example, “Project Phoenix – Sprint 5.”
• Test Case Readiness: All test cases must be in the “Ready” state. I never let my team execute a test case that is still in “Design” because it skews our metrics.

Table: Essential Permissions for Test Execution

Step-by-Step Guide: Executing Manual Test Cases

This is the bread and butter of our daily work. While automation is critical, manual testing remains essential for exploratory checks and UI/UX validation. Here is the exact workflow.

1. Navigating to the Test Runner

First, log in to your Azure DevOps organization and navigate to your project.

1. Click on Test Plans in the left-hand navigation bar.
2. Select Test Plans again from the sub-menu.
3. Select the specific Test Plan you are working on (e.g., “Sprint 12”).
4. Click on the Execute tab. This is where the magic happens. You will see a list of your test points (pairings of test cases and configurations).

2. Launching the Runner

Highlight the test case you want to run. You have two primary options here:

• Run for web application: This launches the web-based runner.
• Run for desktop application: This launches the separate client runner (useful for non-web apps).

For 90% of my web-based projects, I use the Run for web application option.

• Select the test case.
• Click the Run button (or right-click and select Run with options if you need to choose a specific build or configuration). Check out the screenshot below for your reference.

3. The Execution Phase

Once the pop-up window (The Test Runner) opens, the system will dock to the side of your browser. This allows you to see your application on the main screen while having your test steps visible on the side. Check out the screenshot below for your reference.

My Workflow During Execution:

• Step-by-Step Verification: I go through each step carefully. As I complete an action, I click the Green Checkmark next to the step to mark it as passed.
• Recording Evidence: If a step is critical, I use the camera icon within the runner to take a screenshot or record a video clip. This is invaluable when proving to developers like “Mike” or “Sarah” that the bug actually exists.
• Adding Comments: If a step passes but behaves oddly (latency, minor cosmetic glitch), I add a comment directly to the step without failing it.

4. Handling Failures and Logging Bugs

When a step fails, do not just mark it red and move on. This is where you add value.

1. Click the Red X icon next to the failed step.
2. The system will prompt you to enter a comment. Be descriptive. Instead of saying “It failed,” write “Expected User ID 1234, but received Error 500.”
3. Create a Bug Immediately: Click the Create Bug icon within the runner.
   • Why this is awesome: Azure DevOps automatically populates the bug with the “Repro Steps”—it copies all the steps you just executed, including the one that failed. This saves me hours of typing every week.
4. Title the bug (e.g., “Checkout: Error 500 when processing Visa card”).
5. Save the bug and close the bug window. It is now automatically linked to the test result.

5. Finalizing the Run

Once all steps are marked (Pass, Fail, or Blocked), click Save and Close at the top of the runner. The status on the dashboard will update instantly, turning that row Green or Red.

Executing Automated Test Cases

While manual execution is straightforward, automated execution in Azure DevOps requires a shift in mindset. You don’t “run” these manually in the same way; you orchestrate them.

Linking Automation to Test Cases

I often have junior engineers ask, “How do I see my Selenium results in the Test Plans tab?” The secret is Association.

1. Visual Studio Integration: You must associate your automated test method (written in C#, Python, etc.) with the Work Item ID of the Test Case in Azure DevOps.
2. The Build Pipeline: You typically execute these tests as part of a Build Pipeline or Release Pipeline.

The Pipeline Workflow

Here is the high-level process I oversee for my automation architects:

• Define the Pipeline: In the Azure Pipelines section, we add a “Visual Studio Test” task.
• Select Test Mix: We configure the task to run tests based on the “Test Plan” filter rather than just running all assemblies.
• Execution: When the pipeline runs (e.g., nightly build), it executes the code.
• Reporting: The most important part—Azure DevOps publishes the results back to the Test Plans module.

Best Practices for Efficient Test Execution

1. Use Configurations Wisely

Do not create separate test cases for “Chrome,” “Firefox,” and “Edge.” Use Test Configurations.

• I create a single test case (e.g., “Login Functionality”).
• I assign three configurations to it.
• Azure DevOps automatically generates three “Test Points” for execution. This keeps your repository clean and reduces maintenance overhead.

2. The “Block” Status

Teach your team the difference between “Failed” and “Blocked.”

• Failed: The functionality is broken.
• Blocked: We cannot run the test because a prerequisite (like the test environment being down) is preventing us.
• Why it matters: When I present metrics to stakeholders, “Blocked” tells a story of infrastructure issues, whereas “Failed” tells a story of code quality.

3. Bulk Marking

Sometimes, you have a suite of 50 tests that are simple checks, and you verified them via an exploratory session. You don’t need to open the runner for every single one.

• Select multiple test cases in the grid view.
• Right-click and select Mark Outcome > Pass.
• Warning: Use this sparingly. I only allow my senior QAs (like my lead, David) to do this to avoid “false greens.”

4. Parameterization

I love this feature. Instead of writing “Enter ‘John'”, “Enter ‘Jane'”, “Enter ‘Bob'” as three separate test cases, use Parameters.

• Write the step as: “Enter @Name in the username field.”
• In the parameter table below the steps, add rows for John, Jane, and Bob.
• The runner will iterate through these rows as a single execution flow.

Reporting and Metrics

Execution is useless if you cannot track progress. Azure DevOps provides incredible widgets for this.

The Charts I Always Pin to My Dashboard:

• Test Execution Trend: Shows the burn-down of testing activity over the Sprint.
• Requirement Coverage: Shows which User Stories have passed tests and which have open bugs.
• Bug Trend by Priority: This helps me prioritize what the developers need to fix first.

Troubleshooting Common Execution Issues

Even with a tool as robust as Azure DevOps, things go wrong. Here are frequent issues I have encountered and how to fix them.

• “I can’t see the Run button.”
  • Fix: This is almost always a permissions issue. Check if your license is “Basic” instead of “Basic + Test Plans.” You need the paid extension or Visual Studio Enterprise.
• “The Runner disconnects.”
  • Fix: If you are using the desktop runner, ensure your version of the “Test Runner” client is up to date. For the web runner, check your browser pop-up blocker; it often silently kills the runner window.
• “Bugs aren’t linking automatically.”
  • Fix: Ensure you are clicking “Create Bug” inside the runner’s toolbar, not opening a new tab to create a bug separately. The context link happens only from within the active session.

Conclusion

Executing test cases in Azure DevOps requires a blend of technical know-how and process discipline. It is not just about clicking “Pass” or “Fail”; it is about building a traceability web that links your requirements to your code and your defects.

By organizing your Test Plans effectively, utilizing the runner’s rich data-capture features, and integrating automation where possible, you transform QA from a bottleneck into a quality accelerator.

Now, go log into your Azure DevOps project, check your permissions, and start executing with confidence.

• How to Write Manual Test Cases in Azure DevOps
• How to Create Pull Request in Azure DevOps
• How to Resolve Merge Conflicts in Azure DevOps

Rajkishore

I am Rajkishore, and I am a Microsoft Certified IT Consultant. I have over 14 years of experience in Microsoft Azure and AWS, with good experience in Azure Functions, Storage, Virtual Machines, Logic Apps, PowerShell Commands, CLI Commands, Machine Learning, AI, Azure Cognitive Services, DevOps, etc. Not only that, I do have good real-time experience in designing and developing cloud-native data integrations on Azure or AWS, etc. I hope you will learn from these practical Azure tutorials. Read more.