Automated and Manual Testing with Zephyr Enterprise

Thursday, August 18, 2022

Comprehensive software testing has never been more important than it is today.

Software will continue to play an essential role in our daily lives. What is changing – and will continue to change – is what we test and how we test it.

Modern software codebases are immensely complex. It’s simply not feasible to execute all of the necessary test cases manually. The bulk of testing today is automated. The general trend toward building more comprehensive and sophisticated test automation won’t change any time soon.

But if you’re a manual tester worried about automated testing taking over, don’t! Sure, automation can test many features and environments quickly and in parallel, but it’s limited to the scripts already written. The time and resources required to create those scripts is not insignificant. For products with rapid or frequent feature releases, it’s often more cost- and time-efficient to test manually.

And when it comes to testing for bad UI or accessibility compliance, there’s no substitute for a real person. Yes AI tools exist, but the amount of babysitting needed to use them offsets the benefits.

No matter how robust your automation testing, there will always be a need for human testing.

This is why many companies use a combination of manual and automated testing frameworks. When done right, the methods are complimentary. I believe the corporate buzzword for this is synergy.

But that’s not how most folks would describe working on teams that use a combination of automated and manual testing. For one, it’s impossible to say synergy without sounding like a dork. And the reality for most teams is that things could be a lot better.

Automation teams often use different tools than manual teams. You might see manual test cases stored in spreadsheets, or (hopefully) a test management tool that’s separate from the one storing automated test cases. There will be lots of duplicated tests from both teams, or gaps in coverage because one team assumed the other had a test case covered. Oh, and of course, all the actual test requirements are kept in another separate tool (probably Jira), managed by the dev team.

In short: different teams + different tools + difficult communication paths = information silos, difficulty reporting progress to stakeholders, and general dysfunction. Yet many teams soldier on and make it work as best they can.

If you’re on one of these teams, I sympathize. I’ve been on a few myself.

If you’re managing one of these teams, a word of advice: start working on a plan to transition your automated and manual testing teams to a test management tool that supports both workflows natively.

Don’t put the cart before the horse

A test management tool, like any other tool, is a means to an end, not an end in and of itself.

In the physical world, we’re pretty good at not losing sight of this. Nobody leaves a paintbrush and a can of paint in a room expecting that they’ll come back a few hours later to find freshly-painted walls. But in the context of software – especially complex enterprise software – it’s easy to get distracted by the details and lose sight of the obvious: even the best test management software won’t solve your testing woes on its own.

You still need teamwork, good communication, and clearly documented processes.

In general, successful teams will:

  • Have regular team syncs or standups. This ensures the whole testing team is working together towards a common goal of software quality. For example, they plan for automation coverage over certain areas, freeing manual testers to perform more exploratory testing. Or they determine timelines for testing, since manual testing often takes longer to complete than automated testing.
  • Define best practices for your test management. Don’t sleep on the importance of consistent naming conventions for test cases, test cycles, and how/where defects are created and tracked. This is an ongoing battle, but when you create reports and analyze test data across teams, you’ll appreciate the consistency.
  • Put data to good use. Bringing automated and manual testing together generates a lot of data. Make sure you have a plan to examine the data and derive insights to refactor your testing.

If you can bring all of that to the table, read on to learn what benefits a modern test management tool will bring you.

Automated & manual test management with one tool

As a former Test Engineer and current Customer Adoption Engineer for Zephyr Enterprise, I’ve used many test management tools and worked with (or on) my fair share of testing teams. I won’t claim to have seen everything, but I’ve seen enough to know what works and what doesn’t.

For large enterprise teams looking to bring automated and manual testing together, there are only a handful of practical options. Ultimately, it’s up to you to evaluate your options and decide which is best, but the most important questions are:

  • How will you ensure the automated + manual tests in a given test cycle are giving full coverage for all the requirements specified in your project management tool?
  • Can you view a breakdown of all automated and manual tests and the requirements each test is associated with?
  • When are automated tests scheduled to run? How do you add, remove, or otherwise change testing mid-cycle?
  • How are test results reported and traced back to specific requirements?

You’ll need to add context-specific questions to this list, but these are a great starting point for any enterprise-level team.

If you’re expecting me to give you answers about how Zephyr Enterprise tackles the above issues, I’ve got news for you: that’s exactly what comes next.

How Zephyr Enterprise ensures full test coverage by synchronizing with Jira and other project management tools

With Zephyr Enterprise, you can ensure your requirements are being tested by synchronizing directly from Jira, and mapping both automated and manual testing back to those requirements. Insight into your testing activity is then highlighted on your Jira stories, so all stakeholders are informed.

If you’re not using Jira, you can still ensure requirement coverage. Do this either by creating requirements directly in Zephyr, or importing them from another project management tool.


How Zephyr Enterprise manages your repository of automated and manual test cases

You can store both your automated and manual test cases in one tool with Zephyr. The test repository lets you store all your test cases in one project. This gives all testers visibility and improved collaboration.

Manual and automated test cases are distinguished by either M or A on the test cases. When planning test cycles, you can run both manual and automated test cases in a single cycle. Alternatively, you can run an entire automation suite scheduled by vortex.


How Zephyr Enterprise schedules and executes automated tests

With the use of Zbots in Zephyr, anyone on your test team (provided they have the requisite permissions) can schedule and execute automated tests. Zephyr gives you the ability to kick off entire automation suites or run scripts ad hoc.

If you have automation running in the background or on a specific schedule, take advantage of the folder watcher feature. When a folder is ‘watched’ by Zephyr, new test results added to that folder are automatically imported.


How does Zephyr Enterprise handle reporting and traceability?

Testers see results of all testing done on the Test cycle page or through the use of dashboards.

Since test cases are mapped directly back to Jira issues, all traceability can be seen directly on the Jira issues. This gives stakeholders a better idea of how their sprints and deliverables are progressing.