Test cases from requirements specifications and use cases

A while ago I answered the following question on SearchSoftwareQuality.com’s Ask The Software Quality Expert: Questions & Answers.

Assuming both requirements specifications and use cases are available, which one would be more appropriate to use for creating test cases and, further, to be traced to from the test cases?

Here is a clip from my answer:

As for traceability, if that's a requirement, the best person to ask is the auditor who will be ensuring compliance. If you're working in an environment where traceability is important, then I would assume you would want to trace your test cases back to both sets of artifacts, but the best way to be sure is to ask.

You can find the full posting here.

Comments

[Shrini]

Mike --

My answer would be somewhat on following lines ....

In traditional sense, requirements lead to use cases. Hence use cases are in a way, a revised form of specifications. Assuming that business analysts formulate use cases out of specification and articulate in a language that close the the business usage of the underlying application, use cases are expected to be specifications with additional information.

Coming to which one to use for test design ... I would say both and most importantly a holistic view of specifications + use cases ( 1+1 < 2)is needed.

My view on traceability is always very skeptical and low. People who swear by this, tend to treat requirements and test cases as real/quantifiable entities. They tend to count them and apply all sorts of mathematical operations.

I would oppose the notion of traceability (or any variant) as it would trivialize complex entities like requirements and test cases as though they were real objects.

What do you say?

Shrini

[Mike Kelly]

Hey Shrini,

In the past, I've found that traceability can be helpful. It's one of many ways we can look at coverage. If nothing else, it shows me what we have tests for and what we don't. Keep in mind when I say tests, I also mean charters. I'm not talking about something overly formal.

I will agree with you that many test managers and project managers I've worked with have put too much stock in traceability. They believe it's giving them something it's not, and that's complete coverage. It's only one type of coverage.

I view it much like I view code coverage. The number isn't that interesting to me, other than as an indicator of overall effort or completeness. Just as I would be nervous if after all me testing my code coverage was only 40%, I'd also be nervous if my requirements coverage were only 40%.

That does not mean there aren't contexts where either of those might be ok, there are. It is to say that those would be triggers for me to dig in and make sure that what I think is happening with my testing is really the case.

The cautionary tale, again much like with code coverage, is that you can't assume that just because you may have covered all of the requirements, that you're in any way done. The amount of coverage doesn't speak to the quality of the coverage. Which I think is the point you were making.

[Shini]

Mike,

The bigger question or concern I have is the principle behind so called "Traceability".

The story goes like this. I have 10 black color balls (called as requirements) and I will then get 30 red colored balls. I will now set a mapping between black balls to red balls in such way that every black ball gets 3 red balls each. The concept of traceability also goes in some what similar lines - Every requirement shall have at least 3 test cases associated with it. Anything less would lead to poor traceability.

In reality, requirements and test cases are not real objects like colored balls. They are "abstract" ideas expressed by some one at some point of time and documented by some one else (typically) at some other instance of time. To model requirements and test cases as "entities" that could be counted is the fundamental "flaw" in the notion of traceability. I am taking an extreme view of this and middle way is somewhere that you appear to advocate.

Coverage Quality vs Coverage Quantity debate becomes very distinct as we start counting things that do no lend themselves for such "simplification"

Shrini

While I agree that like code coverage, test coverage as manifested through traceability is ONE of the ideas for measuring a bigger picture of "coverage" in general - I strongly feel that idea of traceability simplifies the notion of coverage through test cases beyond the limits causing more harm than good.

[Mike Kelly]

Shrini,

I don't disagree with you that many people simplify it the way you describe, but I'm not one of them. I'm not saying traceability means three test cases to each requirement. I'm not even advocating 100% traceability - even that depends on what the requirements look like, how much time you have, and what other types of coverage you're looking at. However, I'm not going to not use a helpful tool just because other people misuse it.

If I don't know how to properly use a hammer, and I'm always striking with the side of the hammer instead of the head, that doesn't mean you shouldn't use the hammer the proper way. It just means I don't know how to use a hammer. You should still use it if you need a hammer.

Requirements coverage is difficult. I totally agree. But because other people don't know how to do their job, is not a reason for me to not use all the tools I have at hand to do mine.

-Mike