On many projects, you aren't the subject matter expert and you don't write the code. In those cases it can be helpful to talk with the subject matter experts and programmers to better understand the subtleties of the software. I often find it helpful to get them to help me design different types of tests. To help pull out the information I need, I use questions.

A couple of questions (or theme of questions) are "At what level do you interact with the various algorithms? Do you test one algorithm at a time or many algorithms at a time? Or do you know?" This can help you understand the level of complexity in their testing in terms of visibility into the specific algorithms. You can better understand if they are testing one factor at a time, or many factors at a time. Knowing this can help you understand where they might perceive interactions between variables that might be of interest for your testing. Or it might even clue you into some dependencies you didn't previously know about.

It also might just get you a blank stare.... "Algor-what?"

Based on how they answer, you might come up with a lot more questions on that theme.

On many projects, you aren't the subject matter expert and you don't write the code. In those cases it can be helpful to talk with the subject matter experts and programmers to better understand the subtleties of the software. I often find it helpful to get them to help me design different types of tests. To help pull out the information I need, I use questions.

A couple of questions (or theme of questions) are "How would you know if a calculation was correct? How do you check your answers?" For complex systems where logic or calculations might be hidden from you, this can give you an idea of what oracles they might use when they test. It might also give you insight into how much checking they are actually doing. When they test, are they saying "Yea, that looks close enough." Or are they really digging in to make sure something is correct?

Based on how they answer, you might come up with a lot more questions on that theme.

On many projects, you aren't the subject matter expert and you don't write the code. In those cases it can be helpful to talk with the subject matter experts and programmers to better understand the subtleties of the software. I often find it helpful to get them to help me design different types of tests. To help pull out the information I need, I use questions.

One question is "Can you walk me through how you design your test cases?" You should listen for what documents they look at, what data they reference, and who they talk to. By understanding the inputs they use to design their test cases, you can better inform your own test case design process. I also try to listen for techniques they use (or don't use) and if possible might even try to review some of their tests. Not only am I interested in how they design their tests, but I'm also interested in making sure I understand what type of tests they will be running. I may or may not what to duplicate those efforts, but in either cases it helps me understand where I might be able to best augment the testing they will be doing.

Based on how they answer, you might come up with a lot more questions on that theme.

On many projects, you aren't the subject matter expert and you don't write the code. In those cases it can be helpful to talk with the subject matter experts and programmers to better understand the subtleties of the software. I often find it helpful to get them to help me design different types of tests. To help pull out the information I need, I use questions.

One question is "Are there things that you don't have time to test that you wish you did?" Here you're searching for specific things they might be worried about with the implementation. It's another question focused on understanding risk (see yesterday's post)." With this question you'll likely hear about issues related to the software being developed, the technology being used, the environment, or possibly even the methodology being used and how issues related to how the work is being done are manifesting in the code or team.

Based on how they answer, you might come up with a lot more questions on that theme.

On many projects, you aren't the subject matter expert and you don't write the code. In those cases it can be helpful to talk with the subject matter experts and programmers to better understand the subtleties of the software. I often find it helpful to get them to help me design different types of tests. To help pull out the information I need, I use questions.

One question is "Are there specific tests you would like me to run for regression purposes?" This question might tell you about their concept of risk. What are they worried about breaking? Where do they think the important logic is that exists today? Or what ares of the application do they feel is fragile? While this question may not apply to project where there aren't legacy assets, I like it because it gets people to think about some of the implications of the changes being made. Often times, we get into tunnel vision of what we're doing right now.

Based on how they answer, you might come up with a lot more questions on that theme.