Questions to help figure out what to test, question 4
On many projects, you aren't the subject matter expert and you don't write the code. In those cases it can be helpful to talk with the subject matter experts and programmers to better understand the subtleties of the software. I often find it helpful to get them to help me design different types of tests. To help pull out the information I need, I use questions.
A couple of questions (or theme of questions) are "How would you know if a calculation was correct? How do you check your answers?" For complex systems where logic or calculations might be hidden from you, this can give you an idea of what oracles they might use when they test. It might also give you insight into how much checking they are actually doing. When they test, are they saying "Yea, that looks close enough." Or are they really digging in to make sure something is correct?
Based on how they answer, you might come up with a lot more questions on that theme.
A couple of questions (or theme of questions) are "How would you know if a calculation was correct? How do you check your answers?" For complex systems where logic or calculations might be hidden from you, this can give you an idea of what oracles they might use when they test. It might also give you insight into how much checking they are actually doing. When they test, are they saying "Yea, that looks close enough." Or are they really digging in to make sure something is correct?
Based on how they answer, you might come up with a lot more questions on that theme.