When performance testing, a lot of time gets spent calibrating your tests. To do this effectively, you often have to calibrate using multiple methods. One method I use is to look at transactions over a period of time.

When looking at transactions over a period of time, your concern is that your test isn't doing too much or too little over the time interval. Examples might include logins/logouts per minute, form or file submissions rates per minute, reports generated per minute, web service calls per second, searches per second, etc. The idea is that you are looking to roughly approximate load (as determined by transactions) with your test.

Likely, you won't just be looking at one transaction. You'll be looking at many of them. It's a bit of a balancing act. Tuning one might knock one of your other transactions out of whack. You then get to tune again.

You'll need to have some idea of what these transactions might look like in production (both actual and forecasted). You might decide to build a couple of different scenarios based on the data you get. If your tool alows it, try to build in a way to programatically track and throttle these numbers as needed. It might save you a lot of time.

This week we'll be looking at mind-mapping; tips and examples. I'll start us off with an example. Last year, while in a rather boring meeting, I drew out a mind-map for how I might get better at performance testing:



As you can see by how I break out learning, practice, and doing - I'm a fan of deliberate practice. I included marketing in my map, because at the time I think I was still doing some consulting work. However, I think it applies to anyone who want's to do more performance testing - even if you're a full time employee. Sometimes you have to market your new skills internally to get new challenges.

A little over a year ago I met with another tester in town to provide advice on performance testing. I recently found the notes from the meeting in an old notebook. In the five pages of handwritten notes, I documented my understanding of the problem and how I would approach the project. Take these notes for what they are - sketches over the course of an hour or two as I asked questions and clarified the project. However, I thought they would be interesting to share.

The fifth page was a description of a performance degradation curve and how to use it.



As we discussed how to translate the results into a simple chart, I provided an overview of a performance degredation curve. We talked about what it might look like to graph capacity and/or licensing.

A little over a year ago I met with another tester in town to provide advice on performance testing. I recently found the notes from the meeting in an old notebook. In the five pages of handwritten notes, I documented my understanding of the problem and how I would approach the project. Take these notes for what they are - sketches over the course of an hour or two as I asked questions and clarified the project. However, I thought they would be interesting to share.

The fourth page was an attempt to illustrate how I would approach organizing my work and how to present results to the various stakeholders.



I tried to paint a picture of an Excel workbook with different worksheets detailing out various information about the project. We discussed a tab for key project information, a tab for an overview of the architecture, a tab for the usage model, and then some number of tabs for the actual test data, test results, and charts. As we talked, I tried to give an idea of where the information might come from and what it might look like.

A little over a year ago I met with another tester in town to provide advice on performance testing. I recently found the notes from the meeting in an old notebook. In the five pages of handwritten notes, I documented my understanding of the problem and how I would approach the project. Take these notes for what they are - sketches over the course of an hour or two as I asked questions and clarified the project. However, I thought they would be interesting to share.

The third page was an attempt to document the initial usage model.


The notation is the UCML, which stands for User Communty Modeling Language. You'll see four types of users: clients (aka "users"), administrators, customers (of the clients), and system tasks. From there I attempted to capture tasks of interest and some rough percentages for each task across the user population.