Software quality is about increasing value by increasing company understanding of the software
it’s producing. The mission of software testers is to gather information. Low-value work like
discreet bug detection is too prominent in the software testing world. Don’t focus on defects;
those are low-value. Over-quantifying information is another way to provide low value. Focus on
delivering value that the CEO can appreciate. These are a few of the major takeaways from the
inaugural meeting of the Greater Boston Software Testing Interest Group.
Monday, July 22, 2013 will mark the second monthly meeting of the local software testing group.
Their inaugural meeting in June was an intimate event with a lot of great discussion. Their guest
speaker, Keith Klain, head of the global test center for Barclays, led the discussion on the value
of software testing with the help of group organizers Jean Ann Harrison, software testing and
services consultant at Project Realms Inc., and Lorinda Brandon, director of solutions strategy at
SmartBear Software.
While those three led the discussion, every voice was heard at least once over the course of the
event.
Speak like normal people.
Keith Klain,
guest speaker
Harrison summed up the driving message of the night by saying, “Software testing provides the
service of information to stakeholders.” Sometimes a part of that service is challenging the
information stakeholders provide. If it turns out the information they’re working from is
false, business leaders will likely make bad decisions. Testers help comb out the
misinformation.
Testing more than just code
To some extent, software testers can test ideas rather than just testing code. The role
of the tester should be much more than that. “The testers should be thinking about Joe the
CEO,” Klain said. He said testers should never be too focused on numbers, especially in meetings
where they need to be able to translate numbers into information that is interesting and
informative to the CEO.
The CEO doesn’t care about the number of tests run or the
exact number of defects that were found in those tests, according to Klain. The CEO cares if
the application has any flaws that need to be fixed. Just because a flaw exists, doesn’t mean the
business will care about it. “If they don’t care about it, it’s not your problem. You have to talk
to their actual objectives,” Klain said. It’s not up to the testers or project managers to define
value for the business. “Stakeholders define the value they’re looking for and you help them
achieve that value, or identify threats to it,” Klain said.
Having test
conversations with executives is a difficult task for many testers, and it was the topic of a
talk by Scott Barber, chief technology officer at PerfTestPlus Inc., at STP Con 2013. Turning
everything into numbers is the wrong mindset. It feeds the idea that software testing is just an
operating expense and not a value stream. “Speak like normal people,” Klain advised the group of
software testers.
Know what to test for
Software testers need to know the domain they’re working in. You need to be able to put aside
what you would want in an application and focus on what your actual user base requires. User
requirements frequently change slightly from industry to industry or from platform to platform.
It’s important to ask the right questions and take a systematic approach to validating the answers.
The best testers constantly break out of their own software testing biases, Klain said. “You have
to be able to think this way for a while and then switch over and think that way.”
A major problem in one industry may be an expected feature in another, or vice versa. For
example, an attendee brought up a time when he was providing a mobile app for financial
organizations to be used by financial portfolio managers. The app had a black screen with green
type. It was essentially a green screen application on a mobile device. Because of his prior
experience with mobile applications, this design was an immediate red flag.
“Mobile users don’t want this type of user experience,” he said. But he eventually realized he
was wrong. Financial portfolio managers, it turns out, are no-nonsense people. They want to know
what values to punch into the application and they want to know they can trust the value it spits
back at them at the end. After that, anything that takes away from the speed of the calculations is
just bloat.
It’s also important to keep in mind that quality assurance is still a part of the business.
While it’s important to keep the customers and end users in mind, software-quality pros should
really be ensuring software meets the needs of the business. Klain said, “Customer perspective is
important, but so is applying good judgment. [Testers] have to learn to be dispassionate and forego
their own biases.” He suggested that bias
identification and value checking are rarely done in a formal, systematic way. “It’s a very
intuitive process and good project managers just do it.”
Not your traditional software testing skills
Brandon said the most important thing is that when software quality folks do their job right,
the business knows what they’re dealing with and can make the right decisions. “[Business folks]
are going to have to make compromises, so it’s important for [test
managers] to lay out simply what they found and what that impacts.”
For Klain, it sometimes comes down to psychology. The project manager needs to understand what
motivates which stakeholders so they can tailor the information they find to fit that individual.
He said he takes the time to plan and rehearse important conversations he knows he’ll have with
stakeholders. A conversation about the same issues might go totally differently with one
stakeholder than another, depending on what’s important to each stakeholder.
Klain told the software testing group there is a lot of bad training available for them.
Software testing text books, he seemed to think, focus too much on low-value areas like defect
tracking. When inexperienced project managers ask Klain what book they should be reading, Klain
said, “The books I recommend are just as likely to be books on psychology and sociology.”
About the speakers
Keith Klain is a member of the board of directors for the Association for Software Testing, a professional non-profit group.
Klain is also the head of the global test center for Barclays and has nearly two decades of
experience working with software testing and application project management. Klain frequently
writes and speaks about software test education.
Lorinda Brandon has more than twenty five years of experience in high-tech
project management roles including software quality assurance, as well as others. She is currently
director of solutions strategy at SmartBear Software and the official organizer of the Greater Boston
Software Testing Interest Group.
Jean Ann Harrison is a software testing and services consultant at Project
Realms Inc, and a partner and senior technical consultant at Perfect Pitch
Marketing. She has more than 13 years of experience in the field of software testing and
quality assurance and is very involved in organizing and promoting the new Greater Boston Software
Testing Interest Group.