Quality Assurance

From iGeek
Jump to: navigation, search
QualityAssurance.jpeg
Quality Assurance is a bit of an oxymoron, as quality is never assured. I spent a lifetime doing Software Quality Assurance one summer. After that, my dealings with QA, and appreciation for what they do has never been the same.

Quality Assurance

QA people, in general, are the testers who make sure things work and work well, before you ship the product to customers. If your product doesn't work well, customers will make a lot of noise, harm your reputation, demand their money back, sue you, and otherwise make bad things happen. Even the best case, if your customers are dead, nice or stupid, you're still going to have to fix the problems after your product has shipped; which costs far more than if you fixed the problems earlier. So QA is the good guys, saving you money and your reputation.

I was a young programmer; I'd started in my early teens. And I'd done some consulting for a few large organizations including aerospace, government, and education; but I wanted experience in the commercial sector and to have a "real" job, not just consult. I was told the way to do this was "to start at the bottom" and work up, and somehow this was equated to working in Quality Assurance for Pertec (also known as MITS of the Altair fame). This was the very early 80's. They were building a "super-micro" computer that was a very powerful multi-user 68000 System that ran UNIX, PIC and their own proprietary Operating System. That sounded both interesting and fun; OK, I'm truly a geek and always have been.

Monday, my first day on the job, was the intro to the hardware, having lots of documentation dumped in my cube, shown the facilities, and told what I was to be doing; testing the operating system (mostly their own proprietary one) to find bugs in it and various programs. A little open-ended, but it sounded interesting. I was shown how to fill out a bug report on paper, which I could then submit in triplicate, and when the new builds came out on the following Tuesday we were supposed to retest any bugs that were fixed, and then if we had time, go back and retest old bugs as well (full regression testing). Sounded easy enough.

Tuesday I started testing against the new version of the software and quickly documented 50 bugs that hadn't been entered in the system yet, and about 1/2 of them were real showstoppers - crash and burn, system halting and flaming out kinda stuff. Hey, they were making this easy for me, I submitted them all with great pride in my diligence.

Wednesday I was told, "Whoa, slow down there Mr. Enthusiasm. You don't want to be labeled a nitpicker and demoralize the programmers with too many bugs". This was an interesting concept to me since I thought to pick nits and finding bugs was my job. But I was told to slow down. Fine, I found another 50 bugs, but saved them for the following weeks build, and wouldn't submit them too fast.

That meant I had 3-4 days to twiddle and wait for the next build. What to do? Well, the OS didn't have a utility to do live chat (and file transfers) from one terminal/user to the next; so I wrote one. QA and others were spread out over the building, so it was quite handy and took me the rest of the week, and was a huge hit with the QA staff.

The next week, I got a new build. Supposedly 10 of the 50 bugs had been addressed, but when I retested, 5 still existed, 3 still failed but in slightly different ways, and 2 were actually fixed, and 40 had been untouched. I resubmitted the 8, retested the other 40, and submitted the 50 bugs that I'd found the week before but had held off on to avoid more feelings of inadequacy. Then I hunted for about 25 more bugs for the next week's list.

So it was Wednesday again, and I had 4 days to twiddle my thumbs. I wrote my own macro/scripting and testing system to capture keystrokes and scripts and play them back, so I could automate testing. This was the early 80s before everyone had these tools. I created the scripts that would test for all my bugs and was done. Now I could verify bugs in about 1/4 the time of normal. I showed some others how to do that too, and again, a big hit with the QA team.

The next build came in, and the same story; a few were claimed fixed, but most weren't actually fixed. Now I could bash through all 90 of my outstanding bugs in a few hours, and add in a list of 40 or 50 more. I'd found some real doozies too; holes in security, ways to get root access, ways to block administrators from reading my files, ways to reschedule my own programs to get more priority/time than anyone else's and so on. Wow, I was knocking them dead and going to really help the quality of this product.

So Wednesday I got another talking to; I'd already submitted more bugs than anyone else on QA, and this was my third week. And some of my bugs were too obscure; we were selling to schools and businesses, no one would try to break into our system or read each other's files and stuff, so why was I wasting time on that? And my "tests" could temporarily bring down the system or require rebooting when people duplicated them, which was hampering some other people's testing, so stop that. Again, I thought that was my job; fine, I'd documented many of the security issues and was told to leave it alone, so I did.

Again it was Wednesday, and I had nearly the whole week to work on other stuff.

By my second month, I'd decided that this job paid me to work one or two days a week; and to keep retesting things that the programmers themselves admitted they hadn't fixed yet, and to keep from submitting problems that might need to be addressed, but also might hurt feelings. In the few days each week I had free, I got real work done; I got to read and research and do all sorts of things. I did network testing, and read all about IBM's SNA protocols and laughed that they were still using EBCDIC, and created my own network analyzers and snoopers. I wrote network games for their systems like checkers, backgammon, star-trek, and Yahtzee. I created all sorts of tools for QA, but I had to be quiet and not let management find out or I'd get lectured again. I was having an OK time, though I was slightly disturbed that I wasn't able to do my actual job.

Finally, after about 6 months, I'd had enough of this game; I'd submitted a few hundred bugs, and 1 out of 10 ever got fixed. I'd read all their internal manuals, and knew how the system worked. So I started fixing the bugs that I'd reported. I didn't change the source code itself; but I added addendum's to my bug reports so they said, "on line 5 of file "xxx", change this to that, and the bug will be fixed", and after two weeks I submitted about 200 of those at once.

It took a few weeks for people to notice, but when they did, the proverbial you know what hit the fan and splattered all over you know who. Management was less than amused. I was not hired as a programmer, what was I doing fixing their programs? I didn't even have a degree! So obviously, I wasn't qualified to code. Not only that, I'd hurt the programmer's feelings by fixing their bugs, and they were extending their pocket protectors as armor, and their slide rules as swords to charge on the uppity little QA kid that dared to help them. In two weeks I'd fixed more bugs of mine that they had fixed in the prior 6 months, but that's not the point; I'd violated the processes and duties of my job. Furthermore, they'd found out that there were all sorts of utilities and programs written that people were using, that hadn't been written by the software department! How dare I! I wasn't supposed to be doing that! In fact many of those utilities made the QA people so much more efficient (what with scripted testings, communications, electronic bug reporting and so on), that the company needed to get rid of someone; so either I could shape up, and never write another program or touch their OS again, and stop submitting so many bugs, or I could leave. I left.

Today

Now, in today's world, management is far savvier, and most QA people aren't punished nearly as much for automating processes or doing their jobs well. So that's sort of a caricature of today's QA. But there are still elements of that around. Management or other teams think Q.A. is calling the baby ugly, just because the product needs improvement, and those that do their job best are often labeled as nit-pickers, loud-mouths or trouble makers. At least outside the QA or support departments.

Q.A. people are supposed to find and report bugs. But a few Prima Donna programmers or managers get offended that QA people are saving the company money by finding the bugs before they get to customers and tarnish the reputation of the company. How DARE they! And so sometimes there is an adversarial relationship between programmers and software QA when really the QA people are just saving the software people's bacon if only the programmers had the humility to understand that.

There's an attitude by some immature programmers that think of QA people as "people that couldn't cut it as real coders". Certainly a few are like that, but many just chose a different career path. Some were often good programmers that just burned out of the B.S. on that side of the fence, and so choose to create or improve products on the quality side of things. Some just haven't made the transition yet. So these stereotypes are unfair and ignore the difficult job that QA people have to do. Most programmers I've known couldn't cut it as a QA person. QA can be a tedious job, with little appreciation.

This attitude about shit on Q.A. people doesn't just end with just programmers. Management doesn't usually understand what they do, or how much they are saving the company; so they often have this attitude that Q.A. is an annoyance that whines about quality all the time, and slows down releases. I've heard people say, "If it wasn't for them, we could have shipped already". Um, excuse me? You could have, but what would have shipped? There's this constant pressure on Q.A. to just sign off and say "good enough" - and then blame them when they do because customers are underwhelmed with the quality and lots of bugs got through.

QA workloads in most organizations are far higher nowadays; but they still have elements of binge and purge. Times when they have found way more bugs than anyone can address so they have to back off and do other things; and times right after a build and before a release, when they need to work 25 hour days in order to test everything that needs to be tested, for release, tomorrow. Ready or not.

Conclusion

Sometimes I treat QA people like they are half demented postal workers, who hate their jobs, hate their bosses, and hate their lives because the job can do that to them. The idea is that they are often under-appreciated, under-valued, and just one step away from bringing a UZI to work and improving the quality of the product tenfold.

  • "Thank you for finding that bug Mr. QA person, it will make a better product."
  • "You like me, don't you?"
  • "If you were thinking of eliminating the management and software team, you'd tell me to call in sick that day, wouldn't you?"

I do that not only out of genuine fear but with reverence and respect that they deserve. The specialized tedium and torture that their job entails are beyond me, and my tolerance level for pain. They don't deserve to be ridiculed for that but worshipped. They truly are underappreciated for saving companies money and reputations, for preventing engineers from looking like bigger idiots than they sometimes are, and for helping us all to have better products. So I say to them, "thanks" for doing a thankless job.

Written: 2003.01.06