TestBash Sydney 2018 Reflection – Part 2

The second half of my summary and reflections on the TestBash 2018 conference in Sydney. For the first half: TestBash Sydney 2018 Reflection – Part 1

Avoid Sleepwalking to Failure! On Abstractions and Keeping it Real in Software Teams

by Paul Maxwell-Walters – @TestingRants | http://testingrants.blogspot.com.au

The main focus of the talk seemed to be on Hypernormalisation, where you are so much a part of the system that you can’t see beyond it. For example, the message of society is that everything is OK, when really it isn’t, and you know it, but perhaps can’t explain why.

This can apply to testers, where we just accept things the way they are and don’t try and make things better, perhaps because we can’t see/imagine how it could be better.

  • Hyper-reality – A world that you interact with, that isn’t real (e.g  DisneyWorld). This is how we sometimes relate to software testing.

Abstraction #1 – Quality

The definition of quality is different for everyone. James Bach said in 2009 that Quality is dead, this is a mini-hypernormalisation. He didn’t like the way things were going, and was somewhat acceptant of that.

Abstraction #2 – Measurement

Rich Rogers said in his book “Changing times: Quality for humans in a digital age” in 2017 that we assume when we have tested all of our criteria, that we have good quality (but this is wrong!). If a team’s definition of quality or measurement isn’t true to reality, then why persist with it?

Therefore, we don’t have to accept the way things are just because we don’t see how it could be any better.

My thoughts

Personally I found this a really hard talk to follow (as might be evident in my notes above), in between the old news videos of Russia and the large amount/complexity of content. I felt the point of the talk of not just accepting what is being thrown at us in Quality could have been made much simpler and more time spent on applications of how to do this, or what areas of the job we should be actively looking for better ways to do.

I do strongly agree with the idea of not accepting bad practices just because that’s how it is now, and striving to make things better (ie being agile), but I found the delivery missed the mark a little bit.

Test Representatives – An Alternative Approach to Test Practice Management

by Georgia de Pont – @georgia_chunn

Georgia started by giving some of the context of the patterns/techniques used at Tyro, being Pairing XP, TDD and the Spotify model of squads. We then followed the journey that the testing team at Tyro have gone through, starting by moving from a disconnected team into having embedded testers.
This created some challenges:

  • Loss of alignment and knowledge sharing within the testing group
  • Lack of consistency in test approaches and usage of test members within the team
  • Lack of a test manager (it was a flat structure)

The question arose of how to address these issues, should they hire a test manager or take a grassroots approach? (Evidently, they took the grassroots approach).

Each product tribe selected a test representative to act as representative. They would support their tribe’s testers, gather info on issues and be a point of contact for that tribe. The representatives came up with a pipeline of improvement initiatives to work through, with ideas coming in a variety of ways. The representative group, “rep group”, would share the info back to their test teams and would have regular meetings with the TestOps team.
Some of the initiatives:

  • Clarity of role of embedded tester, e.g. what are the test practices they can offer
  • Improving the recruitment process. Candidates were given a take home test, which wasn’t reflective of the workplace because people usually pair. So they created a pairing exercise instead.
  • Onboarding new test engineers and probation expectations
  • Performance criteria for test engineers for them and their leads to use
  • Upskilling test engineers (an ongoing effort) of making sure there is available training, organising internal and external speakers, conferences, etc.
  • Co-ordinating engineering-wide test efforts
  • Developing a quality engineering strategy, involving various stakeholders, to identify any current roadblocks in the testing efforts and work to remove them

Some steps for success to make a representative group work in other workplaces:

  • Start small. Think about how many people per teams (aim for 1 rep per 2-4 teams)
  • In forming the rep group, consider whether best to select people or ask for volunteers. (they selected people to start with)
  • Communicate the work of this group to the rest of the organisation
    • Maybe hook into existing communications
    • Include ways to get involved
  • Ask for feedback (e.g. surveys)
    • Now they do this more informally
    • Asking: Is it working/effective?
  • Run Rep group retrospectives
  • Ensure support from engineering leadership
    • Maybe include them in meetings
    • Get their feedback/input
    • Give them an awareness of upcoming testing concerns
    • Get budget support

My thoughts

Georgia did quite well for her first talk, it was clear that she had practiced presenting this before, she had helpful slides to guide the content and carried confidence in her presentation. With practice presenting she will only get better, and it will be interesting to see her speak again in a few years with more experience.

The content was relevant, describing a smart solution to problems that many organisations are now feeling with the recent direction testing roles are taking. Even for companies not in a position to form a ‘rep group’ there were definite takeaways in how a testing group can, and should, interact with other stakeholders within Engineering. The initiatives the group came up with look really interesting, and I would’ve liked to hear much more about those as there appeared to be some directly applicable opportunities in there, perhaps this will be the subject of future talks?

Enchanting Experiences – The Future of Mobile Apps

by Parimala Hariprasad – @PariHariprasad | http://curioustester.blogspot.co.uk/

Pari finished out the day as the final keynote, speaking on where she sees the design of applications and devices heading in a heavily connected world and what opportunities could be unlocked.

It started with Machine Input, with a definition that this is all about information that devices can find on their own.

Products are developing all the time, getting more complex, with more interfaces, but perhaps what we need is to get simpler again, and automatically detect more of the options we would otherwise have to choose (through machine input).
We then looked at a range of machine input types

  • Camera – e.g. auto detect details of a credit card held in front of the camera and pre-enter
  • Location – detect where you are and make suggestions, including smart defaults like detecting the country and local currency
  • History – Each time you fill out forms, remember what data you put in, particularly between site visits.
  • There’s lots of other sensors in our phones that can be hooked into.

Could we detect which floor someone is on, which shop they are in, what they have recently searched for and offer them a discount as they walk around the shop, without prompting?

The internet of things is leading to a more and more connected home and we looked at some ideas companies like Google are developing in this space.

  • Designing great products is about great experiences.
  • Security and privacy is really important with all this data being available
    • Devices should not dictate us and how we live, they should make things easier and work for us
  • Learn about design and usability to see how it can impact your testing efforts/plan.

My thoughts

No particular takeaways from this one, as I felt it wasn’t really a talk targeted for testers, more of a talk for developers or designers who can more directly incorporate Pari’s design thoughts into their work. It was interesting to think through some of the options made possible in various parts of our lives through machine learning from the point of view of: “that would be cool to see!”. There will definitely be challenges in testing any of these types of features, which wasn’t covered much besides acknowledging that security will be key with all these features. Also the content spawned thought processes on what things would need to be considered in testing these sort of technologies.

99 Second talks

99 second talks then wrapped up the day with about 20 people speaking. For most people it was a chance to practice public speaking in a somewhat non-threatening environment. For others it was a chance to promote their company/group, or just a chance to introduce a topic/technique. Hard to take much in given the format, and I didn’t have notes of any takeaways from this, but some interesting topics raised.

Overall thoughts on the conference

One of the big selling points leading up to TestBash was the community feel of the event, particularly seen in the pre and post conference meetups. I was unable to attend either, so I didn’t get the full experience and can’t really comment on how the conference lived up to that hype. The conference day felt much like most other conferences I’ve been to in terms of interactions between sessions and general flow of the day.

There wasn’t much chance for questions with most of the talks, which was unfortunate, as there can be some great discussion coming out of this as people dive in to the bits that were missed or are really interesting to them. Though, in saying that, question time can be hit and miss depending on which direction the conversation gets steered towards, so not a big concern.

The talk selection was pretty good considering it was a single track, so every talk had to be chosen for being applicable to a wide audience, covering a generally applicable topic, but trying to make that topic interesting enough for anyone to learn from. This is a pretty hard criteria to meet, so hats off to the selection team. If you were wanting a deep dive on particular topics, this is perhaps not going to be the best type of conference to attend, but it could still happen on the right topic.

Speakers are well looked after by TestBash, in communications and compensation, though as a local speaker, there was nothing to be compensated for, so that is definitely more of a perk for non-local speakers.

The swag was better than most conferences, aiming for practical items that can be re-used beyond the day.

Overall I found the day enjoyable, I got good feedback on my talk and I have some renewed ideas and techniques to take back into my workplace from other talks which makes for a successful conference in my regards. This is helped by sharing my thoughts afterwards, which I recommend others try after any conference/training, even if just to peers in their workplace.

Would I go to the next one? Maybe, if I wasn’t speaking I would need to look at the schedule first, and see what talks are being covered. I’m definitely glad we have another testing conference in Australia, we seem to be adding more to the mix every year at the moment, and each one lifts the overall quality of presentations as people get more practice, we get more international speakers and a wider range of topics and conference setups to choose from. Each conference will have to work harder and harder to stand out and provide a good experience to attendees, which is a win for everyone! So I’m glad TestBash has come to Australia, and is coming back next year!

Create the Change You Want – Part 3

Before you start, catch up on Part 1 of this series, and Part 2



What it means

The last section I’ll talk about is self-review. This entails thinking hard about where you are now, the skills and knowledge you have, the tools you use, the techniques you use, everything that is part of you doing your job. And then, take this picture, and measure it against where you want to be, what things you want to be doing. What can you start doing or stop doing? Is the way you do things today, the best way to do it or the most interesting way to do it? If not, then how can you change it? Even with decades of experience, it’s unlikely you know the best way to do everything and have completely mastered every aspect of your role. So there’s always room for growth.

sports-team-reviewing-gameLike a sports team watches video replays of the game to see where they have made mistakes or missed opportunities, we need to retrospectively look at ourselves and our processes and see where we can improve. Or something you just want to do differently to mix things up a bit. Only you will know the parts of your job that are boring to you. Think through, ask people or read about what you can do to change it up?

Is there a manual process you can automate? Is there a new technology or tool you can try out? Can you have a go at part of the process that someone else normally handles? Identify an area you want to change and then look for ways to make it happen, which will probably come back to your reputation and relationships as we discussed earlier.

3 examples

Let me share some experiences I’ve had with self-review.

writing-codeEarly on in my career I was noticing that I was finding great enjoyment in writing
automated code
to test our UI, particularly for the complex parts of the app where I really had to solve some tricky problems. So my change I wanted to make was actually just to spend more time doing this kind of testing. So I endeavoured to find more ways to get benefits out of writing automated solutions for test cases, because I enjoyed doing it.

A review might just be to help identify the parts of your role that you enjoy doing and thinking through how you can bring more of that type of work into your role.

Reviewing the way I was being a bottleneck for testing within our team as I mentioned in part 1 of this series led to changes in our team’s processes and this also enabled me to spend some more time learning some developer skills as I tried my hand doing some developer work within the team, making use of the reduced testing requirements. And I liked it.

Perhaps you can find other roles that you can try your hand at to see if you like them and incorporate them into your role. This is the idea behind being a T-Shaped team member, where you bring multiple skills to the team, and so is of course a desirable characteristic for your company as well.

public-speakingPerhaps the biggest action to come out of reviewing myself in the past year was in my decision to enter the public testing world through blog writing and seeking to speak at a conference. Previously I wasn’t too interested in writing anything or sharing with the general public, I didn’t think anyone would care about what I had to say and I was worried about how people would react, but I saw a great area for growth which would challenge me in a big way to work on my communication skills and respond to any feedback I would get. I’ve now been writing for over a year and have now spoken at my first conference, showing how far I have come in this journey. A huge change from high school Pete who would struggle to do a 3-minute speech with palm cards.

Take-home challenge

My final take home challenge is to take some time, and think through the skills you have, the tools and techniques you use and the processes you use. Where are the boring bits, what do you want to do more or less of? Find the areas you want to change, and make it happen!

Bonus Content

I’m enjoying sharing this journey, so I’ll add in a few extra stories of changes I’ve gone through in my current job.

In my 7 years I’ve worked under 4 different QA team leads. I’ve also worked alongside 6 different testers and am currently in a team of 3 testers. So there have been significant changes in my role that have been somewhat out of my control. It’s quite common that others coming and going from the company provide new opportunities to grow and be challenged in different ways. Each manager had different leadership styles and as I grew in my knowledge, new opportunities and responsibilities were opened up to me. I’ve grown with each manager and I’ve learnt new things from each new tester that I’ve been able to work alongside.

campaign-monitor-staff-meetingSometimes being in a company for a while means that new roles open up and you find yourself being asked to try out new roles as the needs arise because you carry with you a lot of knowledge about the company already. This happened for me 5 years ago and I had a brief stint in a DevOps role. My responsibility was to create testing environments for each product team to work on. That was certainly not a role I had imagined going into and there was a lot of unknowns about what I would be doing, plus I had to learn quite a lot in this new role. It was an interesting experience and I definitely learnt some new skills, but in the end it wasn’t for me so I went back to testing.

Perhaps there are opportunities to try out different roles in the company you might be interested to take up. It’s certainly easier to do that in a company you already know, with people you already know then it would be to start a new role in a new company where there’s twice as much to learn. It might not even be a full-time swap, maybe you could just take on part of a new role alongside testing?

Finally, a challenge for you, coming back to where I started the series with Change.org. If there was one thing you could change about your role at work or your company, what would it be? And how would you go about getting others to sign on to your petition to see it happen? Create the change you want to see.


As we conclude: at the start of this series I asked if you currently or have ever become bored or stagnant in your job, like you weren’t growing anymore and needing to change jobs. I then challenged you to create the change you want where you are now so you can start to grow again without the hassle of changing jobs.

So now, having read, I hope you feel refreshed and excited about new opportunities for growth. Be inspired to make changes and improvements where you are without the hassle of changing jobs.

  • Make use of your reputation and relationships to drive opportunities for change.
  • Start to consider if there is someone you can share your knowledge with who will challenge your assumptions and someone you can learn from.
  • Reflect on whether the things you’ve always done are still the best way and the areas of your work you want to change.

I really hope you are more equipped to shake off any feelings of boredom or stagnation and can start to grow once again.

Thanks for reading.

Create the Change You Want – Part 2

Before you start, catch up on Part 1 of this series



What it means

Next, I want to talk about coaching. I mentioned in the previous post that the best way to master something is to learn it well enough that you can effectively explain it to someone else.

Have you ever tried explaining something to a toddler?

You: “Don’t kick the ball in the house”, toddler
Toddler: “Why?”,
Y: “Because you might break something”,
T: “but Why?”,
Y: “because you might hit a vase and make it fall down and break”,
T: “so?”,
Y: “so your mum would be upset if the vase broke”,
T: “why?”,
Y: “because she likes it”,
T: “why”,
Y: “I don’t know”,
T: “why not” …

You get the point, when you try to explain something, people tend to ask why, which can really stretch your understanding and beliefs. You’ve developed a set of assumptions that the way you do things now is the best way to do it, or that it’s too hard to change. You teach this message to someone else, but they don’t agree, they might say, ‘Why don’t you try writing a program to do that for you?’ … Perhaps you’ll be pleasantly surprised and discover a new solution that works better and say: I hadn’t thought of that, or I don’t know how to do that. Or in other words, that sounds like something I need to change, something new for me to learn. Great!

As you spend time coaching someone, you are being tested each day to deeply understand what you are teaching as you explain it to someone else. It’s a great way to find out if you’ve thought of all the different options or to cement your knowledge.

best-sports-coach-linksIn a sport’s team, it’s the coach’s job to help show people where they need to improve, and guide them through how to improve. A coach within testing works the same way, pointing out areas for growth, and assisting through the growth. Having someone coach you in testing is quite beneficial for you in helping to identify your weak spots and help you develop a plan to get better. Perhaps the areas of your role that you’ve always stayed clear of will become a little less intimidating, or you are challenged to approach a situation differently. Either way, there are changes for you to make, challenges to take on and new growth to be found.

3 examples

Here’s some examples where I’ve tried this. The first one is actually the reason I’m presenting this series. I have a mentor, Katrina Clokie, who I was connected with through the SpeakEasy mentoring program who has been mentoring me in how to write a talk to

CAST2016 talk - 3
Me, speaking at CAST2016

present at a conference. Just over a year ago, we started connecting via fortnightly skype sessions, discussing what I would need to do in order to speak at a conference. Starting from fine tuning ideas, to learning how to communicate these ideas into an abstract and right through to actually putting the talk together and creating slides. I knew something about these skills previously, but definitely learnt a lot from her guidance.

I learnt how to assess what ideas were worth sharing. I learnt about having a target audience in mind when writing so that the content can be more intentional and actionable. And I learnt about how to communicate my ideas. If you are interested in speaking at conferences, I definitely recommend reaching out to SpeakEasy to get yourself a mentor.

From the viewpoint of being the coach, I’ve found my skills and understanding of writing automated testing solutions have really been challenged when I try to explain my test approaches to my workmates and explain how and why I’ve made the choices I did. Solutions I thought were quite clever and effective or just the only way to do things were challenged as i explained and presented them. I’ve been challenged to investigate whole new technologies and code structures to get a better solution which of course means stepping away from my comfort zone and trying new things which means areas for growth.

I’ve learnt a lot about re-using code, readability of code, maintainability, and much more from the feedback of others as I’ve tried to coach them in the way I’ve written my automated solutions.

The other main source of learning I’ve had from others coaching me is actually indirect coaching through attending conferences, going on training courses and reading blogs. These are all great ways to indirectly have other people coach you as they share their knowledge to a group. bloggingThere are hundreds of testing blogs, several testing conferences and training courses that are worth going to and provide great opportunities to learn. Which of course, you already know because that’s why you are reading this blog! Keep using these opportunities all around you.

I’ve also experienced this side for learning as the coach as I’ve shared some of my learnings on this blog and had hundreds of people from all over the world read my posts to learn from me. It’s amazing how many people from so many countries want to learn about testing. Communicating my ideas well is certainly an area requiring constant improvement for me.

Take-home challenge

So the take home challenge for this section is probably pretty easy to guess. Find someone you can coach in a skill you are familiar with so they can challenge you and push you to really master it. And going the other way, find someone who can coach you to teach you new skills, whether someone in your company or perhaps in a local meetup, or if needed even indirectly via the internet. Having someone to coach you means they can teach you new skills that they have and challenge you to grow in different areas.

This is part 2 of 3, Stay tuned for part 3 coming soon! Or take another look at part 1

Create the Change You Want – Part 1


I presented this talk at the Association for Software Testing conference, CAST2016, in Vancouver, Canada in August 2016. This version is slightly updated for an online audience.

wheelchair-beachChange.org is the World’s largest petition website where people submit something they want to change, then gather the support of thousands of other people and as a result, quite often are able to see that change happen. Just this month in August, a petition to create a wheelchair friendly beach on the Gold Coast has been approved, last year a petition led to the bodies of soldiers killed bringing-home-soldiersin the Vietnam War be returned home to Australia from overseas and late last year another petition led to the removal of the Heart Foundation Tick onrefugees our food products after claims it was falsely labelling unhealthy food as healthy. Last year, in Canada, a petition led to the government accepting in 10,000 Syrian refugees.

These people saw something they wanted to change, and by doing something about it, were able to make the change happen. That’s what I want to talk about in this blog series, creating the change you want, in your work life by sharing my experience in creating changes in my work life.

I’ve been working as a software tester for 7 years at a place called Campaign Monitor. My job involves a mixture of writing automated solutions and testing and plenty of other miscellaneous bits and pieces as I imagine a lot of you would say about your work. I love problem solving which is something I get to do a lot of in my job and I love helping my team create awesome products through testing. You can find me on twitter at @pete_bartlett.

The reason you need to read this blog series

As I mentioned, I’ve been at my company for 7 years and as I look around my company, it seems I am an unusual case in sticking around in the same job for so long, particularly when not in a managerial role. And this seems to be the same at other companies too.

Graph created mid 2016

This graph of my company shows the number of years people in the Engineering department have worked at the company. A large proportion are under 3 years. Most people start to get bored or feel they aren’t learning any more after 1-2 years and feel they have to move on to something else. Perhaps you feel the same way in your current role, or have felt that in previous roles?

It doesn’t have to be that way! Why not create the change you want to see in the position you are in, and start growing again now? You know how painful it can be changing jobs. Searching through all the job adds, updating your resume, preparing for interviews and then once you find somewhere, you spend the first few weeks or months learning about the context of your new position. Several months go by and you probably haven’t really learnt anything new or had any growth, you’re just adapting to being in a new place. You can avoid all this and use the position you’re already in to create change and start to grow again.

I hope that this blog series inspires you how to do that.

The areas I’ll be addressing

In particular, over this blog series, we are going to look at influencing change using your reputation and relationships, challenging yourself through coaching and reflecting on the work you are doing and wish to be doing.

Reputation and relationships

public-opinionWhat it means

First, let’s start with reputation and relationships. In your workplace, I want you to think about a complex part of the product you work with, and then think about who you would go to if you had a question about it… Why did you pick that person? I can say pretty confidently you picked that person either because of their reputation or your relationship with them. Perhaps they have been with the company a long time and have shown time after time that they know what they’re talking about, so you trust them to be able to answer your question. Or, perhaps, they are someone you have a good relationship with, and that’s why you trust them to answer your question.

This trust goes beyond just bringing your questions to them, if they had a new idea for a different way to approach the problem you were facing, you would trust them and try it out right? That’s the benefit of having a good reputation and relationships right? When you hang around in a company for a while, you will start to build reputation and relationships, and you will start to become the person others come to and listen to. You can use this to your advantage. Use it to get others on board when you find new areas for growth and new challenges to take on, and it’s more likely that change will be acted upon.

Perhaps you want to learn a new programming language or learn about security testing or performance testing. If you can figure out the change you want to make, then the influence you have built up in the company will mean people are more likely to listen and agree to try it out.

3 examples

Let me tell you some examples where I’ve done this.

screen-shot-2016-12-19-at-8-37-43-amRewind 3 years at my company and as the 1 QA member in my product team, this is how our testing process looked. At a very simple level, the developers in the team would pick up a task, work on it until they thought it was finished and then i would test it. If I gave it the all clear, it went out to production, if not, it would go back to the developers to fix. Probably a scenario most of you know well.

So, I tested every task that our team worked on before it went out live to customers. This created a bottleneck in the testing stage of the product lifecycle, particularly if I was away. There was also a lot of back and forth with tasks as I found bugs and sent them back to the developers, then they sent them back to me to re-test, then i found something else and so on. We were creating great things, but it was inefficient, so I wanted to change this.

The first step was identifying this to the team, that we needed to do better, they trusted my opinion due to my reputation and relationships with those in the team, and they agreed to try my suggestions out and make it better together.

First change, I suggest we start doing test reviews. Before a developer commits their code and finish with it, I’ll come to their computer and get them to show me it working, I’ll ask questions, do some quick, happy-path testing with some simple variations to catch any immediate blockers and then they can commit it. Then if necessary I would do a more thorough test if needed on the committed code, knowing that I’ve already caught anything obvious, so it’s a much higher chance of not getting sent back to the developers. It’s a nicer way to work as well, the developers would feel annoyed if they had let a simple bug slip through that they should’ve picked up and there’s a sense of pride inherent with showing off the work you did, which gives me a nice way to commend the developers work as well.

This change took a little bit of time for the team to adjust to, as you would expect with any meaningful change. But we stuck it out and overall the reception to the change was great, it stopped a lot of the back and forth with bugs and so people were happy.

Second change, I say to the team, let’s add acceptance criteria to the tasks before you start work on them so you know exactly what you are trying to achieve and how to confirm it works yourself, without me needing to be involved. We use acceptance criteria as a simple checklist of things to test against as the developers were working to guide the way they verify their work and help them catch any issues while they are writing the code, rather than after.
Here’s a simple example of what I mean. The task being worked on might be a page for receiving credit card payments. So in the acceptance criteria, I could say, make sure you check for validation on the credit card input fields and make sure you have appropriate confirmation messaging for a successful payment. There’s definitely other areas that could be tested but those two points should inspire the developer to think through enough of the primary test cases to use for verifying their work without having to spell out every scenario for them.

Again, this was received really well, people had experienced the pain of me catching something at the test review they hadn’t thought of, and while this was still much better than me finding it hours or days later, it was still at the point where they thought they were done and ready to move on and they wanted to find these issues earlier. So knowing these things I would be looking for up front really helped.

So, not only is our team becoming more efficient, but I’ve been challenged to critically analyse product demo’s to catch issues as the developers were showing me their work, and improve in communicating clearly, as I helped provide acceptance criteria to the team for the tasks they were working on.

The other problem we were seeing was that I was a bottleneck as the only tester, so the third change was to shift that by allowing everyone in the team to help out with testing. Now, when a task was ready for testing, anyone in the team could pick it up and test it, to make sure it can work as expected and I no longer needed to have a hand in every task we worked on. Great! Of course this requires training in showing the developers how to test, what sort of things I look for, and still working with them up front to create acceptance criteria for the tasks. But it’s definitely a worthwhile investment.

With the whole team being involved in testing, tasks were getting finished quicker and getting into the hands of customers quicker which is a win for everyone. This change would take the most convincing to get the team on board and takes the longest to properly implement. We are doing well in this area now within my team after trying it for a few months, but there’s still improvements to be made. Some developers are strong believers in the “I code, you test” approach, but to the credit of my team, they were happy to give it a try, and call out for help when they needed it.

Through these changes, there was more learning for me, this time in teaching testing itself, and I find, the best way to get better at something or to master it is to know it well enough that you can effectively explain it to someone else. This is what is happening when you teach someone about testing, which I’ll touch on more in the next section.

My response to the scenario I found myself in was I changed my environment, I learnt new skills and I improved my existing skills by using my reputation and relationship within the team to create the change I wanted.

Take-home challenge

My challenge to you to take home with you for this section is to find an aspect of your job that you want to change, then, use your reputation and relationships to convince others that it’s a good change to make and guide them through it if needed, working through any hurdles that come along the way. You’ll soon find yourself with a new challenge, and new ways to grow, without the hassle of finding a new job.

This is part 1 of 3, Stay tuned for part 2 coming soon!

Australian Testing Days 2016 Reflection – Day 2

For Part 1 of this series, see Australian Testing Days 2016 Reflection – Day 1

Day 2

Coaching Testers (workshop)Coach showing the right way to follow

I chose Anne-Marie Charett‘s workshop for day 2 to learn about coaching testers. Even though I’m not directly in a coaching role, I thought there would be lots of great things to learn and a number of things that would apply to teaching non-testers in my team about testing. The day started on discovering why each of us had come and what we were hoping to learn from the day so we could focus our discussions accordingly.

First up we thought about coaching sessions and how they should not transform into a “How are you going” session. Much more effective is to make the sessions task-based (e.g. let’s work through an activity and I’ll offer assistance along the way) or for the student to come with a question.

The next lesson was a great approach to teaching new techniques or skills by starting with applying it to a familiar, non-testing situation. For example, creating a mind-map of your family. This takes away the fear of getting it wrong since it’s known and familiar, and greater focus can be put on the technique or skill being learnt.

In understanding what things to coach and how to go about coaching them, we need to understand the context of both the coach and the student. They will each have 2 images of themselves, one of how they view themselves as a tester, and the other as how they view themselves as a coach/student. As well as many other factors that need to be considered in knowing that different approaches work for different people.

The day was broken up with a number of movie clips to show different coaching styles. First up was the scene in The Matrix where Morpheus asks Neo to show him the Kung-Fu he has just learnt. Throughout the scene, more pressure and direction is added from Morpheus as Neo progresses through the task. It starts of easy, then gets harder and more physical as Neo is pushed further and further to learn the lesson Morpheus was teaching him (the rules can be broken).

Next we watched a clip from The Blind Side where Sandra Bullock’s character was better able to teach her foster son how to defend in Grid Iron better than the team coach. This was because she better understand the needs of the student and what motivates them (in this case, family). The question that he needed answered was “Why” not “How”. Trust was also a big factor in the success of the coaching efforts.

Taking this lesson further, as a coach you want to find out what people’s drive is, why are they in testing, how did they get here and where do they want to be? Then the coach can focus on directing them on how to get there and where they need most help, based on the journey they’ve had so far.

Our next video was from The Magnificent Seven, where a young character was challenged to match his speed at drawing his weapon against the team he hoped to join. This was more about putting people out of their comfort zones and seeing how they will respond to pressure. It was a way to show this man that his skills weren’t up to scratch, and that he would need to lift his game.

Following this we talked about Socratic questioning, which is all about not directly answering people’s questions but instead asking questions in return to help lead them to discover the answer for themselves. This is a very powerful tool in coaching, an answer you come to by yourself is much more powerful and memorable than one you were told. But it can also be taken the wrong way and be demotivating when people feel like they never get straight answers.

The coach will need to help manage the way people feel about themselves and base their coaching on the students needs, energy and context. There isn’t a one-size-fits-all approach.

Our final movie clip was from Million Dollar Baby where Hillary Swank’s character is trying to convince Clint Eastwood’s character to coach her. There was certainly reluctance in this coaching partnership, but biases had to be overcome and the student had to reflect and show they did indeed have the passion and desire to not give up, even though it would be hard. Sometimes we too might not feel comfortable at first about coaching someone or being coached.

2 final points were made around managing expectations and reflecting on progress. Manage people’s expectations to learn something quickly by showing that confusion is just a learning state and is not a weakness or lack of ability. It’s also powerful if you can show someone how much they have grown already to help give motivation to continue on that journey.

The day finished with a practical exercise of simulating a coaching session between a Coach, a tester and a developer, trying to come up with acceptance criteria for a payroll system. This was great to put into practice some of the things we had been learning throughout the day and see how even in controlled environments, people have quite different needs, opinions and expectations and so coaching will need to adapt all the time to suit.


I had a great time at Australian Testing Days 2016 and am looking forward to going again next year! I’d also love to hear what other people got out of the conference or if you have a different view point to anything discussed in either of my posts about it.

How do you track Quality?

I was recently asked how we measured quality in our product and was immediately cautious about what to say next. I had these warning bells ringing in my head:

“High code coverage doesn’t equal a high quality product if the most important code isn’t covered”

“A low bug count doesn’t mean anything if those bugs are all critical, similarly a high bug count might not be bad if they are all trivial”

But just because there are bad ways to try and measure quality, doesn’t mean there aren’t helpful ways as well!

Watching the trend

The most important thing to factor in when trendingtrying to measure quality is the trend over time.
5 bugs raised this month is good if the last 3 months averaged 30 bugs each, however if they averaged 0 bugs each month, it’s a concern worth investigating.

Numbers on their own don’t mean a whole lot, there will always be a need to look beyond the numbers to see the type of bugs you are finding, and what impact they have on the customer. The numbers can influence when and where you should be investigating.

Therefore, the measures you decide on need to be tracked over time to see where you are making improvements worth celebrating and where you are getting worse that needs addressing. Though regardless of trends, this is only part of the picture of the quality of your project. Manual verification will always be needed in parallel.

What should I measure?

In my view, the most important thing you can measure to give you an indication of the quality of your project and where to focus your efforts is:

Total Open Bug Count, grouped by priority

Total open bugs graph
A graph of total open bugs, split into the different priority levels (e.g. Critical, Major, Minor and Trivial) grouped by week or day. This will let you know if you are making a dint in your backlog of bugs, as well as the trend in each category of bugs.

This will help you answer the following questions, which you should then decide on how to respond based on your project’s needs:

  1. Are we creating more bugs than we are closing?
  2. How fast are we resolving our critical and major bugs?
  3. Are we creating more critical/major/minor/trivial bugs than we used to?
  4. Do we need to take some time out to tackle our bug backlog?

Other Measures?

Here are some other measures that could be helpful, but I would do these as a secondary focus to the one above as they are mostly covered.

  • Total Bugs Raised, grouped weekly by priority.
    How: Count how many bugs were raised this week.
    Ask: Are we raising more bugs this week than we normally do? Are we raising more high priority bugs than we normally do?
  • Average Bug Resolution time, grouped weekly by priority.
    How: For the bugs resolved this week, measure how long they were open for and get the average.
    Ask: How long is it taking us to fix critical bugs? Are we getting slower or faster at resolving bugs?
  • Open Bug Count By Feature, grouped weekly.
    How: Break the total open bug count down into the feature area affected.
    Ask: Are certain parts of the project more problematic than others?

Is there a different measure you use to track quality in your project? I’d love to hear about it in the comments below.

The Champion for Quality

victory-clipart-II-400x400Being the Champion for Quality doesn’t mean being the only one who cares about it, the only one who does anything to improve it or even the one who is best at improving quality (though that might be true). The Champion for Quality is the voice of motivation and direction for the team to create a high quality product. That’s how I picture the role of a QA/Tester.

You’re the Voice

In the words of Aussie Legend John Farnham, “You’re the Voice”:

ASK what people consider as important for quality so that you know what areas you should focus on to deliver this. Is it more important for the product to be fast than accurate? Is it expected to handle a large amount of users? What sort of failures are acceptable and what are unacceptable?

TEACH people that quality matters. In order to achieve the level of quality desired in the target ideas identified above, the whole team needs to be on board. Show people the full story, how a lapse of quality in 1 area eventually leads to a negative customer experience and then a negative experience for the company.
Teach them about cutting corners now leading to more work later on to do it right.
This shows them the importance of their work and makes them feel a bigger part of the company and encourages them to put in the extra effort now, not later.

TRAIN people in how to build a quality product (by your definition of quality) by giving them the tools they need, the test cases, the conditions to experiment with, a peer to review and a clear understanding of what they need to do to ensure they are creating a high quality product.

Be the Champion of Quality in your team, Ask, Teach and Train!