For the last year I have been running crowd-sourced test
cycles for my company while testing Android and iOS native apps. I learned a
lot, met many good testers, and had the privilege of working with and test leading
a few unbelievably passionate and talented people. Leading testers in a remote model where you never meet your team in person is difficult, especially when you have to figure out their strengths and aspirations. Plus test management in this context is something I was never sure about
because of the overall responsibility and worrying about the end
result and gathered information on product quality. I keep thinking that what if the mission fails due to
misunderstanding and misinterpretation of the goals and objectives? What if the
team energy and dynamics will not form correctly?
To my advantage I was able to participate in hundreds of uTest cycles as a tester and could observe and learn from other participants as well
as network and build relationships. Still, when it was time for me to launch my
first test cycle I was a nervous wreck even though most of the testers had been hand-picked. Now, imagine working a test run for wider device and OS coverage with over
30-40 testers who are located in different countries and time zones, speak
different native languages and think in their cultural values terms. Diversity
is a great strength in software testing. However, how do you harness and make
the best of it in a short 2-3 day turnaround while testing early beta builds? It
felt like testing on steroids sometimes with the discussion forum exploding
with questions and the number of bug reports growing like mushrooms. It was hard
to set up the right expectations with so many unknowns and partially
finished/functioning features. We did the best we could with what we had. We
kept learning about the product as it evolved and as a team as we continued
working together.
Eventually I came to the realization that having a bigger team
does not guarantee better results and that common risk patterns could be
identified on a smaller set of devices and ultimately with a smaller group of
designated testers. And so I convinced my boss to allow me to form two
designated teams for iOS and Android testing with 2 testers on each team. Mobile
app tester selection process was not hard for me as I followed a few true gems
in the community. The challenge was their willingness and availability as this testing involved working after normal business hours. Plus none of the folks that I wanted to recruit
resided in the US. I must have really wanted this to happen as all 4 testers that we wanted were available – 2 in Europe, one in India and another in Brazil. The iOS Special
Forces team sprinted first. Weekly test cycles with test goals and details
frequently updated in the platform, open communication via Skype and email, quick
feedback from the dev team and non-competition over the number of bugs
submitted allowed us to work effectively. The first couple of weeks were a bit
rough as we had to get used to testing internal alpha builds and lots of things
were still incomplete and ‘broken’. It was hard for the team to hold back and not log obvious
inconsistencies but we learned to restrain ourselves and shift focus to areas
that required our input and attention. For example, the testers provided constructive
and valuable usability and UX feedback and updated the regression test script
while waiting for new builds.
When people talk about a match made in heaven that’s what this
partnership felt like. It was truly amazing to see that people who never met,
live in different cultural and time zones can come together to test software
and have so much fun. And yes we debated and disagreed but we worked like a
band of brothers empowered and driven by the same goals, beliefs and
principles. We were doing what we love most, learning and delivering good value
for our customers. We craved working together as we progressed further into
testing more stable builds and new features. A native app life cycle is much
shorter than a web or desktop one – just 3-5 months. And so it was amazing to
watch the program evolve and be a part of the process from start to release.
Granted we did not like many things and disagreed with design and dev folks, and
advocated and pressed for change. Sometimes it worked and sometimes it didn't, and sometimes we were wrong as something we thought was a bug turned out to be
a feature. The whole experience was very empowering and rewarding.
And so to sum it up I’d like to think we did a great job. I managed to select the right people, set up goals and
expectations, empower and motivate, and create the atmosphere for learning and
self development. It felt really good.
No comments:
Post a Comment