WHAT DOES IT MEAN TO BE A TEST ENGINEER? Andrii Dzynia http://bit.ly/1GYquoR
MY BACKGROUND • 8 years in software engineering • started as a test engineer in a team of 20 testers • currently working on improving product quality and development productivity at Spotify @adzynia https://www.linkedin.com/in/andriidzynia
SATURDAY EVENING https://twitter.com/mﬂoryan/status/655433887015948288
LET ME SHOW YOU THE PROCESS video: https://www.dropbox.com/s/un7m49kdgeuzsbz/test-sessionandrii.mov?dl=0
MY COLLEAGUE, FRONT-END DEVELOPER.
WHAT DID I LEARN? • Software always has bugs. If you have not found any, does not disprove their existence • Developers do not spend much time looking for edge cases, testers do
WHAT I WANTED YOU TO LEARN? Testing is not a set of artefacts.
TESTING IS A PROCESS. @adzynia
GUESS WHAT BUG MARCIN WAS REFERRING TO?
MORE LESSONS LEARNED • People are different. It is hard to predict every single combination when user can get stuck • Think slower, sometimes
WHAT MAKES YOU A TEST ENGINEER? Critical thinking Fault models Patterns recognition @adzynia
CRITICAL THINKING https://www.criticalthinking.org/ctmodel/logic-model1.htm
FAULT MODELS “From the model, the designer or user can predict the consequences of a particular fault.” - Wikipedia video: https://www.youtube.com/watch?v=rEumMkAhwzA
PATTERNS RECOGNITION https://www.nasa.gov
BASED ON EXPERIENCE BRAIN RECOGNISES SEQUENCES WHICH LEADS TO UNDESIRED OUTCOME. NOTE: BE CAREFUL WITH FAULTY ASSUMPTIONS. FACE ON THE MOON SURFACE IS PAREIDOLIA. “Pareidolia is a psychological phenomenon when the mind perceives a familiar pattern of something where none actually exists.” - Wikipedia
LOOK FOR PATTERNS WITH HIGH FAILURE RISK. video: https://www.youtube.com/watch?v=qoQssWPRNR0
BRINGING VIDEO TO SPOTIFY USERS http://techcrunch.com/2015/05/20/spotify-introduces-video-clipspodcasts-and-activity-based-playlists/
COMMUNICATION CAN BE A PROBLEM http://dilbert.com/strip/1989-05-31
SOLUTION WAS Create a #slack channel Start asking video integration questions Rise bugs early and facilitate conversations
BUT HOW DID THAT HELP? Integration ended up to be much easier Testing started earlier # of production issues was low
WAS THAT EASY? Not really. You have to ask questions people do not want to hear http://media2.giphy.com/media/wrTHBbla7suPu/giphy.gif
Tester: Shall video be playing when I lock a screen? Programmer: I do not know. Tester: who can know? Programmer: Product manager? Product manager: I do not know. Tester: who can know? Product manager: let’s take a look how Youtube does it. After a while we ﬁgured why video has to be paused when you lock a screen. Licensing constraints.
DEFECT PREVENTION VS DEFECT DETECTION “Bugs are so 90s”
LEAN MANUFACTURING https://en.wikipedia.org/wiki/Lean_manufacturing
KEEP YOU WORK ENVIRONMENT IN A GOOD SHAPE • engineering quality • product quality • strive for improvements
ENGINEERING QUALITY • Start testing as early as possible • Build safety nets • Fix bugs as fast as they occur • Conduct root-cause analysis and take actions http://www.slideshare.net/AndrewDzynia/quality-built-in
PRODUCT QUALITY • Product success metrics Time to ﬁrst video frame Number of users streaming video content Average viewing time, etc. • Number of production defects
SPOTIFY TEST CERTIFIED MODEL Inspired by Google https://mike-bland.com/2011/10/18/test-certiﬁed.html
TAKE AWAYS • Testing is a process, not a set of artefacts • Cost of defect can be lowered. Think about that • Find ways to inﬂuence quality on organization level. It is hard, but possible, even for a test engineer
THANK YOU! @adzynia [email protected]