Friday, December 29, 2017

Automation Framework: Wrap it up!

TL;DR you should wrap Selenium, Appium, etc. with a class of your own, even if it's not adding much value at this moment.  

The simplest reason is that if today you're calling myButton.click() and you're using Selenium's click() method in a hundred different places in scripts and libraries, what if you want to change the behavior of click()?  Why would anyone ever want to do that?  There are a few different reasons including waiting before clicking, retrying after a failed click, or my favorite: logging.

Maybe you want to go into debug mode and log every click.  If you're using Selenium's click() method, you may be out of luck.  If you're using your own click method, which previously did nothing but call Selenium's click(), just add a logging line:

Automation Framework: Specifying Element Type such as Button/Link/Dropdown Considered Harmful

I have worked in 5 frameworks over the past 12 years and although I keep seeing it, I am still unconvinced that knowing the exact type of your element adds value.  If I want to click an Element, I call .click().  Knowing that it's a Button or Link certainly makes no difference because the interfaces to those element types are the same.  Knowing whether it's a Button or a Dropdown I still argue make no difference, my_element.click() will do what I want it to do.  It's like duck typing in Ruby.  In highly complex systems, having very robust typing may be helpful in avoiding mistakes and making the code easier to read, but most UI automation just isn't that complex.

If I want to know exactly what type of element I'm dealing with, I have to write a bunch of classes such as

class Button (webElement):
  def click():
     self.click()
  ...


class Dropdown (webElement):
  def getOptions():
    select = new Select(self.driver, self.by)
    return select.getOptions()
  ...


Or maybe we could group them into related groups (but how related is related?) such as

class

class Clickable (webElement):
  def click():
     self.click()

class Button (Clickable):
  pass

class Link (Clickable):
  pass
...

This way at least we're saving a little redundancy with shared functionality, but still, what's the point?  I'm all for writing a wrapper for whatever automation tool you're using (Selenium), but without being specific about types.  When the rubber meets the road - i.e. when you're testing your test script - you'll find out if you've done it right.  What I mean is if you think you have a Dropdown and you call getOptions() but you actually have a Link, it'll fail.  You don't have to write the extra code that allows you to call that object a Button or Dropdown object vs an Element object.  If you had that extra code, maybe it would've failed 10 seconds earlier when you tried to instantiate a Dropdown object but passed a button web element to it.

This leads to my next point:  the greater specificity you have in what type of Element you have, the LESS flexible your test is.  If you're testing your application's logic, you shouldn't care as much about clicking a button or a link.  What you should care about is what the user cares about: logging in, placing an order, viewing a document... some business value.  If you care about testing the UI and whether that web element is a button or a link then ok, maybe you should be specific, but I've never worked anywhere that this was the case.

This lack of flexibility will rear its head when your app undergoes a refactor.  When all those buttons turn to links as you move from an older-style UI to a newer one, your tests all break. Even if they don't, you now have all these elements called orderButton or showMapButton which are actually links in the UI.  If you had been using generic Element objects, there would have been no code to change.

QA Basics: Bug Priority vs Severity by Example

Severity is how adversely the bug affects the software or the user's experience.  Priority is simply the order in which a bug will be fixed.  This is most easily explained by example.
  • High severity, high priority: Not being able to log in.  Major functionality is lost and the impact is critical.
  • Low severity, low priority: A typo on some screen that users rarely use.  No functionality lost, low impact.
  • Low severity, high priority: At Nike, the "Swoosh" is considered sacred. A backwards or upside down Swoosh is seen extremely rarely and if it's an accident, people over there LOSE THEIR MINDS. If a page incorrectly showed an upside down Swoosh, it would be low severity because the app still functions just fine, but it would be high priority because they would want it fixed immediately. 
  • High severity, low priority: If your app has a feature with a button that, when clicked, crashes the whole app, that would be high severity.  If that feature is almost never used by any of your customers, it would be low priority.  If that sounds odd, believe me, there are millions of these kinds of bugs out there in the wild.  Known fatal bugs that no one has plans to fix.
Typically the QA Engineer will set the severity (and sometimes priority) when they log the bug, and then it will be reconsidered and possibly updated when the bug is triaged. Triage is a process, often weekly, in which a representative of QA, Dev, and Product get together and go through new bugs and decide if and when to fix them.  Priority may be assigned during triage or by the product team independently of others.