Email: Git best practice - break up Pull Requests into smallish commits

Hi All,

We have talked about the size of commits, the size of PRs, etc. and here is an example where I’m creating a small/medium sized feature which touches a lot of parts of the code

• environment data
• test data
• helper methods
• extending existing classes
• the test method itself

My goal here is to show an example of breaking a PR up into small, related commits so that it’s easier to code review.  The whole feature has to go in 1 PR because all the parts of this PR depends on all the other parts, and together it’s several hundred lines of code.  But if you look at the individual commits, it’s easier to see what I was trying to do.  I did an ok job with this but surely could have done better, especially the last commit, which is pretty big.

An important thing here is that just because you have changed lot of files or even if you have changed several parts of a single file, you do not have to commit everything you have changed.  Sourcetree makes it easy to “stage” files or parts of a file which will get committed when you call commit (for more on staging, see here).  This way, even if you have a lot of changes, you can still make commits that group related pieces of code together to make life easier for the code reviewer – and hopefully for all of us since the reviewer is more likely to catch any issues with your code.

Automation Strategy: Flow vs Page Objects Part 1 - or - If You Think Page Objects are Enough, Think Harder

Motivation

The Page Object Model (POM) has been the de facto standard in UI automation using Selenium for years now.  Here* is a typical primer on the subject.  This primer does a perfectly reasonable job of showing how people normally use POM, and it also clearly shows the reason that using POM the way I've seen a lot of people use it is insufficient for writing maintainable tests.

The problem is illuminated in the author's 5th reason for using POM: "Any change in UI can easily be implemented, updated and maintained into the Page Objects and Classes."  If you've done this for a while, you'll notice what is not said here - that a change in business process or higher-level user interaction with the site cannot be easily implemented in our automation.  For that type of change, if we were to only use POM, we'd have to update all of the related tests. That is to say, this style of POM is sufficient for encapsulation and maintainability of UI elements and structure, but we need something higher-level to encapsulate and maintain business processes.

Typical POM

As a trivial example, let's say you have this PO class and test:

You're going to have hundreds of other tests that contain this same login code of course.  What happens when the id of the username field changes?  With POM, you're golden.  You just update the username element on your LoginPage class.  But what happens when there's a new step in logging in?  Now all of your tests have to change to look like:

As an automation developer, you can be certain that your site will change - and not just in rearrangement or renaming elements on a page. Some people will say "Hey, I've got a method called login() in my LoginPage class, so that's all I'll have to update." Well kudos, you're a big step closer, let's check it out. We have the login() method to our LoginPage class, so all we do is add the element and the line in the login() method. While we're at it, we'll put the goto() in there. That way our test will be super clean and maintainable:

You will be good to go for this trivial example. But what about when your company decides to change a business process in your app? Let's say you want to add a step in a checkout process such as buying a couch. In the past, you had the code below... you were smart so you even wrapped clicking submitButton with a method, not wanting to mix the use of PO methods like loginPage.login() and use of the PO elements themselves like loginPage.submitButton.click():

The Issue...

The problem is that we're still modeling the interaction with the UI - we're focused on the website. What we should be modeling is what the user wants to do. The user doesn't want to select from a dropdown on pageA and click some buttons on pageB, they're trying to buy a couch, and we're not encapsulating that anywhere, except arguably in the test. That's too late, because we're going to have dozens of really similar tests where the user just wants to do a tiny thing different, such as choosing express shipping vs. regular shipping, so all that test code is going to be duplicated, and we're not going to be DRY at all.

In this example our app changed, and now we have an "assembly and delivery" step. All our tests have to change again:

Flow classes

The Flow class is intended to do this encapsulation and abstraction for us.  It models what the user is trying to accomplish (user stories?) when they're using your application.  Let's check it out:


And now your tests can model the use cases, and they simply looks like:

So maybe the title of this post shouldn't be Flow vs Page Objects, because both layers are useful and work together. The POs encapsulate the UI, and the Flow encapsulates the business rules. Then the tests simply encapsulate the use cases. Honestly, if your app is small enough, like a mobile app, you can do both in the Flow class. I've done this successfully, but a UI refactor probably would have cost me more than if I'd done a proper PO layer.

Looking at the test case, you probably see the potential problems:

1. The more work we make our methods do, the more data we have to pass in to let them know how to do it. For this, we'll have to start consolidating data elements into their logical groups. I'll talk about that in the next post here.
2. Using flows works great for positive tests, but what about negative tests?  I'll talk about that in this post.

Continue...

The next post in this series is Automation Strategy: Flow vs Page Objects Part 2

*In case the site I referenced is deleted, here is a webArchive of it

Automation Strategy: Cucumber for Testing - You're (Very Possibly) Doing it Wrong

Potential Issues

The first time I worked on test automation was around 2006, and when I started seeing people using Cucumber I didn't get it.  I always thought there were a lot of problems with thinking that a human-readable English DSL (domain-specific language) was going to solve your problems.  The most immediate one was that English doesn't compile.  Enter Step Definitions, which take a lot of time to write, translating the English to the method calls in your code.  In addition, having everyone agree on the exact syntax of the Gherkin seemed like a huge challenge.  For the simplest case of being logged in, is it "Given I am logged in" or "Given I'm already logged in" or "Given I have logged in?"  All of this just to have tests written in English... no way.

Flows

I have been using Flows in addition to or instead of Page Objects in my frameworks for years, and I wanted to show someone a side-by-side comparison of my Flow methods calls vs Gherkin.  Here's the example from a test a few companies ago.  This (flow code) is actual code from the test.

Gherkin	Using Flows
Given a user is logged in	flow.login(user01)
When a user adds a pen to the cart	flow.add_to_cart(pen)
And the user checks out	checkout_total = flow.checkout()
Then the receipt is viewable	flow.verify_receipt_exists()
And the cart is empty	flow.verify_cart_is_empty()
And the receipt total equals the order total	flow.verify_receipt_total_equals(checkout_total)

The beauty of having this flow class is that with a decent IDE, you type "flow." and you'll be given a list of all the methods in the class, which is easier than remembering the exact Gherkin phrase.

I had been having these thoughts for a while.  I have never seen people using Cucumber but always wanted to.  Some people I know and respect say it's useful or that they've seen it done.  Then I saw this post* by Aslak Hellesøy, the author of Cucumber, and was so happy to have my beliefs confirmed.  It's a quick read, but basically he says if you're just using Cucumber for automated testing, you're missing the point.

Cucumber is a BDD tool, and the automated tests are a byproduct of the process.  If the testing team is the ones "doing Cucumber" you likely need to reevaluate your situation.  Using Cucumber correctly involves buy-in from product and development as well as QA.

For more on Flow vs Page Object, check our my post here.

*web archive of Aslak's post

Information Science: Visualizing x^4 and Higher Powers

Someone online asked "We can imagine x^2 as a square and x^3 as a cube. How can we imagine x^4 and other high powers?" and I had considered this thought experiment in college working with multidimensional arrays.  Many people get hung up thinking about physical dimensions of height, width, length and fail to realize that many attributes of an object or system can be considered a dimension.  So here's my answer:

Hopefully it's clear the upper left (of the 9) boxes is your first cube.  Then you can see you have arrays and eventually cubes of cubes (x^6). 

This works well for data representation, at least to x^5.  For example let's take x^4, and let our dimensions be length, width, height, and time:

We can see the length, width, and height at each point in time.  The object we're representing is getting smaller as time progresses.  Maybe it's an ice cube.

Maybe in the x^5 scenario, our 5th dimension could be temperature, so the top row could be some "medium temperature," the middle row could be "very hot" and the bottom could be "very cold" so the ice cube doesn't melt at all:

If you are working in higher order math, being able to visualize (and hopefully the need to visualize) goes out the window pretty quickly.

Automation Framework: Wrap it up!

TL;DR you should wrap Selenium, Appium, etc. with a class of your own, even if it's not adding much value at this moment.  

The simplest reason is that if today you're calling myButton.click() and you're using Selenium's click() method in a hundred different places in scripts and libraries, what if you want to change the behavior of click()?  Why would anyone ever want to do that?  There are a few different reasons including waiting before clicking, retrying after a failed click, or my favorite: logging.

Maybe you want to go into debug mode and log every click.  If you're using Selenium's click() method, you may be out of luck.  If you're using your own click method, which previously did nothing but call Selenium's click(), just add a logging line:

Automation Framework: Specifying Element Type such as Button/Link/Dropdown Considered Harmful

I have worked in 5 frameworks over the past 12 years and although I keep seeing it, I am still unconvinced that knowing the exact type of your element adds value.  If I want to click an Element, I call .click().  Knowing that it's a Button or Link certainly makes no difference because the interfaces to those element types are the same.  Knowing whether it's a Button or a Dropdown I still argue make no difference, my_element.click() will do what I want it to do.  It's like duck typing in Ruby.  In highly complex systems, having very robust typing may be helpful in avoiding mistakes and making the code easier to read, but most UI automation just isn't that complex.

If I want to know exactly what type of element I'm dealing with, I have to write a bunch of classes such as

class Button (webElement):
  def click():
     self.click()
  ...


class Dropdown (webElement):
  def getOptions():
    select = new Select(self.driver, self.by)
    return select.getOptions()
  ...


Or maybe we could group them into related groups (but how related is related?) such as

class

class Clickable (webElement):
  def click():
     self.click()

class Button (Clickable):
  pass

class Link (Clickable):
  pass
...

This way at least we're saving a little redundancy with shared functionality, but still, what's the point?  I'm all for writing a wrapper for whatever automation tool you're using (Selenium), but without being specific about types.  When the rubber meets the road - i.e. when you're testing your test script - you'll find out if you've done it right.  What I mean is if you think you have a Dropdown and you call getOptions() but you actually have a Link, it'll fail.  You don't have to write the extra code that allows you to call that object a Button or Dropdown object vs an Element object.  If you had that extra code, maybe it would've failed 10 seconds earlier when you tried to instantiate a Dropdown object but passed a button web element to it.

This leads to my next point:  the greater specificity you have in what type of Element you have, the LESS flexible your test is.  If you're testing your application's logic, you shouldn't care as much about clicking a button or a link.  What you should care about is what the user cares about: logging in, placing an order, viewing a document... some business value.  If you care about testing the UI and whether that web element is a button or a link then ok, maybe you should be specific, but I've never worked anywhere that this was the case.

This lack of flexibility will rear its head when your app undergoes a refactor.  When all those buttons turn to links as you move from an older-style UI to a newer one, your tests all break. Even if they don't, you now have all these elements called orderButton or showMapButton which are actually links in the UI.  If you had been using generic Element objects, there would have been no code to change.

QA Basics: Bug Priority vs Severity by Example

Severity is how adversely the bug affects the software or the user's experience.  Priority is simply the order in which a bug will be fixed.  This is most easily explained by example.

• High severity, high priority: Not being able to log in.  Major functionality is lost and the impact is critical.
• Low severity, low priority: A typo on some screen that users rarely use.  No functionality lost, low impact.
• Low severity, high priority: At Nike, the "Swoosh" is considered sacred. A backwards or upside down Swoosh is seen extremely rarely and if it's an accident, people over there LOSE THEIR MINDS. If a page incorrectly showed an upside down Swoosh, it would be low severity because the app still functions just fine, but it would be high priority because they would want it fixed immediately. 
• High severity, low priority: If your app has a feature with a button that, when clicked, crashes the whole app, that would be high severity.  If that feature is almost never used by any of your customers, it would be low priority.  If that sounds odd, believe me, there are millions of these kinds of bugs out there in the wild.  Known fatal bugs that no one has plans to fix.

Typically the QA Engineer will set the severity (and sometimes priority) when they log the bug, and then it will be reconsidered and possibly updated when the bug is triaged. Triage is a process, often weekly, in which a representative of QA, Dev, and Product get together and go through new bugs and decide if and when to fix them.  Priority may be assigned during triage or by the product team independently of others.