Automation Strategy: Externalize All Your Strings

Motivation

We need to externalize all the strings we use in our locators, such as "User order number" or validation methods, such as "Order Created."  Why?

1. It'll make your code more maintainable if that string changes
2. It keeps things organized
3. Internationalization (i18n)
4. Multi-platform automation

From the outset, you might as well plan for i18n by including a switch for locale (not language). Language is "English."   Locale is "American English" and looks like "en_US" or "en-US" depending on who you ask.  Google it.  I like to model the strings exactly as they are, including capitalization and punctuation:

	def set_strings_generic(language):
	if (language == 'en_US'):
	set_strings_en_US()
	current_language = "English (United States)"
	elif (language == 'es_MX'):
	set_strings_ex_MX()
	current_language = "Español (Mexico)"
	...
	def set_strings_en_US():
	strings.Login_Here = 'Login Here'
	strings.No = 'No'
	strings.No_ = 'No.'
	strings.NO_WAY_ = 'NO WAY.'
	strings.Purchase = 'Purchase'
	strings.Welcome_ = 'Welcome!'
	...
	def set_strings_es_MX():
	strings.Login_Here = 'Entre aquí'
	strings.No = 'No'
	strings.No_ = 'No.'
	strings.NO_WAY_ = 'DE NINGUNA MANERA.'
	strings.Purchase = 'Compra'
	strings.Welcome_ = '¡Bienvenidos!'
	...

view raw strings_generic_01.py hosted with ❤ by GitHub

This works well for automating small to medium apps that themselves maximize code reuse.  But for big (or legacy) applications it may not be the best approach due to non-standardization, your mileage may very.

And for multi-platform automation... (there is a lot more work involved, but this is part of the deal...)

	def set_strings_generic(language, platform):
	if (language == 'en_US'):
	set_strings_en_US(platform)
	current_language = "English (United States)"
	elif (language == 'es_MX'):
	set_strings_ex_MX(platform)
	current_language = "Español (Mexico)"
	...
	def set_strings_en_US(platform):
	if (platform == "ios"):
	strings.System_Settings = 'System Settings'
	...
	if (platform == "android"):
	strings.System_Settings = 'Settings'
	...
	if (platform == "chrome"):
	strings.System_Settings = 'chrome://settings' # whatever...
	...
	def set_strings_es_MX(platform):
	...

view raw strings_generic_02.py hosted with ❤ by GitHub

Ok I'm going to be honest right now. I wrote the part above tonight and this next part 2 years ago and I don't even know what language this is. The filenames are .cs but I don't even know C# well enough to know if you can define methods like this, I might have just used .cs because gist colorization was good.  Hmm, now I'm thinking it's just pseudocode.

Anyway, the last thing I'll say about externalizing strings is about dynamic strings.
Here's option 1, strings live in the data file and you compose them in the page object class:

	//dataStrings file
	if (locale == "en_US")
	final String Order_number_ = "Order number "
	final String _is_on_its_way_ = " is on its way!"
	//page class file
	class MyPage
	void validateOrderOnItsWay(orderID)
	String validationString = dataStrings.Order_number_ + orderID + dataStrings._is_on_its_way_
	element e = driver.findElement(By.text(validationString))

view raw strings02.cs hosted with ❤ by GitHub

This is fine, but will require everyone who uses those strings to do all that string concatenation themselves, which is code duplication. It will also become a problem when the translation is weird. Maybe in American English we'd say "Order number 404 is on its way!" but in Canadian English they'd say "404 is the number of the order comin' at ya, eh!" If this were the case, our stringA + orderID + stringB wouldn't work, or would be wonky. Let's just write methods to generate those strings instead, and notice how it's used in the page class:

	// dataStrings file
	if (locale == "en_US")
	String orderNumberIsOnItsWay(orderID)
	return "Order number " + orderID + " is on its way!"
	if (locale == "es_CA")
	String orderNumberIsOnItsWay(orderID)
	return orderID + " is the number of the order comin' at ya, eh!"
	// page class file
	class MyPage
	void validateOrderOnItsWay(orderID)
	String validationString = dataStrings.orderNumberIsOnItsWay(orderId)
	element e = driver.findElement(By.text(validationString))

view raw strings03.cs hosted with ❤ by GitHub

And you could even get really crazy and do kind of a mix, which lets you have both the individual strings as well as the pre-composed dynamic strings. It just depends on what you need for your situation:

	//dataStrings file
	if (locale == "en_US")
	final String Order_number_ = "Order number "
	final String _is_on_its_way_ = " is on its way!"
	String orderNumberIsOnItsWay(orderID)
	return Order_number_ + orderID + _is_on_its_way_
	if (locale == "es_CA")
	final String Order_number_ = ""
	final String _is_on_its_way_ = " is the number of the order comin' at ya, eh!"
	String orderNumberIsOnItsWay(orderID)
	return orderID + _is_on_its_way_
	//page class file
	class MyPage
	void validateOrderOnItsWay(orderID)
	String validationString = dataStrings.orderNumberIsOnItsWay(orderId)
	element e = driver.findElement(By.text(validationString))

view raw strings04.cs hosted with ❤ by GitHub

Automation Strategy: Keep Going With Page Objects (Don't Stop at Locators)

Stopping at the Simplest Implementation of POM

I don't love the Page-Object Model (POM).  Actually, I just don't love the way many teams implement it.  When they stop at giving their page elements locators, they end up with elements that look like

	//great:
	processReturnsButton = new element(driver, By.id("process_return_button"));
	//sure, no problem (except I'd probably externalize the strings, see https://jws-testing-blog.blogspot.com/2018/09/automation-strategy-externalize-all.html)
	processTheseItemsButton = new element(driver, By.xpath("//a[text()='Process These Items']"));
	createReturnTextLabel = new element(driver, By.xpath("//span[text()='Create Return']"));
	processReturnsTextLabel = new element(driver, By.xpath("//span[text()='Process Returns']"));
	//not so great:
	identificationNumber = new element(driver, By.xpath("//table[starts-with(@id,'details_table_')]/tbody/tr/th[text()='Identification Number']"));
	orderTitleTextLabel = new element(driver, By.xpath("//*[@id=\"content\"]/form/div[1]/h2"));
	userTextLabel = new element(driver, By.xpath("//table[@class='order_notes']/tbody/tr/th[text()='User']"));
	roleTextLabel = new element(driver, By.xpath("//table[@class='order_notes']/tbody/tr/th[text()='Role']"));
	noteTextLabel = new element(driver, By.xpath("//table[@class='order_notes']/tbody/tr/th[text()='Note:']"));
	firstNoteAddedPstTimeTextLabel = new element(driver, By.xpath("//table[@class='order_notes']/tbody/tr[2]/td[1]"));
	secondNoteAddedPstTimeTextLabel = new element(driver, By.xpath("//table[@class='order_notes']/tbody/tr[2]/td[4]"));

view raw locators01.cs hosted with ❤ by GitHub

Reusing Page Objects in Other Page Objects

It's these last few that get me. Any time I see redundant code it makes me cringe. Let's level-up:

	//great:
	processReturnsButton = new element(driver, By.id("process_return_button"));
	//sure, no problem (except I'd probably externalize the strings, see https://jws-testing-blog.blogspot.com/2018/09/automation-strategy-externalize-all.html)
	processTheseItemsButton = new element(driver, By.xpath("//a[text()='Process These Items']"));
	createReturnTextLabel = new element(driver, By.xpath("//span[text()='Create Return']"));
	processReturnsTextLabel = new element(driver, By.xpath("//span[text()='Process Returns']"));
	//not so great:
	identificationNumber = new element(driver, By.xpath("//table[starts-with(@id,'details_table_')]/tbody/tr/th[text()='Identification Number']"));
	orderTitleTextLabel = new element(driver, By.xpath("//*[@id=\"content\"]/form/div[1]/h2"));
	//fancy!
	orderNotesTable = new element("//table[@class='order_notes']")
	//re-use! Notice the * syntax
	userTextLabel = new element(orderNotesTable, By.xpath("*/th[text()='User']"));
	roleTextLabel = new element(orderNotesTable, By.xpath("*/th[text()='Role']"));
	noteTextLabel = new element(orderNotesTable, By.xpath("*/th[text()='Note:']"));
	firstNoteAddedPstTimeTextLabel = new element(orderNotesTable, By.xpath("*/tr[2]/td[1]"));
	secondNoteAddedPstTimeTextLabel = new element(orderNotesTable, By.xpath("*/tr[2]/td[4]"));

view raw locators02.cs hosted with ❤ by GitHub

This can probably only be achieved if you've wrapped your Selenium tool's WebElement and WebDriver class. Hopefully you did this. It will take a little cleverness and adding some constructors, but it should be doable.

Dynamic Elements Using Methods, Not Variables

I find myself kicking the can down the road, or pushing the work down to a lower level like this all the time. "Make someone else do the work" I say. If I'm in the middle of writing a method and I think for one second that what I'm about to write could be used somewhere else, I try to encapsulate that work in a method. Better yet, I imagine what I'd pass into that method, what it would return to me, and continue writing my current method as if that helper method exists and keep going. Later on I'll write the helper methods into existence. 

In this case, while we're writing createAndViewOrder(), we don't want to worry about how we find the orderConfirmation element, we just want to assume it works and use it, so let's move it out of the method.  Before:

	//avoid having element specifics in your functional page methods
	class OrderPage
	public void createAndViewOrder(orderID)
	//create the order...
	element orderConfirmation = driver.findElement(By.xpath("//div[text()='Order " + orderID + " is confirmed.']"));
	orderConfirmation.waitUntilExists()
	//view the order...

view raw locators04.cs hosted with ❤ by GitHub

After:

	//make someone else do the work!
	class OrderPage
	public element confirmOrderCreation(orderID)
	return driver.findElement(By.xpath("//div[text()='Order " + orderID + " is confirmed.']"));
	public void createAndViewOrder(orderID)
	//create the order...
	confirmOrderCreation(orderID).waitUntilExists()
	//view the order...

view raw locators05.cs hosted with ❤ by GitHub

Does this buy you a lot right now? Not really. But

1. it might buy you something down the road during a refactor
2. it makes your functional method easier to read
3. it literally takes 15 extra seconds

Summary

It goes to a frame of mind that is good to get into when writing automation code - avoid UI specifics in functional methods, because you're likely to re-use that UI element again. There's a LOT of patterns when writing automation code from how your tests are constructed to how you find page objects to how you interact with the app, so you really have to be on guard and work against code duplication, this is just one of the ways.

Automation Strategy: Positively Do Negative Testing With should_fail()

Motivation

An old adage is "don't code for the exception," which is along the lines of Keep It Simple Stupid. This doesn't mean you don't expect and handle exceptions, bad data, etc., but that you should try to avoid extensive, complex coding for very rare cases.

I try to employ this in automation development just like any other software development, therefore when I write higher-level methods, I write them for the 95% case (the rule), and avoid the 5% case (the exception). For example, if I'm writing a login() method and after logging in, there's a popup that I will dismiss in 95% of my test cases, I will include the dismissal in the login() method. If I'm using a language that supports optional parameters, God forbid I have to work in a language that doesn't, maybe I'll take an optional parameter dismiss=True.

I'm actually quite proud of this should_fail() stuff.  I always aim to simplify the test scripts so they can be written by less technical people (hey, isn't that the point of cucumber/gherkin blah blah blah) and should_fail() aids in that as well as keeping your tests user-centric instead of UI-centric. This way, you can use your great Flow methods even though they feel like they're designed for positive-path testing.

For this example, we're logging in, then verifying we're on the Home page.  In this system, if the "passed" variable isn't true by the time the test is over, the test is considered to have failed.
Here's the positive test:

	class LoginPage()
	username = driver.TextField(:id, "username")
	password = ...
	submit = ...
	close_welcome_modal = ...
	def login(uname, pwd)
	username.type_text(uname)
	password.type_text(pwd)
	submit.click()
	close_welcome_modal.click()
	def verify_on_page()
	# assert that our username element is present, else throw an exception with Error message
	assertTrue (username.present(), "Error: we are not on the Login page")
	class HomePage()
	def verify_on_page()
	assertTrue (main_menu.present(), "Error: we are not on the Home page")
	class TestClass()
	def Test_Positive()
	lp = LoginPage()
	hp = HomePage()
	lp.login(data.uname, data.pwd) # valid user/pwd
	hp.verify_on_page() # we're on the homepage
	passed = true

view raw should01.rb hosted with ❤ by GitHub

Now For Negative Testing

Everything's hunky dory, which now that I type it is a very strange phrase. For the simple negative test, we'll try to login and then verify that we're still on the login page... that we have not advanced to the homepage (we could also look for an error message such as "User not found"). We can't use login() because it will try to dismiss that modal after we login and throw an ElementNotFoundException or something like that, so we'll have to use clicks:

	def Test_Negative_Using_Clicks()
	lp = LoginPage()
	lp.username.type_text("invalidUname", "invalidPwd")
	lp.submit.click()
	lp.verify_on_page() # verify we're still on LOGIN page
	passed = true

view raw should02.rb hosted with ❤ by GitHub

Using Exceptions the Simple Way

Well that doesn't look too bad, but it was very-UI centric, and as our automation gets more complex, maintainability of these negative tests is going to become a factor. If all our positive testing is done using Flows, we've thrown it out the window and reverted to using page objects. Ok, let's actually try it with login() and just catch the exception:

	def Test_Negative_Using_Try_Catch()
	lp = LoginPage()
	try
	lp.login("invalidUname", "invalidPwd")
	catch ElementNotFound e
	if (e.error_msg == "Error: we are not on the Login page")
	passed = true

view raw should03.rb hosted with ❤ by GitHub

Using should_fail()

That works, but it's a lot of code. Let's hide and generalize that code in our should_fail() method:

	def Test_Negative_Using_Should_Fail()
	loginPage = LoginPage()
	# the arguments are:
	# the object you're calling,
	# the method you're calling,
	# any data you're passing to the method (possibly multiple values),
	# the expected error message
	passed = should_fail(loginPage,
	login,
	"invalidUname",
	"invalidPwd",
	"close_welcome_modal not found")

view raw should04.rb hosted with ❤ by GitHub

Ok, there's definitely some syntax to learn there, but not too much. Let's see how should_fail() works:

	# Here's the magic! I'm not sure how to do this in java,
	# the params would probably have to be an arary or something like that.
	# This is a simplified pseudocode version of how I've done it in
	# ruby in the past. There are a couple simple gotchas I've omitted.
	def should_fail(*args)
	target_object = args[0]
	method_to_call = args[1]
	arg_array = args[2..-2] # i love ruby
	expected_error = args[-1]
	try
	target_object.call(method_to_call, *arg_array)
	catch Exception e
	# if we get the error we expect, just return:
	if (e.to_string().match(expected_error))
	return true
	#if we get an error we DON'T expect, re-throw it with an explanation:
	else
	err="Failed: we expected the error: '{expected_error}' but got a different error: {e.to_string()}"
	throw(RuntimeError, e.to_string())
	# We were expecting an error, so if we did NOT get an error then that's bad,
	# so we throw an error with fact that we DIDN'T see any error...
	# Or you might want to return false instead of throwing an exception,
	# depends on your system design
	throw(RuntimeError, "Failed: we expected the error '{expected_error}' but the call '{target_object}.{method_to_call}' succeeded")

view raw should05.rb hosted with ❤ by GitHub

One thing you might notice is the hardcoding of the error messages. Those would be in some kind of dataStrings.rb file or something. The other is that I'm throwing around exceptions here quite a bit. This is not necessarily something you can shove into a pre-existing system that doesn't use exceptions very much, or doesn't use custom exception messages. All my wrapper click() and find() and type_text() and everything else methods throw exceptions with clear, relevant messages in order to make debugging easier.

Summary and Tying in Flows

So again, the whole point of this is so you can write your negative tests using Flow methods. Let's say in the example below that returning an order is a multi-step process. If we were to write this negative test using the page objects only similar to my Test_Negative_Using_Clicks() above, we would be reverting to a long test that was very tied to the UI. So by using positive testing of negative tests, we can stay user-centric and high-level, even if your task is complex. In the example below we're going to try to return an non-returnable item:

	# negative test - we're going to try to return a NON-RETURNABLE item
	class TestClass()
	def Test_Negative_Return()
	order_flow = OrderFlow()
	return_flow = ReturnFlow()
	test_data = get_test_data(...)
	order_flow.login(test_data.username, test_data.password)
	order_id = order_flow.order(test_data.order)
	order_flow.logout()
	return_flow.login(test_data.username, test_data.password)
	# try to return a NON-RETURNABLE item
	should_fail.(return_flow, return, order_id, "Error: item is not returnable")
	passed = true

view raw should06.rb hosted with ❤ by GitHub

The last thing I'll say is the error I'm looking for here is very high-level. Sometimes it's difficult or tedious to make your Flow methods do this, so maybe in this example, we would have tried to click a button as part of the return process but it wasn't there or was disabled, etc. In that case, we'd just have to catch the more generic "... does not exist" exception, and this does tie us to the UI. The usefulness of this system, like all automation, depends on the context of the system you're testing.

Automation Strategy: Flow vs Page Objects Part 2 - Test Data Abstraction

In my first post on Flow vs Page Objects, it became clear that as we move up in layers of abstraction, our tests get shorter, but the data required to execute those tests remains the same. If we were just using page objects, we could do something simple such as...

	userFirstName = dataFile.getFirstName
	userLastName = dataFile.getLastName
	userStreet = dataFile.getStreet
	userCity = dataFile.getCity
	userState = dataFile.getState
	userZip = dataFile.getZip
	void updateAddressTest
	updateAddressPage.navigateTo()
	updateAddressPage.firstName.set(userFirstName)
	updateAddressPage.lastName.set(userLastName)
	updateAddressPage.street.set(userStreet)
	updateAddressPage.city.set(city)
	updateAddressPage.state.set(state)
	updateAddressPage.zip.set(zip)
	updateAddressPage.submitButton.click()
	assert(updateAddressPage.message == "Address updated", "Error: address not updated)

view raw data01.cs hosted with ❤ by GitHub

Using Flows

...but again, we want to focus on the user, and if you asked the user what they do on this page, they wouldn't say "This is the page where I put in my first name, then last name, then street, then city, then state, then zip code," they would say "This is where I put in my address." So let's do that:

	userFirstName = dataFile.getFirstName
	// and the rest...
	void updateAddressTest
	addressFlow.updateAddress(userFirstName, userLastName, userStreet, userCity, userState, userZip)
	addressFlow.validateAddress()

view raw data02.cs hosted with ❤ by GitHub

Maintenance

This is going to get overwhelming quickly, on top of which it's not maintainable. You might think the structure of a user's address doesn't change much, but maybe it's a voting website and they now require you to put in your state representative district number. Yeah, it can and will change. It needs to be maintainable. Now all our address tests need to be updated.

If you ask a user what their address is, they don't say "My street is 123 Main St. My city is Austin. My state is Texas," they say "My address is 123 Main St, Austin, Texas." So instead of passing in all the pieces of information that form an address, let's just pass in an address:

	userAddress = dataFile.getAddress
	void updateAddressTest
	addressFlow.updateAddress(userAddress)
	addressFlow.validateAddress()

view raw data03.cs hosted with ❤ by GitHub

Maintenance Moves from Tests to the Data Source and Related Classes

The trick here of course is that our data source is going to have to get smarter. That's up to you, whether it's coming from a file, a DB, XML, JSON, who knows. The point is, when we add the state rep district number, the things that have to change are:

1. The data itself.  And yes, if we have 1 data file for every test, then yes, we'll have to update a bunch of files.  This will have to be done in any system, and if you reuse the same data file in N tests, then it won't be a ton of files to update.
2. The code to get the data from the file.  If well designed, this should be trivial, and if very well designed, this might not have to change at all.
3. The classes or containers (such as Structs in C++) that hold the data. This should be trivial.
4. The PO class needs a new object for the new UI element. Trivial.
5. The Flow code that actually enters the data into the UI using that PO. Trivial.

Let's say our data file looks like this, then we add the district number to the end:

	"Joe", "Jones", "123 Main St.", "Austin", "Texas", "78704", "12"

view raw data05.cs hosted with ❤ by GitHub

and our code for holding the data looks like this, then all we have to do is update that last line:

	class Address
	String firstName
	String lastName
	String street
	String city
	String state
	String zip
	String districtNumber // new line

view raw data04.cs hosted with ❤ by GitHub

and our Page class (the Flow doesn't do much in this trivial case):

	class AddressFlow
	void updateAddress(address)
	updateAddressPage.navigateTo()
	updateAddressPage.updateAddress(address)
	void validateAddress
	assert(updateAddressPage.message == "Address updated", "Error: address not updated)
	class updateAddressPage
	void updateAddress(address)
	updateAddressPage.navigateTo()
	updateAddressPage.firstName.set(address.firstName)
	updateAddressPage.lastName.set(address.lastName)
	updateAddressPage.street.set(address.street)
	updateAddressPage.city.set(address.city)
	updateAddressPage.state.set(address.state)
	updateAddressPage.zip.set(address.zip)
	updateAddressPage.districtNumber.set(address.districtNumber) // new line
	updateAddressPage.submitButton.click()

view raw data06.cs hosted with ❤ by GitHub

Payoff

So that's it. You're going to have to update the data and how the data gets input into the UI regardless, there's no magic bullet for those. But this way, your tests don't change. Again, this is a simple example. The more complex it gets or the bigger a change to the application's workflow, the more this extra abstraction will pay off.

The last thing I'll say is this: notice we're updating the address, but the flow isn't updateAddressFlow, it's just addressFlow. I encourage grouping related functionality together. In fact, where we're doing this now, for the smaller of our dozen or so applications, there's just a single flow for each application. How you break that up is a trade-off you make between having tons of code files and having tons of code in just a few files.