BDD Behave framework

BDD-Behave + Python + Selenium + PyCharm IDE. 

It is automation framework that i am working on. I can say that it is a good test automation framework design and I am really proud of  what we have been building using this framework.

So why I think it is a good framework:

  • BDD approach. Develop the test case first, then write the code
  • Red/Green/Refactor cycle
  • Handle test scenarios, scripts and data separately. Separate locator from page objects.
  • Follow coding standards, no thread.sleep, explicit wait for page synchronization.
  • Strictly on using test design pattern and principles
  • Script/Framework version control using Git
  • Integrate into CI tool Jenkins
  • Ability to perform multiple types of testing : smoke tests, full regression tests, API tests.
  • Ability to run specific test scenario or group of tests using tags.
Advertisements

Flaky Tests

Have you ever seen the test sometimes passed and sometime failed with the same code and configuration during testing ?

I have facing this in every projects I worked. Lets talk about what is flaky tests and how to handle it in your automation test suite.

“Flaky tests are tests that fail intermittently. This can happen if there is an underlying bug that only happens intermittently. Or it could also happen if a test is timing-dependent and affected by environmental conditions such as the speed of the machine running the test.” ___ https://github.com/nodejs/node/wiki/Flaky-tests ___

When the test that are marked as flaky, its status is set to unstable (yellow) instead of failed (red). If we already know which tests in our test suite are Flaky test, the failure may be considered to reduce from fatal to warning level.

In my project which using BDD to write test scenarios, we run the feature file first then we have a step to rerun all failed tests in that feature if any. It makes sure some failures like timing will be rerun and the status will be updated.

I faced some flaky tests that i dont even know the reason why it is failed sometimes. It may because of computer resources. i dont know… In this case, i manually check the functionality of that test and make sure nothing is broken and ignore the flaky test status.

What to do when you encounter a new flaky test.

Flaky tests should be fixed to be made non-flaky. Usually this means fixing the underlying bug, or changing the test so that it doesn’t depend on timing.

Below is nice post on Google Testing blog:  https://testing.googleblog.com/2016/05/flaky-tests-at-google-and-how-we.html

 

 

 

 

Quality Engineer vs Automation Engineer vs SDETs

What is Quality Engineer ? here is CSQE definition:

The Certified Software Quality Engineer understands software quality development and implementation, software inspection, testing, verification and validation, and implements software development and maintenance processes and methods.

What Are the Core Competencies of a CSQE?

  • Enhanced fundamental understanding of quality philosophies, principles, methods, tools, standards, organizational and team dynamics, interpersonal relationships, professional ethics, and legal and regulatory requirements.
  • Thorough understanding of systems architecture and how to implement software development and maintenance processes to reduce the risks associated with various software development methodologies.
  • Define, select, and apply product and process metrics and analytical techniques, and have a sound understanding of measurement theory and how to communicate results.
  • In-depth understanding of verification and validation processes, including early software defect detection and removal, inspection, and testing methods.
  • Advanced knowledge of configuration management processes, including planning,
    configuration identification, configuration control, change management, status accounting, auditing, and reporting.

 

 

Reference links:

http://www.getreskilled.com/what-is-a-quality-engineer/

 

 

 

QA(PQA, SQA) vs QC

Some companies usually interchange the term QA and QC, so many people think QA and QC are same and interchangeable but this is not true.
Both are tightly linked and sometimes it is very difficult to identify the differences.
Fact is both are related to each other but they are different in origins.

In my current project, QA is acting like Process QA. My role is automation QC, it is like Software QA which aims to identify and fix defects.

Typically , PQA will work with the whole software development team and audit how the team followed the process every sprint (2 weeks per sprint) , some specific projects will be audited quarterly.

Quality Assurance vs Quality Control

Quality Assurance Quality Control
It is a process which deliberate on providing assurance that quality request will be achieved. QC is a process which deliberates on fulfilling the quality request.
A QA aim is to prevent the defect. A QC aim is to identify and improve the defects.
QA is the technique of managing the quality. QC is method to verify the quality.
QA does not involve executing the program. QC always involves executing the program.
All team members are responsible for QA. Testing team is responsible for QC.
QA e.g. Verification. QC e.g. Validation.
QA means Planning for doing a process. QC Means Action for executing the planned process.
Statistical Technique used on QA is known as Statistical Process Control (SPC.) Statistical Technique used on QC is known as Statistical Quality Control (SPC.)
QA makes sure you are doing the right things. QC makes sure the results of what you’ve done are what you expected.
QA Defines standards and methodologies to followed in order to meet the customer requirements. QC ensures that the standards are followed while working on the product.
QA is the process to create the deliverables. QC is the process to verify that deliverables.
QA is responsible for full software development life cycle. QC is responsible for software testing life cycle.

Key Points

  • In QA, processes are planned to evade the defects.
  • QC agreements with discovery the defects and modifying them while making the product.
  • QA detects weakness.
  • QC detects defects.
  • QA is process oriented
  • QC is product oriented.
  • QA is failure prevention system.
  • QC is failure detection system.

Reference links:

http://www.softwaretestinghelp.com/quality-assurance-vs-quality-control/

https://itviec.com/blog/qc-la-gi-qa-la-gi/

https://www.guru99.com/quality-assurance-vs-quality-control.html

Common GIT scenarios in my project

There are three main sections of a Git project: the Git directory, the working tree, and the staging area.

The Git directory is where Git stores the metadata and object database for your project. it is what is copied when you clone a repository.

The working tree is a single checkout of one version of the project. These files are pulled out of the compressed database in the Git directory and placed on disk for you to use or modify. It is your scratch space, used to easily modify file content.

The staging area is a file, generally contained in your Git directory, that stores information about what will go into your next commit. Its technical name in Git parlance is the “index”, but the phrase “staging area” works just as well.

The Three States in Git : modified(deleted), staged and committed.

The HEAD in Git is the pointer to the current branch reference, which is in turn a pointer to the last commit you made or the last commit that was checked out into your working directory. That also means it will be the parent of the next commit you do. It’s generally simplest to think of it as HEAD is the snapshot of your last commit.

The Index is your proposed next commit. Git populates it with a list of all the file contents that were last checked out into your working directory and what they looked like when they were originally checked out. It’s not technically a tree structure, it’s a flattened manifest, but for our purposes it’s close enough. When you run git commit, that command only looks at your Index by default, not at anything in your working directory. So, it’s simplest to think of it as the Index is the snapshot of your next commit.

git_tree

Scenario: you pushed a commit to remote repository (eg: github) with your credential
but you see someone else name instead of your name.
reason:  Git always takes the name and email in git config –global when pushing a commit , no matter what credential you entered.

$ git config --list
user.email=hoangthach2502@gmail.com
user.name=Thach Hoang

how to config this:

git config --global user.email "email@example.com"
git config --global user.name "Thach HOang"

Scenario: you want to download a repository in local , so that you can work on it.

git clone https://github.com/libgit2/libgit2

Scenario:  add one file , modify one file , remove one file, then commit and push to remote repo

$ git status
 Changes not staged for commit:
    modified: README.md
    deleted: oldfile.md
 Untracked files:
    newfile.txt
$ git add README.md
$ git rm oldfile.md
$ git add newfile.txt

$ git commit -m "First commit"
$ git push origin master  (git push  )

Scenario:  you already have a commit. now you find out that  you missed something and need to add it into that commit.

how:  update what you want and create new commit with option –amend. it will replace previous commit:

$git commit --amend -m "This is new commit that will replace old one"

Scenario: you modified and messed up a file but not yet in stage. you want to revert it back to original.

git checkout README.txt

Scenario: you already added/removed wrong file into stage. now you want to unstage it , you don’t want that file in the next commit.

git reset HEAD README.txt

Scenario: you already committed the change. now you want to undo the committed and leave file change in the stage(INDEX).

git reset --soft HEAD^

Scenario: Compare modified files
(feature-1100)$git diff develop

Create a branch

git checkout -b 

Scenario:  Update the branch and resolve conflict if any using rebase.

(feature-1100)$git checkout develop 
(develop)$git pull
(develop)$gc feature_1100
(feature-1100)$git rebase develop
resolve conflict then continue the rebase process:
(feature-1100)$git rebase --continue

Scenario:  Squash N last commits into one commit. It make resolving conflict easier.

git rebase --interactive --autosquash HEAD~N

GIT STASH

Scenario: you are working on branch feature-A with changes. Now you need to switch to branch feature-B to do something urgent. git stash help you store the change of feature A and allow you to switch to another branch. you can return to branch feature-A and restore the changes from stash.

Put in stash:

git stash save "Message"

Show stash:

git stash list

Show stash stats:

git stash show stash@{0}

Show stash changes:

git stash show -p stash@{0}

Use custom stash item and drop it:

git stash pop stash@{0}

Use custom stash item and do not drop it:

git stash apply stash@{0}

Delete custom stash item:

git stash drop stash@{0}

Delete complete stash:

git stash clear

…to be update…

Reference links:

http://ndpsoftware.com/git-cheatsheet.html#loc=remote_repo;

https://git-scm.com/book/en/v2/

http://lennonjesus.github.io/git-cheatsheet/

https://gist.github.com/hofmannsven/6814451

 

 

Common Basic and Advanced Xpath/CSS Selectors

Here are the list of common basic to advanced selectors which I usually use in my test code:

I. Basic

1. DIRECT CHILD
A direct child in XPATH is defined by the use of a “/“, while on CSS, it’s defined using “>”
Eg: xpath= //div/a ; css= div > a
Note: Single slash ‘/’ anywhere in xpath signifies to look for the element immediately inside the parent element.
Double slash ‘//’ signifies to look for any child or nestedchild element inside the parent element.

2. CHILD OR SUBCHILD
If an element could be inside another or one it’s childs, it’s defined in XPATH using “//” and in CSS just by a whitespace.
Eg: xpath= //div//a ; css= div a

3. ID
An element’s id in XPATH is defined using: “[@id=’example’]” and in CSS using: “#”
Eg: xpath = //div[@id=’example’]//a ; css= div#example a

4. CLASS
For class, things are pretty similar in XPATH: “[@class=’example’]” while in CSS it’s just “.”
Eg: xpath= //div[@class=’example’]//a ; css=div.example a

II: Advanced

1. NEXT SIBLING
This is useful for navigating lists of elements, such as forms or ul items.
The next sibling will tell selenium to find the next adjacent element on the page that’s inside the same parent.
Let’s show an example using a form to select the field after username.
css=form input.username + input

2. ATTRIBUTE VALUES
If you don’t care about the ordering of child elements, you can use an attribute selector in selenium to choose elements based on any attribute value.
A good example would be choosing the ‘username’ element of the form without adding a class.
css=form input[name=’username’]

We can even chain filters to be more specific with our selections.
css=input[name=’continue’][type=’button’]

3. CHOOSING A SPECIFIC MATCH
CSS selectors in Selenium allow us to navigate lists with more finess that the above methods.
If we have a ul and we want to select its fourth li element without regard to any other elements, we should use nth-child or nth-of-type.
<p>Heading</p>
Cat
Dog
Car
Goat

If we want to select the fourth li element (Goat) in this list, we can use the nth-of-type, which will find the fourth li in the list.
css=ul#recordlist li:nth-of-type(4)

On the other hand, if we want to get the fourth element only if it is a li element, we can use a filtered nth-child which will select (Car) in this case.
css=ul#recordlist li:nth-child(4)

Note, if you don’t specify a child type for nth-child it will allow you to select the fourth child without regard to type.
This may be useful in testing css layout in selenium.
css=ul#recordlist *:nth-child(4)

4. SUB-STRING MATCHES
CSS in Selenium has an interesting feature of allowing partial string matches using ^=, $=, or *=. I’ll define them, then show an example of each:

^= Match a prefix
css=a[id^=’id_prefix_’]
=> A link with an “id” that starts with the text “id_prefix_”

$= Match a suffix
css=a[id$=’_id_sufix’]
=> A link with an “id” that ends with the text “_id_sufix”

*= Match a substring
css=a[id*=’id_pattern’]
=> A link with an “id” that contains the text “id_pattern”

5. XPATH – USING TEXT OF ELEMENT
//div[@class=’homepage-hero’]//a[text()=’Enroll now’]

6. XPATH – USING CONTAINS
Syntax: //tag[contains(attribute, ‘value’)]
//div[@id=’navbar’]//a[contains(@class,’navbar-link’) and contains(@href,’sign_in’)]

7. XPATH – TRAVERSE UP TO PARENT:
Syntax: xpath-to-some-element//parent::<tag>

8. XPATH – Preceding Sibling
Syntax: xpath-to-some-element//preceding-sibling::<tag>

9. XPATH – Following Sibling
Syntax: xpath-to-some-element//following-sibling::<tag>

Sources:
https://saucelabs.com/resources/articles/selenium-tips-css-selectors

[ERROR] COMPILATION ERROR when running maven test

Problem: Error below is shown during run maven test:

[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] No compiler is provided in this environment. Perhaps you are running on a JRE rather than a JDK?
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.424s
[INFO] Finished at: Tue Oct 29 15:21:01 IST 2013
[INFO] Final Memory: 5M/15M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler- plugin:3.1:compile (default-compile) on project TEST-WEB: Compilation failure
[ERROR] No compiler is provided in this environment. Perhaps you are running on a JRE rather than a JDK?

Solution:

On your Eclipse IDE, go into Window > Preferences > Java > Installed JREs > and check your installed JREs. You should have an entry with a JDK there.

Eclipse_IDE

Control Panel > Java > Java Control Panel > Java > View > change to jdk.

java
Then Right-Click on your Project -> Maven -> Update Project

 

Generate/search cssSelector and Xpath on Firefox 55 onward

As you know, Firebug extension is no long working on Firefox 55 and later. The latest version of Firebug is 2.0.19 which compatible with Firefox 30 – 54. So you may want to install the old Firefox build 54 , disable auto update and happy with the latest Firebug.

But it is quite inconvenience. So I will share you 2 tools that we can do the same functionalities of Fire Bug such as : inspect Element, generate css Selector or xpath of element, search an element using css/xpath…

The first tool is Firefox DevTools .This is the tool suggested by http://www.getfirebug.com .

“The Firebug extension isn’t being developed or maintained any longer. We invite you to use the Firefox DevTools instead, which ship with Firebug.next”

generate locator

How to use Firefox Developer tool:

  • Press F12 or right click on element and select “Inspect Element”.
  • Click icon pickE  to pick an element from the page. Then you can right click on Element in Inspector tab and select Copy (CSS Selector, XPath ) as you see in the screenshot above. however the XPath generated here is absolute XPath . so it is not useful for me.
  • Search Element using css selector: Firefox Dev tool only allows us to search an element using CSS Selector. searching an Element using XPath is not supported yet. There is an enhancement S2 for this Bug 963933 . I hope they will add this feature soon.

XPath locator is very important for us. It is quite handy to handle some elements which there is no ID and cssselector cannot help. you have to do an XPath search, then walk up the DOM, then back down to get to the element you want.

Fortunately , there is another extension on Firefox can help us to generate and search xpath locator. It is AutoPager.

(this add-on also help to generate xpath: https://addons.mozilla.org/en-us/firefox/addon/element-locator-for-webdriv/)

AutoPager

How to use AutoPager Extension:

  • Generate xpath of Element: right click on Element on the web page, select Auto Pager >XPath Utils > Create xpath. It will show the AutoPager window. you can get the xpath in Link XPath field. It also have a list of possible xpath and the number of match found. Make sure you select the one which have only one match found for your testing. You also can click on find button pick and pick an element on web page to generate xpath for it.
  • Search xpath:  This is the feature that i use frequently . it is very handy. You can define your own xpath and paste it into Link XPath and enter to check if there is any element match with your defined xpath. Click … on the right of Link Xpath field, it will show you all functions used to define an xpath. I love it.  When you search xpath , make sure you are on the correct Frame.

 

copyxpath_AutoPager
XPath Utils
generate xpath of element
Generate an Xpath for Element
XPath Expression
XPath Expression: help us to define Xpath
Frame

Handle Native Windows upload popup using Robot and Clipboard class

When we click on upload button on a web page, an native windows popup will be displayed and we need to give the path to the file we want to upload. Selenium WebDriver cannot handle this popup.

There are two solutions for this scenario.

  1. Using Java Robot and Clipboard classes to simulate copy and paste action. we can paste the file path in the popup window and simulate enter key event.
  2. Using AutoIT tool to handle the pop up.

In this post, I will show you how to handle windows upload popup using the first solution.

1. import these class into your test script:

import java.awt.AWTException;
import java.awt.Robot;
import java.awt.datatransfer.StringSelection;
import java.awt.event.KeyEvent;

2. Create instance of Robot class and Clipboard class
Robot robot = new Robot();
Clipboard clipboard = Toolkit.getDefaultToolkit().getSystemClipboard();

3. use StringSelection to cache the upload file path string.

StringSelection filepath = new StringSelection(“F:\\DOCUMENT\\SELENIUM\\myCV.doc”);

4. Copy the file path to Clipboard

clipboard.setContents(filepath , null);

5. Use Robot class to simulate CTRL+C and CTRL+V key events :

robot.keyPress(KeyEvent.VK_CONTROL);
robot.keyPress(KeyEvent.VK_V);
robot.keyRelease(KeyEvent.VK_V);
robot.keyRelease(KeyEvent.VK_CONTROL);
robot.keyPress(KeyEvent.VK_ENTER);
robot.keyRelease(KeyEvent.VK_ENTER);

below is full script to send an email with attachment:

gmail_upload

package mypacket;

import java.awt.AWTException;
import java.awt.Robot;
import java.awt.datatransfer.StringSelection;
import java.awt.event.KeyEvent;
import java.util.concurrent.TimeUnit;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.testng.annotations.AfterTest;
import org.testng.annotations.BeforeTest;
import org.testng.annotations.Test;
import java.awt.Toolkit;

public class SendGmailwithRobot {
 WebDriver driver;
 String baseURL;

@BeforeTest
 public void beforeTest() {
 System.setProperty("webdriver.gecko.driver", "F:\\SOFTWARE\\Automation\\geckodriver.exe");
 baseURL = "https://www.google.com/gmail/about/";
 driver = new FirefoxDriver();
 driver.manage().window().maximize();
 driver.manage().timeouts().implicitlyWait(40, TimeUnit.SECONDS);
 driver.get(baseURL);
 }

@Test
 public void sendGmail() throws InterruptedException, AWTException {
 driver.findElement(By.linkText("Sign In")).click();
 driver.findElement(By.id("identifierId")).sendKeys("yourgmail@gmail.com");
 driver.findElement(By.cssSelector("span.RveJvd")).click();
 Thread.sleep(3000);
 driver.findElement(By.id("password")).sendKeys("your gmail password");
 driver.findElement(By.id("passwordNext")).click();
 driver.findElement(By.xpath("//div[text()='COMPOSE']")).click();
 driver.findElement(By.xpath("//div[@class='wO nr l1']//textarea")).sendKeys("destinationemail@gmail.com");
 driver.findElement(By.xpath("//input[@name='subjectbox']")).sendKeys("email title here");
 WebElement attachbt = driver.findElement(By.xpath("//div[@class='a1 aaA aMZ']"));
 attachbt.click();

Robot robot = new Robot();
Clipboard clipboard = Toolkit.getDefaultToolkit().getSystemClipboard();
StringSelection filepath = new StringSelection("F:\\DOCUMENT\\SELENIUM\\myCV.doc");
clipboard .setContents(filepath, null);

 robot.keyPress(KeyEvent.VK_CONTROL);
 robot.keyPress(KeyEvent.VK_V);
 robot.keyRelease(KeyEvent.VK_V);
 robot.keyRelease(KeyEvent.VK_CONTROL);
 robot.keyPress(KeyEvent.VK_ENTER);
 robot.keyRelease(KeyEvent.VK_ENTER);

driver.findElement(By.xpath("//div[text()='Send']")).click();
 }

@AfterTest
 public void afterTest() {
 driver.close();
 }
}