Process SDKs - .Net

  

Installing Applitools Eyes SDK for Dot.Net

Applitools Eyes SDK for Selenium .Net can be installed from Nuget (the name of the package is Eyes.Selenium.DotNet).

To install the SDK using the Nuget package manager console, run the following command in the console (from Visual Studio 2012, you can use the ‘Nuget Package Manager’ as an alternative and simply search for the package and install it):

 

Install-Package Eyes.Windows

 

 

Using the Applitools Eyes SDK

 

Using the Eyes SDK involves 4 main steps:
  • Create and config an Eyes object.
  • Start a test.
  • Add UI validation check points.
  • Close the test.

Let’s look at a short .Net example for creating a single test:

 

using Applitools.Windows;
using System;
using System.Diagnostics;

namespace windowsTest
{
    class TestApplitools
    {
        static void Main(string[] args)
        {
            //Loading notepad
            Process notepad = Process.Start(Environment.ExpandEnvironmentVariables(@"%WINDIR%\system32\notepad.exe"));
            //Thread.Sleep(2000) //Might be required if running too fast.

            Eyes eyes = new Eyes();
            // This is your api key, make sure you use it in all your tests.
            eyes.ApiKey = "YOUR_API_KEY";

            // Define the OS and hosting application to identify the baseline
            eyes.SetAppEnvironment("Windows 7", "Notepad test");

            try
            {
                // Start visual testing
                eyes.Open(notepad.Id, "Notepad", "Testing main notepad window");

                // Visual validation point
                eyes.CheckWindow("Notepad start");

                // End visual testing. Validate visual correctness.
                eyes.Close();
            }
            finally
            {
                // End visual testing. Validate visual correctness.
                eyes.AbortIfNotClosed();

                notepad.Close();
            }
        }
    }
}

 

 

Eyes Object

Eyes object is the main object in Applitools Eyes SDK.

 

Creation

 

var eyes = new Eyes();


 API Key

Before anything else, you should set the API key which allows you work with the Eyes server. Note that you use a static function for that, so it will be available to all of your objects. In order to obtain your API Key, login to Applitools Eyes web application and open your account details page.

 

eyes.ApiKey = "MY_API_KEY";

 

Replace MY_API_KEY with the API key of your account (you can find it in your signup Email).

 

Initialization

The open method returns a custom web driver object that monitors all the driver actions. From this point, the new custom driver object should be used for all the actions in the test.

For Ruby users, the initialization and the section for ending a test can be replaced with a call to eyes.test with the same parameters as eyes.open which handles initialization and ending the test.

driver = eyes.Open(driver, appName, testName, viewportSize);

driver – the relevant Selenium Webdriver object.

appName – string that represents the logical name of the AUT (this name will be presented in the test result)

testName – string that represents the name of the test (this name will be presented in the test result

viewportSize – a predefined viewport size that will be used for the browser window during this test (this parameter is very important for visual tests to ensure that the test will run of a defined viewport size and the results will be consistent)

appWindow (only relevant for CodedUI) – the WinWindow or BrowserWindow object (when using BrowserWindow, a casting to WinWindow is required) of the application under test.

 

Visual validation checkpoint

 

The check window command will take a snapshot of the current window in the AUT and will perform smart visual comparison of that window with the baseline. In case this is a new window that does not exist in the baseline, this window will be added to the baseline.

 

eyes.CheckWindow(matchTimeout, windowName);

 

windowName – string that represents the logical name of this window/validation point that will appear in the test result report.

matchTimeout - (optional) MatchTimeout in seconds, given for the step to pass.

 

Using OCR

In .Net SDKs it is possible to get OCR resulting text from given region(s). In order to extract text from a region start by calling ‘InRegion(...)’ and after that call ‘GetText()’.

String ocrText = eyes.InRegion(region).GetText();

 

It is possible to chain few regions from the same screen together by calling ‘and(...)’ after InRegion:

 

String ocrText = eyes.InRegion(region1).And(region2).GetText();

Note, When chaining few regions together as shown above, the OCR analysis will run in parallel so its usually should be faster than running every region individually.

region/region1/region2 - Region selectors to take text from. Each region must contain english uniform text. Region selector can be one of the types ‘System.Drawing.Rectangle’ (All SDKs) or ‘OpenQA.Selenium.By’ (Selenium-eyes SDK).

ocrText - The resulting text from OCR analysis.


 

Ending a test

At the end of each test, make sure to call the eyes.close method to notify the service that the test completed successfully. It is recommended to also call the method eyes.abortIfNotClosed at the end of the test as part of the ‘finally’ block to distinguish between completed tests and tests which were aborted abnormally (e.g. an exception was thrown).

...
    eyes.Close();
}finally{
    eyes.AbortIfNotClosed();
}

In the “finally” block of Selenium tests, you would usually also want to include “driver.quit()”, since the test has either finished or failed.

 

 

(Optional) Creating a batch of tests

The object BatchInfo represent a batch/collection of tests that will be unified as one group in the Test Manager screen and the result of all tests in a batch will determine the batch result. In order to start a batch, you should create a BatchInfo object, and associate it to the Eyes object (before calling the “eyes.open”). In order to add additional tests to the same batch, make sure to call eyes.setBatch for each of the tests/Eyes objects (so every test in the batch should start with creating an Eyes object, calling eyes.setBatch, eyes.open and ending the test with eyes.close).

 

 

var mybatch = new BatchInfo(batchName);
 
eyes.Batch = mybatch;

batchName – string that represents the logical name that will be assigned to this batch.

Use the command eyes.setBatch to associate a test to the batch using the eyes instance

 

 

(Optional) Enabling Logs

 

The logHandler method of the Eyes object enables logging for troubleshooting purposes.

eyes.SetLogHandler = new StdoutLogHandler(true);

 

(Optional) Working with branches

 

Applitools Eyes include built-in support for branching, and allows creating separate baselines per branch, so whenever a developer makes changes in the app, he can run his visual tests locally without affecting other branches, and only commit the baseline changes after making sure that the tests are passing properly.

In order to create a branch, simply call the method eyes.setBranch after creating the Eyes object and before starting the test with the name of the branch as the parameter for this method. As a result, the subsequent tests will run as part of that branch and any baseline changes that you will perform on these tests from the web application will be private to this branch.

eyes.BranchName = branchName;

branchName – the name of the branch to create.

The initial baseline used in a branch is taken from its parent branch. You can specify the parent branch by calling the setParentBranchName method with the parent branch name as a parameter. If you don't specify a parent branch, the default branch will be used. Note that just like branching code; the baseline is copied from the parent branch only the first time it is accessed. Once the baseline is accessed in a branch, it is no longer affected by changes made in the parent branch.

 

eyes.ParentBranchName = parentBranchName;

 

parentBranchName – the name of the parent branch to inherit from

When the developer is ready to commit the code changes that he made to the app, he will need to also commit the baseline changes to the main branch to ensure the tests will keep passing. In order to push/commit the baseline changes of a private branch, Applitools Eyes exposes a web API that allows taking a baseline from a private branch and overwriting the baseline of the main branch with it.

 

POST https://eyes.applitools.com/api/branches/overwrite

In the request body you provide the object { "source" : "≶name≷", "target" : "≶name≷" } which indicates the source and target branch names and causes the server to copy all baselines and models from the source branch to the target branch.

It is recommended to add hook to the source control client such that after every successful merge from the private branch to the main branch (or alternatively a "push" from the local repository to the main one), a web call will be made to the above API. This is a source control framework specific configuration and each framework has different APIs for accomplishing this.

It is also recommended to configure your test framework, so the current branch name that the developer is working on and the parent branch name will be obtained automatically from the source control client when the test starts.


 

(Optional) Overriding test comparison level

Comparison sensitivity, aka ‘match level’ is set by default to Strict to get optimal coverage. Sometimes for specific tests it is required to change the default comparison level

eyes.MatchLevel = MatchLevel.<Level>;

 

Level - The match level of the test:
  • Exact - Pixel to pixel comparison, for demonstration purposes and debugging, will fail a test if a pixel is not in place. (not lace. (not recommended)
  • Strict - strict is the default match level, it mimics human eyes so only significant visual changes will be spotted, while small changes that are not visible to a human eye will be ignored.
  • Content - ignores style and anti-aliasing differences, while spotting content changes. (the content level can be useful if your website includes different styles which are not relevant for your tests).
  • Layout - ignores content changes, and detects only layout changes. (useful for pages with dynamic content and localized pages)

Note: Overriding the match level is only effective if set before initialization.


 

(Optional) Ending tests without throwing exception on failure

The default behavior on failure is to throw exception or raise error on test ending. To override this behavior use one of the overrides which takes boolean.

 ...
    TestResults testResults = eyes.Close(False);
}finally {
    eyes.AbortIfNotClosed();
}

 

testResults - Test results details, Contains attributes about the test and the failure

such as:
    Steps - Total steps count
    Matches - Total matches count
    Mismatches - Total mismatches count
    Missing - How many missing steps found
    ExactMatches - Amount of matches compared in exact match level
    StrictMatches - Amount of matches compared in strict match level
    ContentMatches - Amount of matches compared in content match level
    LayoutMatches - Amount of matches compared in layour match level
    isNew - Boolean value indicates whether the test classified as a new test.

 

 

(Optional) Automatic baseline creation (Default: True)

New tests presented to the server are automatically saved as baseline. To override this functionality so baseline creation will be applied manually by reviewing and approving the steps in applitools eyes Test Manager, set ‘SaveNewTests’ to false before test stat.

eyes.SaveNewTests = true/false;

 

*true/false - Either true or false.

 

 

(Optional) Auto-save on failure (Default: False)

For maintenance and debugging purposes it is possible to make eyes to save automatically failed results as baseline by setting ‘SaveFailedTests’ property to true before test started.

* It is highly unrecommended to set this property in production since all failures will be saved automatically without distinguishment between bugs and features.

eyes.SaveFailedTests = true/false;

*true/false - Either true or false.

 

 

(Optional) Setting app environment attributes

 

Normally applitools eyes SDKs automatically identifies the environment that is used to run the tests and creates a separate baseline for each environment. In order to override app environment parameters and ‘force’ Eyes to compare results of different environments, you can explicitly call ‘setAppEnvironment’:

eyes.SetAppEnvironment(hostOs, hostApp);

hostOs - The OS name and its version or any other os-alike identifier. hostApp - The application name, if there is, such as browser name or any other app-alike identifier.

This functionality can be used to ‘force’ eyes to preset the hostOS and hostApplication so identical tests on different environments will be unified under the same baseline. Note that using this in such way will most likely cause the tests to fail, since in most cases there are some differences between how the AUT is presented in the different environments.