Documentation

ScalaWebTest has two main features.

• Handling of web requests and state
• Gauges for simple tests

The short description of the two main features is followed by a getting started section suitable for beginners and a detailed description of all features.

Handling of web requests and state

ScalaWebTest's IntegrationSpec trait helps you to structure your tests and keep them boilerplate free. It extends the most important traits, to write integration tests for web applications, from ScalaTest. It provides an easy way to configure the Selenium webdriver and it takes care of login, web requests and cookies. Before executing your tests, it takes care of login, if needed. Next it requests the resource, which your test wants to verify. The URL to the tested resource is split in two segments. The baseUri is part of the Configuration, use config.useBaseUri to change it. The path, which is usually different for every test, is then appended to the baseUri. In most cases a test should only work with a single resource, in this case the path doesn't change throughout the test. Per default the IntegrationSpec will, before each test, take care that the current resource in the WebDriver is the one belonging to the defined path. This behavior can be changed. Therefore your test does not have to interact with the WebDriver. Instead it can directly verify the content of the current resource via webdriver.

Gauges for simple tests

Selenium is a powerful tool, but testing the returned HTML from a web application is cumbersome and the resulting tests are often hard to read. We think the easiest way to define the expected result of a web request is using the returned data format itself. Therefore we use pseudo HTML (alternatively JSON to verify JSON) to describe the expected result. Of course a simple string comparison would be to naive. Therefore we treat the HTML (Scala XML literal, to be precise), which is used to describe the expected result, as gauge definition (other would call it a template). If all elements from the gauge definition are present in the verified HTML page, it does fit the gauge and is therefore valid.

Add ScalaWebTest to your project

Before you get to use all the nice features, you have to introduce ScalaWebTest to your project. All you have to do is adding the core module to your test or integration test dependencies.

Add ScalaWebTest to your SBT project

You can add ScalaWebTest to your testing dependencies in the build.sbt/scala of your sbt project as follows.

libraryDependencies += "org.scalawebtest" %% "scalawebtest-core" % "3.0.1" % "test" 

We recommend to bind the ScalaWebTest dependencies to the IntegrationTest configuration, which can be referenced with "it". The IntegrationTest configuration is not active by default. Follow the guide for SBT 1.0 or SBT 0.13 to make it part of your build. Then you can add ScalaWebTest to your project as follows.

libraryDependencies += "org.scalawebtest" %% "scalawebtest-core" % "3.0.1" % "it" 

ScalaWebTest uses the slf4j-api to write to the console or logfiles. If your project does not contain an implementation of the Slf4j Logger, add the following dependency.

libraryDependencies += "org.slf4j" % "slf4j-simple" % "1.7.26" 

If you want to manage the most important transitive dependencies of ScalaWebTest, you might configure it as follows (replace "it" with "test", if you do not make use of the IntegrationTest configuration). Especially selenium-java and htmlunit-driver have to align with each other.

libraryDependencies ++= Seq(
    "org.scalawebtest" %% "scalawebtest-core" % "3.0.1" % "it",
    "org.scalatest" %% "scalatest" % "3.0.8" % "it",
    "org.seleniumhq.selenium" % "selenium-java" % "3.141.59" % "it",
    "org.seleniumhq.selenium" % "htmlunit-driver" % "2.35.1" % "it"
) 

Add ScalaWebTest to your maven project

You have to add the following dependency to your maven project to use ScalaWebTest.

<dependency>
    <groupId>org.scalawebtest</groupId>
    <artifactId>scalawebtest-core_2.13</artifactId>
    <version>3.0.1</version>
    <scope>test</scope>
</dependency> 

ScalaWebTest uses the slf4j-api to write to the console or logfiles. If your project does not contain an implementation of the Slf4j Logger, add the following dependency.

<dependency>
    <groupId>org.slf4j</groupId>
    <artifactId>slf4j-simple</artifactId>
    <version>1.7.26</version>
    <scope>test</scope>
</dependency> 

ScalaWebTest is also available for Scala 2.12 and 2.11, change the version after the underscore, to get a ScalaWebTest version which is binary compatible with the one you are using.

<dependency>
    <groupId>org.scalawebtest</groupId>
    <artifactId>scalawebtest-core_2.12</artifactId>
    <version>3.0.1</version>
    <scope>test</scope>
</dependency> 

In case you want to manage the most important transitive dependencies of ScalaWebTest, you can add our bill-of-materials to your project. Just add the following to your dependencyManagement.

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.scalawebtest</groupId>
            <artifactId>scalawebtest-bom_2.13</artifactId>
            <version>3.0.1</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
    </dependencies>
</dependencyManagement> 

Write your first test

To write your first test, you have to create a scala file in src/it/scala or src/test/scala, depending on your test setup. Lets assume we want to verify our homepage. Create a class named HomepageSpec, and let it extend IntegrationFlatSpec. This uses FlatSpec style from ScalaTest, other styles are available.

import org.scalawebtest.core.IntegrationFlatSpec
import org.openqa.selenium.By

class HomepageSpec extends IntegrationFlatSpec {
  config.useBaseUri("https://www.scalawebtest.org")
  path = "/index.html"

  "Our homepage" should "contain a succinct claim" in {
    webDriver
      .findElement(By.tagName("h2"))
      .getText shouldEqual "Reduce the effort needed to write integration tests"
  }
} 

That's all you need. The IntegrationSpec, inherited from IntegrationFlatSpec will automatically call https://www.scalawebtest.org/index.html before executing a test. When using FlatSpec style the block following the in keyword is the body of the test. Within the body of a test you can access the webDriver, to test the content of the current page. The webDriver by default backed by HtmlUnit. It emulates a browser and supports among other things JavaScript execution, clicking elements and submitting forms. HtmlUnit has its limitations with JavaScript and it is always headless, for those that need more advanced features we recommend SeleniumChrome.

Write your first gauge

This test can be rewritten making use of the HtmlGauge. Writing gauges provides a detailed introduction to this feature.

import org.scalawebtest.core.IntegrationFlatSpec
import org.scalawebtest.core.gauge.HtmlGauge

class HomepageSpec extends IntegrationFlatSpec with HtmlGauge {
  config.useBaseUri( "http://www.scalawebtest.org")
  path = "/index.html"

  "Our homepage" should "contain a succinct claim" in {
    currentPage fits <h2>Reduce the effort needed to write integration tests</h2>
  }
} 

Build your base class

import org.scalatest.AppendedClues
import org.scalatest.concurrent.PatienceConfiguration.Timeout
import org.scalatest.time.SpanSugar._
import org.scalawebtest.core.gauge.HtmlGauge
import org.scalawebtest.core.{FormBasedLogin, IntegrationFlatSpec}

import scala.language.postfixOps

abstract class MyProjectBaseSpec extends IntegrationFlatSpec with FormBasedLogin with AppendedClues with HtmlGauge {
    config.useBaseUri("http://localhost:9090")
    loginConfig.useLoginUri("http://localhost:9090/login.php")

    override def loginTimeout = Timeout(5 seconds)
} 

Selecting a style

ScalaTest provides a wide variety of testing styles. ScalaWebTest supports all available styles. As we believe the FlatSpec style is easy to read for most people, we choose it for our own documentation and most of our integration tests. There is no technical reason behind it, it is simply a matter of taste. We recommend that you extend one of the Integration*Spec/Suite classes from Styles.scala in your abstract base class.

Configure your tests

A reasonable default configuration is provided, which can easily be adapted. If you want to change a configuration for all your tests, we recommend to change it in your base trait, otherwise best practice is to change it in the constructor of your TestSpec. Two config objects exist, one which is used during login (if login is used) and one which is used during test execution.

In the following TestSpec JavaScript errors are swallowed during Login phase, no matter if JavaScript execution is enabled or disable. During test execution JavaScript is enabled and JavaScript errors let the test fail.

import org.openqa.selenium.By
import org.scalawebtest.core.IntegrationFlatSpec

class HomepageSpec extends IntegrationFlatSpec {
  config.useBaseUri("http://www.scalawebtest.org/index.html")

  loginConfig.swallowJavaScriptErrors()
  config.enableJavaScript(throwOnError = true)

  "Our homepage" should "contain a succinct claim" in {
    webDriver
      .findElement(By.tagName("h2"))
      .getText shouldEqual "Reduce the effort needed to write integration tests"
  }
} 

By default JavaScript is not executed and JavaScript errors don't throw. Also CSS is not interpreted. This applies to both loginConfig and config.

Using gauges

Using gauges to write your integration tests, is the core idea of ScalaWebTest. We believe Selenium has good primitives to navigate a website (triggering events, filling forms), but the tools to verify the resulting output are lacking. We are convinced that the HTML DOM is what should be verified. An alternative would be visual tests, but those tend to be brittle and it is difficult to test only a certain aspect of the page. But a few of them might be a valuable addition. using org.scalatest.selenium.WebBrowser.Query, i.e CssSelectorQuery, to select elements in a HTML document and then verifying it is cumbersome and resulting tests are hard to read. Instead we borrow a concept from the manufacturing industry. They build gauges (or templates), which they then lay their workpiece into. If the workpiece fits in the gauge, it satisfies the requirements.

Our gauges are defined in HTML (XML literals to be more precise, or scala.xml.NodeSeq to be exact) instead of being forked from steel. Let's have a look at a simple example to get an idea how those gauges work. When the browser receives HTML it parses it into a DOM (document object model) tree. This is also the abstraction, with which we work in ScalaWebTest. Let's first have a look at the document, which we would like to test with a gauge, and its representation as DOM tree.

<!DOCTYPE html>
<html lang="en">
<head><title>ScalaWebTest Homepage</title></head>
<body>
<div>
    <ul>
        <li>
            <a href="https://scalatest.org"
               class="nav active">
                Scala Test
            </a>
        </li>
        <li>
            <a href="https://unic.com"
               class="nav">
                Unic
            </a>
        </li>
    </ul>
</div>
</body>
</html>
                            

We would like to verify if the following gauge fits the above document. The gauge itself is also parsed into a DOM tree.

import org.scalawebtest.core.IntegrationFlatSpec

class HtmlGauge extends IntegrationFlatSpec {
  config.useBaseUri("http://localhost:8080")
  path = "index.html"

  "index.html" should "contain a nav item for Unic" in {
    currentPage fits
      <div>
        <ul>
          <li>
            <a href="https://unic.com">Unic</a>
          </li>
        </ul>
      </div>
  }
}
                        

Let's have a closer look at how ScalaWebTest checks if the document fits the gauge. On the right hand side you can see the DOM tree of the gauge, on the left hand side the document. The right hand side drives the verification process. For every element/attribute in the gauge, an according element/attribute has to be found in the document.

We can see that currentPage fits ignored html, head and body, elements before the navigation. ScalaWebTests gauges do not only allow for gaps in your gauge definition, such gaps are encouraged. The idea is to make tests stable and vocal about their intend, by focusing on the elements and attributes, which matter for a specific test.

<!DOCTYPE html>
<html lang="en">
<head><title>ScalaWebTest Homepage</title></head>
<body>
<div>
    <ul>
        <li>
            <a href="https://scalatest.org"
               class="nav active">
                Scala Test
            </a>
        </li>
        <li>
            <a href="https://unic.com"
               class="nav">
                Unic
            </a>
        </li>
    </ul>
</div>
</body>
</html>
                            

The document remains the same as in the previous example. The gauge definition is missing the element div and li.

import org.scalawebtest.core.IntegrationFlatSpec

class GapGauge extends IntegrationFlatSpec {
  config.useBaseUri("http://localhost:8080")
  path = "index.html"

  "index.html" should "contain a nav item for Scala Test" in {
    currentPage fits
      <ul>
        <a href="https://scalatest.org">Scala Test</a>
      </ul>
  }
}
                        

While gaps are allowed in the gauge definition. The order of elements has to be identical for the gauge definition and the document. The following example illustrates this, with a document, which does not fit the provided gauge.

<!DOCTYPE html>
<html lang="en">
<head><title>ElementOrderGauge</title></head>
<body>
<div>
    <ul>
        <li>Second</li>
        <li>First</li>
    </ul>
</div>
</body>
</html>
                            

In the document the element <li>Second</li> is before <li>First</li>. This is not as intended and the following gauge definition expects them in natural order.

import org.scalawebtest.core.IntegrationFlatSpec

class ElementOrderGauge extends IntegrationFlatSpec {
  config.useBaseUri("http://localhost:8080")
  path = "index.html"

  "index.html" should "a correcly ordered list" in {
    currentPage fits
      <div>
        <ul>
          <li>First</li>
          <li>Second</li>
        </ul>
      </div>
  }
}
                        

As expected this document does not fit the gauge. In the last step, the text of the element is not verified, because even before that it is known that the element <li>Second</li> is at the wrong position, relative to <li>First</li>

Test for classes on elements

The class attribute is special, because it is basically an unsorted set of class attributes. Usually we don't bother, if additional classes are present in our HTML and for sure we never care about the order. ScalaWebTest therefore handles the class attribute different from the rest. Per default the attribute content, has to match exactly the one you provided in your gauge, but for classes, it only asserts, that the classes which an element contains within your gauge definition, are all present on that element in the HTML document. The following gauge therefore matches all the elements shown below.

currentPage fits
    <div class="container red"></div>

<div class="container red"></div>

<div class="red container"></div>

<div class="red important container main"></div>

Test single elements

Often trying to fit the complete document into the defined gauge isn't the most natural and efficient thing to do. Especially when a website contains multiple elements of the same kind, such as content cards, gallery images or items of a product list. It is more natural to first findAll those elements, and then trying to fit each element into the gauge.

import org.scalatest.AppendedClues
import org.scalawebtest.core.IntegrationFlatSpec
import org.scalawebtest.core.gauge.HtmlElementGauge

class ElementGaugeSpec extends IntegrationFlatSpec with HtmlElementGauge with AppendedClues {
  config.useBaseUri("http://localhost:9090")
  path = "/galleryOverview.jsp"

  val imageGauge =
    <div class="columns image_columns">
      <a>
        <figure class="obj_aspect_ratio">
          <noscript>
            <img class="obj_full"></img>
          </noscript>
          <img class="obj_full lazyload"
               data-sizes="auto"></img>
        </figure>
      </a>
    </div>

  "The gallery" should "contain the expected HTML for every image" in {
    def images = findAll(CssSelectorQuery("ul div.image_columns"))

    images.size should be > 5 withClue " - gallery didn't contain the expected amount of images"

    for (image <- images) {
      image fits imageGauge
    }
  }
}

When using findAll with a for-comprehension, always verify, that the expected amount of elements was found. If zero element where found, no element is checked. Therefore an additional check is required to make sure the test fails, if findAll didn't return anything.

When using fits or doesntFit, the same features are available, no matter if the whole document, or a single element is checked. Therefore the following chapters are applicable for both ways of using it.

Test for containment

It is common that you don't want to verify the complete attribute value, or text of an element. You may use @contains to trigger the ContainsMatcher instead of the DefaultMatcher. Thanks to this matcher, this gauge matches the following HTML element

fit(<a href="@contains ScalaWebTest"></a>)

This gauge will fit

<a href="https://github.com/unic/ScalaWebTest"></a>

The containsMatcher is also available, when matching text.

fit(<a>@contains available</a>)

This gauge will fit

<a>ScalaWebTest is available on github</a>

Hint: always add a space after the matcher annotation. We enforce this space to improve readability of tests.

Test for regex matches

When we want to enforce textual rules on our content, we need a more powerful Matcher. The RegexMatcher is our friend. It is triggered using @regex

fit(<a href="@regex http:\\/\\/[a-zA-Z]+\.scalawebtest.org.*,"></a>)

This gauge will fit

<a href="http://www.scalawebtest.org/documentation.html"></a>

but it won't fit

<a href="http://scalawebtest.org/documentation.html"></a>

Negating your tests

Sometimes you want to make sure that a certain content is not shown. For example you want to make sure, that no login form is shown, when a user is logged in. Or you want to make sure that a post doesn't appear before it is made public. To do so you may use currentPage doesNotFit or currentPage doesntFit. They are synonyms. If you prefer gauges, which do not start with currentPage, but directly with the gauge, you may use doesnt fit() or not fit() Choose whatever reads better in your current context.

import org.scalawebtest.core.IntegrationFlatSpec
import org.scalawebtest.core.gauge.HtmlGauge

class LoggedInSpec extends IntegrationFlatSpec with HtmlGauge {
  path = "/protectedContent.jsp?username=admin&password=secret"

  "When logged in the protectedContent page" should "not show the login form" in {
    currentPage doesNotFit <form name="login_form"></form>
  }
}

In case the login form is mistakenly rendered, the following error will appear.

Current document matches the provided gauge, although expected not to!
Fitting node
<html>
<head>
    <title>Mock of a protected content page
    </title>
</head>
<body>
<form name="login_form" action="protectedContent.jsp" method="get">
    <label for="username">username
    </label>
    <input type="text" name="username" id="username">
    </input>
    <label for="password">password
    </label>
    <input type="password" name="password" id="password">
    </input>
    <button type="submit">login
    </button>
</form>
</body>
</html>
found 

Testing a complete process

Today's web applications are of course not that static. Usually it is more interesting to test a complete process. Selenium has great support to do so. You can press buttons, fill and submit forms and execute JavaScript. As ScalaWebTest's gauges are evaluated against the same browser window, as the executed Selenium commands, gauges can be used to verify pre-conditions and results of actions. Remember ScalaTest executes the tests in order. As soon as an action is executed, the currentPage is updated and the next call to fit() or doesnt fit() will be executed against the updated page/DOM.

Lets try this out using a page with protected content. Initially a login form is shown, after submitting it with correct credentials, the form disappears, and the text sensitive information appears.

import org.scalawebtest.core.IntegrationFlatSpec

import org.scalawebtest.core.gauge.HtmlGauge

class LoginSpec extends IntegrationFlatSpec with HtmlGauge {
  path = "/protectedContent.jsp"

  "When accessing protectedContent it" should "show the login form" in {
    currentPage fits
      <form name="login_form">
        <input name="username"></input>
        <input name="password"></input>
      </form>
  }

  it should "hide the protected content, when not logged in" in {
    currentPage doesNotFit <p>sensitive information</p>
  }

  it should "show the protected content, after logging in" in {
    textField("username").value = "admin"
    pwdField("password").value = "secret"

    submit()

    currentPage fits <p>sensitive information</p>
  }
}

This example test was split into three smaller tests. This is technically not necessary, but has the following advantage. When only part of the tests fails, you have a better idea what went wrong. For example, if the login fails, but the login form is correctly shown, the issue has to be your login process. But if the login form isn't shown, the problem has a very different cause. In addition the verbose output from the gauge should help you finding the root cause.

Testing lifecycle

ScalaTest has a same lifecycle as most testing frameworks. beforeAll and afterAll are executed first, respectively last and only once per suite (a class extending IntegrationFlatSpec is a suite). beforeEach and are executed before, respectively after every single test.

Tests itself are executed in the same order, as they are defined. For ScalaWebTest to manage the state of the browser for you, it has to overwrite beforeAll and beforeEach. You can still overwrite beforeAll or beforeEach in your code, but remember to call super.beforeAll(), respectively super.beforeEach(). With beforeAll, chances are, you want to have more finegranular control when your code is executed in the beforeAll process. Therefore ScalaWebTest provides the hook methods beforeLogin and afterLogin.

Configuration

To change configurations for a test execution, ScalaWebTest reads system properties, environment variables and Runner arguments. They are tried in the following order, with the first one being defined winning. The rule of precedence is Runner arguments environment variables over system properties. While lowercase and dot separated is idiomatic for system properties, uppercase and underscore separated is idiomatic for environment variables in Unix and Linux (dots are not allowed). Therefore we have the following rule.

• Runner arguments are lowercase and dot separated, for example scalawebtest.base.uri
• Environment variables can be either
  • lowercase and dot separated, for example scalawebtest.base.uri
  • uppercase and underscore separated, for example SCALAWEBTEST_BASE_URI
• System properties arguments are lowercase and dot separated, for example scalawebtest.base.uri

The following configurations are always available

• scalawebtest.login.uri
• scalawebtest.base.uri

While these are specific to chrome

• webdriver.chrome.driver.service.url
• webdriver.chrome.driver
• webdriver.chrome.arguments

ScalaWebTest informs its users about found and used configurations via log output.

[ScalaTest-run] INFO org.scalawebtest.core.IntegrationSpec$$anon$1 - No ConfigMap entry (runner argument) scalawebtest.login.uri found
[ScalaTest-run] INFO org.scalawebtest.core.IntegrationSpec$$anon$1 - No environment variable scalawebtest.login.uri found
[ScalaTest-run] INFO org.scalawebtest.core.IntegrationSpec$$anon$1 - No environment variable SCALAWEBTEST_LOGIN_URI found
[ScalaTest-run] INFO org.scalawebtest.core.IntegrationSpec$$anon$1 - No system property scalawebtest.login.uri found
[ScalaTest-run] INFO org.scalawebtest.core.IntegrationSpec$$anon$2 - ConfigMap entry (runner argument) scalawebtest.base.uri found with value http://www.scalawebtest.org
[ScalaTest-run-running-ResponseHeaderValueSpec] INFO org.scalawebtest.integration.doc._012.ResponseHeaderValueSpec - Going to http://www.scalawebtest.org/responseHeaders.jsp
                    

Your own tests might add additional configurations. To do so, extend Configurable trait and make use of configFor or requiredConfigFor. For example configFor[URI](configMap)("scalawebtest.base.uri"). Both methods expect the lowercase and dot separated representation of the config property.

Runner arguments

To read Runner arguments, ScalaWebTest uses configMap from ScalaTest. The arguments can be provided with -Dkey=value, for example -Dscalawebtest.base.uri=http://localhost:8090/myapp.

scala -classpath scalatest-<version>.jar org.scalatest.tools.Runner -R compiled_tests -Dkey=value

The Intellij Testrunner works with arguments as well, add -Dkey=value to the Test options

Environment variables

Environment variables are an alternative way to change ScalaWebTest configurations. It has lower precedence then runner arguments. Environment variables are especially useful with Docker containers, as they are the de-facto standard to provide configuration to containers.

System Properties

System properties have the lowest precedence. They can be set using System.setProperty("key", "value");

Executing JavaScript

With Html-Unit Driver it is possible to execute JavaScript, but some edge-cases are not covered perfectly. As JavaScript on the JVM is interpreted with Rhino. For JavaScript heavy web applications, we recommend to use a native browser, such as Chrome. Per default ScalaWebTest doesn't execute JavaScript, because tests run faster without. To execute JavaScript, you have to enable it.

 config.enableJavaScript(throwOnError = true)

In addition, you have to decide whether a JavaScript error should cause your test to fail or not. Usually failing on JavaScript errors is good, but sometimes you have to write tests for a web application with major issues in it's JavaScript. This is one of the rare moments, where you have to disable throwOnError to be able to test the web application.

After receiving document, your browser needs some time to execute JavaScript. The same is true for the Selenium webdriver. We have to give it some time to execute JavaScript, before we can expect it to have transformed the HTML. ScalaTest provides eventually, which does exactly what we need. It repeats a given test until it succeeds or the given timeout has been surpassed.

eventually(timeout(3 seconds)) {
    fits(
        <div id="container">Text loaded with JavaScript</div>
    )
}

Access response code and response headers

HTTP response codes and response headers are two important parts of the HTTP protocol. Therefore it is important to assert that they have the correct values. The ResponseAccessors trait provides convenient access to this information.

The HTTP response code can be accessed with the responseCode method.

import org.scalawebtest.core.{IntegrationFlatSpec, ResponseAccessors}

class ResponseCodeSpec extends IntegrationFlatSpec with ResponseAccessors {
    path = "/doesNotExist"
    "When accessing a resource which does not exist the response code" should "be 404" in {
        responseCode shouldBe 404
    }
}

The HTTP response headers are exposed as Map[String, String] by the method responseHeaders

import org.scalatest.OptionValues
import org.scalawebtest.core.{IntegrationFlatSpec, ResponseAccessors}

class ResponseHeadersSpec extends IntegrationFlatSpec with ResponseAccessors with
    OptionValues{
    path = "/responseHeaders.jsp"
    "The responseHeaders map" should "contain the Content-Type" in {
        responseHeaders should not be empty
        responseHeaders.keySet should contain("Content-Type")
        responseHeaders.get("Content-Type").value shouldBe "text/html; charset=UTF-8"
    }
}

The responseHeaderValue method simplifies the common "verifying that response header X has value Y" use-case.

import org.scalawebtest.core.{IntegrationFlatSpec, ResponseAccessors}

class ResponseHeaderValueSpec extends IntegrationFlatSpec with ResponseAccessors {
    config.useBaseUri("http://localhost:9090")
    path = "/responseHeaders.jsp"

    "The responseHeaderValue of Content-Type" should "be text/html with charset utf-8" in {
        responseHeaderValue("Content-Type") shouldBe "text/html;charset=utf-8"
    }
    "The responseHeaderValue" should "merge response header field-values, when multiple entries with the same field-name exist" in {
        /**
        * Cache-Control: no-cache
        * Cache-Control: no-store
        *
        * should be merged into
        *
        * Cache-Control: no-cache, no-store
        *
        * See http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9 for details
        */
        responseHeaderValue("Cache-Control") shouldBe "no-cache, no-store"
    }
}

Although response headers name should not be case sensitive according to HTTP/1.1 specification this isn't necessarily true for all implementations. We believe a testing framework shouldn't obfuscate what's sent over the wire. Therefore we expose the HTTP response headers in case-sensitive fashion.

Unfortunately, we can provide those accessors only in combination with HtmlUnit webdriver (the default webdriver with ScalaWebTest). The Selenium webdriver interface and most of it's implementations do not expose the needed information. Details on those limitations can be found in ScalaWebTest github issue #74 Remember, you can use multiple browsers within your tests.

Other Browsers

Some web applications don't behave correctly with HtmlUnit. Sometimes users want to see what is actually going on during test execution. In this situations a headed browser is preferable over the headless and JVM based HtmlUnit. ScalaWebTest support all implementations of Selenium Webdriver. To use a different webdriver just set webDriver to a different value, for example webDriver = new MyDriver(). As you might need access to the configMap, when doing so, there is a special hook to do so prepareWebDriver(configMap: ConfigMap). ScalaWebTest ships with built-in Chrome support, the according classes are a good example to add support for different browsers.

Using Chrome

To use Chrome in your tests extend the SeleniumChrome trait, install ChromeDriver on the executing machine and provide the webdriver.chrome.driver configuration. ScalaWebTest will manage the ChromeDriverService and run it with suitable arguments. If you prefer to manage ChromeDriverService yourself, use webdriver.chrome.driver.service.url instead and make sure it is running before executing the test. The arguments for ChromeDriverService can be overwritten using webdriver.chrome.arguments configuration. The Configuration chapter describes all possiblities to set those configurations.

Docker is a useful tool. We believe running ScalaWebTest in a docker container can be a good solution in multiple scenarios. When using it in none-JVM based projects, in projects which containerize their complete build process or to quickly try it out.

Using the default image

The scalawebtest/sbt image is a good starting point. Provide your tests with via an according volume and you are ready to go.

Building your own image

An interesting option is to build an own docker image, containing the compiled tests. This image can then be used to run regression tests against different environments. This might be even used as part of your monitoring. The docker example shows the easiest way to achieve this and how it can be combined with using scalawebtest/sbt during test development.

Modules

Every web application framework has its own specialities, which are relevant for testing. ScalaWebTest allows to share this framework specific logic via modules. We encourage others, to create additional modules for their most used framework.

What should be shared via modules?

We think even simple things such as default login method, ports and urls might be worth sharing. Functions to create content, update accounts, or requesting a page in debug mode, could be even better candidates for a module.

AEM module

As the creators of ScalaWebTest work a lot with Adobe Experience Manager, they decided to create the first module for this CMS

To use this module extend AemTweaks from the aem module in your BaseTrait. This will cause the following.

• FormBasedLogin is activated
• loginUri is set to the default AEM 6 login path
• pages are requested with wcmmode cookie set to DISABLED

PageProperties

The PageProperties trait populates the pageProperties, and if applicable componentProperties and suffixProperties fields with a JsValue representing the properties of the currentPage, component and suffix respectively. It does so by retrieving the JSON representation of the currentPage. This works by default on all CQ/AEM author instances. In addition it provides convenience methods to access the pageProperties content.

• pageProperties(name: String) - retrieve a page property by name
• jcrContent(name: String) - retrieve a property from jcr:content by name
• findByResourceType(value: String) - search through a parsys field in the pageProperties and find all component with given resourceType
• findByProperty(name: String)(value: String) - search through a parsys field in the pageProperties and find all component with given property name and value

It populates the pageProperties field with a play.api.libs.json.JsValue, which represents the properties of the currentPage. In case the url/path points to something below jcr:content, the componentProperties will be populated with the properties of the component, and the pageProperties with those of the containing page. In case the url/path contains a suffix, the suffixProperties will be populated with the properties of the page referenced in the suffix. It does so by manipulating the url field, to request the JSON representation of the currentPage from CQ/AEM. This feature is available on CQ/AEM author instances by default. The enable.json property of the org.apache.sling.servlets.get.DefaultGetServlet of your CQ/AEM instance has to be set to true. Only extend this trait in tests which need the feature, as it otherwise unnecessarily slows down your tests, due to additional requests for page properties.

JSON module

JSON is currently the most important data exchange format in the web. We think writing integration tests for web-services, which return JSON, should be as easy as for websites. Therefore we created the JsonGauge, which provides functionality comparable to the HtmlGauge. All its features are exposed via the according builder, which provides implicit conversions for JsValue, JsLookup and functions retrieving the currentPage from the webDriver. We choose to use play-json to parse the JSON response. To keep the core module clean from additional dependencies a module was created.

Only verifying a few values

To verify the value or type of a few values, we consider standard play-json sufficient. Just use Json.parse to parse the response and then use \ and \\ to navigate the json structure.

import org.scalawebtest.core.IntegrationFlatSpec
import play.api.libs.json.Json

class ReducedJsonSpec extends IntegrationFlatSpec {
    path = "/dijkstra.json"
    def json = Json.parse(webDriver.getPageSource)

    "Dijkstra" should "have the correct firstname" in {
        def firstName = (json \ "firstName").as[String]
        firstName should equal("Edsger")
    }
}

All JSON tests in the following examples use the following JSON response

{
  "name": "Dijkstra",
  "firstName": "Edsger",
  "yearOfBirth": 1930,
  "theories": [
    "shortest path",
    "graph theory"
  ],
  "isTuringAwardWinner": true,
  "universities": [
    {
      "name": "Universität Leiden",
      "begin": 1948,
      "end": 1956
    },
    {
      "name": "Mathematisch Centrum Amsterdam",
      "begin": 1951,
      "end": 1959
    },
    {
      "name": "Technische Universiteit Eindhoven",
      "begin": 1962,
      "end": 1984
    },
    {
      "name": "University of Texas at Austin",
      "begin": 1984,
      "end": 1999
    }
  ]
}

As this way of verifying JSON doesn't scale well, we created JsonGauge

JsonGauge - the ScalaWebTest way

Same as when verifying HTML, we think the easiest and most natural way to formulate your expectation is using the same language, as the tested response itself. With the JSON gauge you can use JSON to specify the gauge, into which the JSON response has to fit.

import org.scalawebtest.core.IntegrationFlatSpec
import org.scalawebtest.json.JsonGauge
import org.scalawebtest.json.JsonGaugeFromResponse.fitsValues

class DocumentFitsValuesSpec extends IntegrationFlatSpec with JsonGauge {
  path = "/dijkstra.json"

  "FitsValues" should "report success, when the json gauge contains the same values as the response it is tested against" in {
    fitsValues(
      """{
            "name": "Dijkstra",
            "firstName": "Edsger",
            "yearOfBirth": 1930,
            "isTuringAwardWinner": true,
            "theories": [
                "shortest path",
                "graph theory"
            ]
            }"""
    )
  }
}

Often verifying the complete response isn't ideal. We recommend to parse and traverse the document using play-json and then verify if the JsLookup or JsValue fits the defined gauge.

import org.scalawebtest.core.IntegrationFlatSpec
import org.scalawebtest.json.JsonGauge
import play.api.libs.json.Json

class JsValueJsLookupFitsValues extends IntegrationFlatSpec with JsonGauge {
  path = "/dijkstra.json"
  def dijkstra = Json.parse(webDriver.getPageSource)

  "The response for Dijkstra" should "contain the expected values" in {
    dijkstra fits values of
      """{
          "firstName": "Edsger",
          "name": "Dijkstra",
          "yearOfBirth": 1930,
          "theories": [
              "shortest path",
              "graph theory"
              ]
          }
      """
  }
  it should "contain the correct universities" in {
    val universities = dijkstra \ "universities"
    universities fit values of
      """
        [
            { "name": "Universität Leiden","begin": 1948, "end": 1956 },
            { "name": "Mathematisch Centrum Amsterdam", "begin": 1951, "end": 1959 },
            { "name": "Technische Universiteit Eindhoven", "begin": 1962, "end": 1984 },
            { "name": "University of Texas at Austin", "begin": 1984, "end": 1999 }
        ]
        """
  }
}

As with the HTML gauge, controlled variance is allowed. The original response contains additional key/value pairs (isTuringAwardWinner and universities). Also the order of the name and firstName is the other way around. Nevertheless this test would succeed, when run. It would report an error, if the response would:

• contain a different value for one of the keys
• miss a key, which is contained in the gauge
• contain a different hierarchy of the key/value pairs

Behaviors are a good way to share tests. When sharing test via behaviors, verifying values is often too specific. Therefore we provide fits types of in addition to fits values of.

import org.scalawebtest.core.IntegrationFlatSpec
import org.scalawebtest.json.JsonGauge
import play.api.libs.json.Json

class FitsTypeSpec extends IntegrationFlatSpec with JsonGauge {
  path = "/dijkstra.json"

  def dijkstra = Json.parse(webDriver.getPageSource)

  "The response for Dijkstra" should "contain the expected types" in {
    dijkstra fits types of
      """{
            "firstName": "",
            "name": "",
            "yearOfBirth": 0,
            "theories": [ "" ]
        }
    """
  }
}

Now the values no longer matter. They are only used to determine the expected type. Therefore we recommend to use values, which indicate that only the type matters. Use empty strings and 0. This test would succeed. It would fail, when the response would:

• contain a value with a type different from the one expected for the given key
• miss a key, which is contained in the gauge
• contain a different hierarchy of the key/value pairs
• when an array value would contain an element with a mismatching type

Lets have a closer look at arrays. Usually we expect all array elements to match certain expectations. When using fits types of, you only have to define those expectations once. All array elements then have to fulfill them. Lets have a look at an example. Lets assume the test of the previous example would be replaced with the following

"The response for Dijkstra" should "contain the expected types" in {
    dijkstra fits types of
        """{
            "firstName": "",
            "name": "",
            "yearOfBirth": 0,
            "theories": [ "" ]
            "universities":
            [{
                "name": "",
                "begin": 0,
                "end": 0
            }]
        }
        """
} 

This would enforce all elements of the universities array to contain a key name with a value of type string and the keys begin and end with a value of type number. The elements might contain additional keys, such as department, but no contradictions. Sometimes this is not the behavior one is looking for. Two more options exist to verify arrays. One common scenario is, that the size of the array matters, i.e. coordinates in a GeoJson document. To test this use fits typesAndArraySizes of. In this case every array element is considered unique, you have to provide specific expectations for every element, but those expectations are allowed to differ. For example, we expect Martin to have studied and worked at three universities. For the first two an end is known. This isn't the case for the last one.

"The response for Odersky" should "contain the expected types and array sizes" in {
    odersky fits typesAndArraySizes of
        """{
            "firstName": "",
            "name": "",
            "universities":
        [{
            "name": "",
            "begin": 0,
            "end": 0
        },
        {
            "name": "",
            "begin": 0,
            "end": 0
        },
        {
            "name": "",
            "begin": 0
        }]
        }
        """
} 

One more common situation exists with arrays. Often the array elements are not sorted. Although we expect the array to contain a specific element, we don't know it's position. For this situation containsElementFitting values/types/typesAndArraySizes of exists.

package org.scalawebtest.documentation

import org.scalawebtest.core.IntegrationFlatSpec
import org.scalawebtest.json.JsonGauge
import play.api.libs.json.{JsLookupResult, JsValue, Json}

class ContainsElementFittingSpec extends IntegrationFlatSpec with JsonGauge {
    path = "/jsonResponse.json.jsp"
    def dijkstra: JsValue = Json.parse(webDriver.getPageSource)
    def universities: JsLookupResult = { dijkstra \ "universities" }

    "The universities array" should "contain an element with the expected types" in {
        universities containsElementFitting types of
            """{
                | "name": "",
                | "begin": 0,
                | "end": 0
                | } """.stripMargin
    }
    it should "contain an element with the expected values" in {
        universities containsElementFitting values of
            """{
                | "name": "Technische Universiteit Eindhoven",
                | "begin": 1962,
                | "end": 1984
                | }""".stripMargin
    }
} 

If we know all elements of an array, but we do not know their position, we can use fits valuesIgnoringArrayOrder of instead of checking only for a single one with containsElementFitting.

import org.scalawebtest.integration.json.{FitsTypeMismatchBehavior, ScalaWebTestJsonBaseSpec}
import play.api.libs.json.{JsValue, Json}

class FitsValuesIgnoringArrayOrderSpec extends ScalaWebTestJsonBaseSpec with FitsTypeMismatchBehavior {
  path = "/dijkstra.json"

  def dijkstra: JsValue = Json.parse(webDriver.getPageSource)

  "Dijkstra" should "contain the correct firstName and lastName and the correct universities in any order" in {
    dijkstra fits valuesIgnoringArrayOrder of
      """
        |{
        | "name": "Dijkstra",
        | "firstName": "Edsger",
        | "universities":
        | [
        |   { "name": "Universität Leiden","begin": 1948, "end": 1956 },
        |   { "name": "University of Texas at Austin", "begin": 1984, "end": 1999 },
        |   { "name": "Technische Universiteit Eindhoven", "begin": 1962, "end": 1984 },
        |   { "name": "Mathematisch Centrum Amsterdam", "begin": 1951, "end": 1959 }
        | ]
        |}
      """.stripMargin
  }
}

Because of its simplicity we prefer play-json over Argonaut or Circe. A good alternative to our JSON module is cornichon. Cornichon requires your tests to be a bit more specific then the ScalaWebTest ones, but it provides nice ways to do so. As cornichon uses Akka Http instead of Selenium to retrieve the JSON response, it is only partially compatible with ScalaWebTest. They might coexist in the same codebase and both are based on ScalaTest, but cornichon can't make much use of the boilerplate reduction provided by our IntegrationSpec.

One of the goals for ScalaWebTest is to make it as as easy as possible, to extend it. Therefore the foreseen extension points are a part of the documentation.

Create a custom matcher

One of the core extension points for gauge testing, are the matchers. ScalaWebTest uses two types of matchers, one for text and one for attributes. You may create your own TextMatcher or AttributeMatcher by extending those traits. You can then add your own matcher, by prepending to the list of textMatchers or attributesMatchers in the Matchers object.

ExtendingMatchers.scala contains an example, of how to implement your own Matcher

Create a custom login

If the available login mechanisms don't work with your web application. No worries, simply create your own implementation of the Login interface and extend it in your base trait. Ideally you then contribute it back to the project via pull request.

The FormBasedLogin provides a good example on how to implement your custom login

Create your own module

If you implement multiple web applications with the same framework, you might want to create a framework specific module. This allows to share framework specific testing code among multiple projects. You can start using it within your company/project and share it later with the rest of the community by creating a pull request. If you provide your own module to ScalaWebTest, we ask you to help maintaining it.

IntegrationSpec

The IntegrationSpec is the base trait, which all testing style specific traits extend. When writing a spec you should not extend IntegrationSpec, but one of the testing style specific traits, such as IntegrationFlatSpec

The base trait extends the following traits

• Webbrowser - Selenium DSL for ScalaTest
• Suite - encapsulates a conceptual suite of tests
• BeforeAndAfterEach - provides beforeEach and afterEach test hooks
• BeforeAndAfterAllConfigMap - provides beforeAll and AfterAll tests hooks
• IntegrationSettings - a set of fields to configure IntegrationSpec
• Eventually - provides eventually function

Integration_Spec

There is a complete set of test style specific base traits. Choose a testing style and then extend the according style specific Integration_Spec trait to create your test.

This traits extend the following traits

• _Spec i.e. FlatSpec - provides the testing style specific DSL
• IntegrationSpec - integrates ScalaTest and Selenium with additional features from ScalaWebTest
• Matchers - provides DSL for assertions using the word should
• Inspectors - provides nestable inspector methods that enable assertions to be made about collections

_SpecBehavior

One way to share assertions between testing suites, is to use behavior. A behavior contains a set of assertions. You can then use X behaves like SomeBehavior within the suite. Behavior and Suite have to use the same testing style. Therefore ScalaWebTest builds its style specific base traits on the style specific behavior traits. The following traits, which are part of the Integration_Spec, are in fact inherited from the behavior trait

• _Spec i.e. FlatSpec
• Matchers
• Inspectors

Gauge

The Gauge provides the methods fit, fits, doesnt fit and not fit. All of them are used to verify if the current page matches the given gauge specification. To do so, the gauge, searches for elements, as defined by the specification, in the current webpage. Elements are found by element name and containing classes. All elements which fulfill the search criteria, are considered candidates. Candidates are then verified for correct attributes and text, using Matchers, next they are verified by checking whether their children match. As soon as something doesn't match a Misfit is reported. If no candidate matches all criteria, the gauge doesn't fit. While verifying the candidates, a list of Misfits was acquired, the gauge will only report the most specific of all Misfits.

Misfit

A Misfit is a container for an error message, when something didn't fit a given Matcher. It contains an error message and a relevance. The deeper the current check in the gauge specification is, the higher its relevance. In the end only the Misfits with the highest relevance will be part of the error message.

Matchers

When working with gauges for testing, Matchers are used to test attributes and text content of elements. By default the following Matchers are available.

• Default Matcher: tests whether an attribute or text exactly matches a given String
• Contains Matcher: tests whether an attribute or text contains a given String
• Regex Matcher: tests whether an attribute or text matches the given Regex

All Matchers, which are available by default, can be used for attributes and text. When creating a new Matcher, one is not forced to implement for both. For each matcher type a specific trait exists. Extend AttributeMatcher or TextMatcher or both, when creating your own Matcher

When creating a custom Matcher the most important thing is to provide a detailed Some(Misfit) in case the Matcher doesn't match. In case of a match, the Matcher returns None.

Login

The Login trait is very simple. It only defines what the username and password field should be called. It is used to mark specific implementations as Login and to assert that username and password are consistent over all implementations. Depending on the authentication process for which a Login trait was implemented, it might make sense to overwrite the login function from IntegrationSpec.

WebClientExposingDriver

ScalaWebTest per default uses a custom web driver, more precisely a wrapper for the HtmlUnitDriver. Thanks to this wrapper, we can expose the WebClient to enable more control over the WebClient.

WebClientExposingDriverConfigFixtures

The WebClientExposingDriverConfigFixtures provide the option to execute a closure with a specific configuration. After the closure is executed, the WebDriver configuration will be reverted to what it was before the call of the fixture.