Wiki

Clone wiki

rest-api-blueprint / AutomaticTesting

Automatic testing

A REST API can be tested using Gherkin BDD syntax, a neat way of expressing and automatically confirming the desired behaviour. The behave tool works really well, particularly when combined with jpath, purl, requests, and Jina2. During ad hoc interactive testing and experimenting the httpie tool is usually easier than curl.

Like many aspects of this whole blueprint project, the automatic testing of a REST API can easily be pulled out and used in isolation. See section at the bottom of this page for the details.

Features

Express automatic tests using Gherkin sentences/steps

The code supports a number of basic Gherkin steps. It is trivial to add more or make compound steps.

Declare base URL:

  • I am using server "{server}"

Declare headers e.g. for Accept and Content-Type headers:

  • I set {var} header to "{value}"

Use authentication:

  • I use query OAuth with key="{key}" and secret="{secret}"
  • I use header OAuth with key="{key}" and secret="{secret}"

HTTP methods. Note step data is optional:

  • I send an OPTIONS request to "{url_path_segment}"
  • I send a PATCH request to "{url_path_segment}"
  • I send a PUT request to "{url_path_segment}"
  • I send a POST request to "{url_path_segment}"
  • I send a GET request to "{url_path_segment}"
  • I send a DELETE request to "{url_path_segment}"

Poll until given condition (see below for more detail on this):

  • I poll GET "{url_path_segment}" until JSON at path "{jsonpath}" is

Store and check return JSON (see below for more detail on this):

  • I store the JSON at path "{jsonpath}" in "{variable}"
  • the JSON should be
  • the JSON at path "{jsonpath}" should be {value}
  • the JSON at path "{jsonpath}" should be

Misc:

  • the response status should be "{status}"
  • I set context "{variable}" to {value}
  • the {var} header should be "{value}"
  • the {var} header should be

The code is always authoritative, so check the above list using grep features/steps/*.py -e '@behave'.

Accessing JSON data

Data within JSON structure is accessed succinctly using jpath expressions. For example:

    Scenario: Simple GET
      When I send a GET request to "test"
      Then the JSON at path "name" should be "Tim"
      And the JSON should be:
        """
        {
          "name": "Tim",
          "age": "never you mind!"
        }
        """

Templating tests from previous values

Returned JSON data can be stored and referenced in later steps using Jina2 template syntax. For example:

    Scenario: GET and remember value for future GET
      When I send a GET request to "test"
      And I store the JSON at path "name" in "my_name"
      And I send a GET request to "test"
      Then the JSON should be:
        """
        {
          "name": "{{ my_name }}",
          "age": "never you mind!"
        }
        """

Using values from environment variables

Step parameters will look up values from environment variables if they start with a "$". For example, after setting the following environment variable:

export BEHAVE_SERVER=http://localhost:5000/v1

we can run BDD tests which reference this variable

    Background: Set server URL and reset database
      Given I am using server "$BEHAVE_SERVER"
      And I set Accept header to "application/json"
      When I send a DELETE request to "people"
      Then the response status should be "200"

Test web callbacks (webhooks) using a mock HTTP server

Start a mock-http-server and write BDD as follows:

    TODO include example

Handle system delays

Some minimal support is provided for polling a URL for a given condition:

    Scenario: Polling for a state change
      Given I poll GET "counter" until JSON at path "value" is
        """
        5
        """

Note that this can be combined with the "templating tests from previous values" approach described above. The default number of polls and delays between polls may be found in the environment.py. They can also be adjusted as follows:

    Scenario: Polling for a state change
      Given I set context "n_attempts" to 3
      Given I poll GET "counter" until JSON at path "value" is
        """
        5
        """

(this example fails because 3 < 5)

The pause between attempts can be adjusted in a similar way.

Examples

See the *.feature files in the features folder.

Using this approach in another project

When it comes down to it there are only a few files which matter:

Simply copy the features directory into your project and replace the *.feature with your own tests. Invoke by running behave.

Updated