Jan 1, 2018

In this post I will go over a Python module I made that makes it really easy to test HTTP JSON APIs and print a pretty result in the terminal. I made this module really for myself as I create some new APIs using TypeScript and express. I originally dabbled a bit with Mocha and Chai since I was using npm anyways, but ultimately realized that Python and requests was a great candidate for a no fuss HTTP testing client, given how great they are for web crawling. In a future post I will go over the actual application I was testing in more depth, but only superficially here.

Overview

We will go through the following topics:

  1. Example
  2. Installing the TestServer
  3. HTTP Assertions
  4. Reusable Tests
  5. Creating a Generic Test
  6. Test Ideas

Example

I am making an authentication API in TypeScript similar to how I did in both Python and in PHP in the past. I enjoy using Python but ultimately wanted to rely on a statically typed language for my APIs similar to how I have been for my front end applications in Angular. Funny enough I thought Python was a better candidate for testing due to how great and simple it is at web crawling and http requests.

See below for an example where in Postman I had thought a user could not do admin functions because they were receiving a 403 response (Unauthorized). The Python tests showed that even though they received a 403 response the code continued to execute and a user was able to create an admin! This was solved simply by adding a return to the unauthorized response.

Full Code

If you’d prefer to just get straight to the code, you can find it here on GitHub.

Installing the Test Server

Navigate to the directory you’d like to perform tests in. Type or copy the following commands into the terminal:

git clone https://github.com/adcostanza/http-json-api-tests-python .
virtualenv -p python3 env
source ./env/bin/activate
pip install -r requirements.txt
touch tests.py

You may now import the TestServer as a variable into your own test file tests.py:

import TestServer as test

HTTP Assertions

With the module I have provided you must do an “any” assert using a POST, GET, PUT, or DELETE request. See below for the arguments you may pass into your HTTP assertion. It will by default check if the status of the response is 200, which you may change to a different status code (i.e. 403) if desired.

anyAssert(type, url, description, data=[], tests=[], headers={}, statusExpected=200):

Reusable Tests

The following are tests/assertions that I have built so far, but you may build others just as easily:

  1. hasProperty(self,property,should)
  2. hasTrueProperty(self,property)
  3. hasFalseProperty(self,property)
  4. hasPropertyEqualTo(self, property, equals, should)

In all cases, should is a string that should contain language such as have a token for the error messages. These tests are reusable because once you create one you can use it with many anyAsserts:

#Tests that we can use in any of our assertions
success = test.hasTrueProperty('success')
fail = test.hasFalseProperty("success")
hasToken = test.hasProperty("token","have token in response")

#Logging in and loading the response body into a variable
body = test.anyAssert('post','/login','first login as adam',{'username':'adam','password':'tacos'},[hasToken])
#Retrieve the token from the response
token = test.getValueFromResponse(body,'token')

#Some other tests
test.anyAssert('post','/passwords','change adams password to tac0', {'password':'tac0'},[success],{'x-access-token':token})

As you can see the same success assertion is reused in the call to POST /passwords/

Generic Tests

Below you can see an example of one of the tests above, the hasTrueProperty which is a little simpler than the tests that have a should statement. Ultimately you can structure a test however you want, I pretty much made this up based on wanting more flexibility but liking some of what I saw in Chai and Mocha.

class hasTrueProperty:
    def __init__(self,property):
        self.property = property
    def check(self, url, body):
        try:
            if body[self.property]:
                print(c.OKGREEN + "✓ POST ⤳ "+url+": should have property "+str(self.property)+":true" + c.ENDC)
                return
            exception = "✗ POST ⤰ "+url+": should have property "+str(self.property)+":true but it is "+str(body[self.property])
            print(c.FAIL + exception+ c.ENDC)
        except Exception as e:
            exception = "✗ POST ⤰ "+url+": should have property "+str(self.property)+":true but the property does not exist "+str(body[self.property])
            print(c.FAIL + exception+ str(e) +c.ENDC)

The basic structure of a test is as follows:

class hasTrueProperty:
    def __init__(self,property,should,...):
        self.property = property
    def check(self, url, body):
        if <<logic>>:
            print(c.OKGREEN + "✓ POST ⤳ "+url+"<<should statement>>" + c.ENDC)
            return
        exception = "✗ POST ⤰ "+url+"<<should statement>>
        print(c.FAIL + exception+ c.ENDC)

The should property is not required, neither is property. You can really do whatever you’d like with a Generic Test but it should be just that, generic and have a check return method.

Test Ideas

What is really nice about how this module is set up is that you can set up a few very basic tests, such as whether the response has a property or if it is true, etc. and then pass those tests as objects into your http assertion. You’ll get pretty console output right out the gate and know if your JSON API is really doing what it should be.

One thing you can do is make a list of objects related to your api and use a for loop to post them all to your API really quickly, or perhaps even asynchronously. Doing tests automatically and quickly like this ultimately just better ensures your software does what you think it does. It’s better to be caught with an error now rather than the customer catching it in three months.