Writing Tests for github3.py — github3.py 4.0.1 documentation (original) (raw)
Unit Tests
In computer programming, unit testing is a method by which individual units of source code, sets of one or more computer program modules together with associated control data, usage procedures, and operating procedures are tested to determine if they are fit for use. Intuitively, one can view a unit as the smallest testable part of an application.
In github3.py we use unit tests to make assertions about how the library behaves without making a request to the internet. For example, one assertion we might write would check if custom information is sent along in a request to GitHub.
An existing test like this can be found intests/unit/test_repos_release.py:
def test_delete(self): self.instance.delete() self.session.delete.assert_called_once_with( self.example_data['url'], headers={'Accept': 'application/vnd.github.manifold-preview'} )
In this test, we check that the library passes on important headers to the API to ensure the request will work properly. self.instance is created for us and is an instance of the Release class. The test then calls delete to make a request to the API. self.session is a mock object which fakes out a normal session. It does not allow the request through but allows us to verify how github3.py makes a request. We can see that github3.py called deleteon the session. We assert that it was only called once and that the only parameters sent were a URL and the custom headers that we are concerned with.
Mocks
Above we talked about mock objects. What are they?
In object-oriented programming, mock objects are simulated objects that mimic the behavior of real objects in controlled ways. A programmer typically creates a mock object to test the behavior of some other object, in much the same way that a car designer uses a crash test dummy to simulate the dynamic behavior of a human in vehicle impacts.
We use mocks in github3.py to prevent the library from talking directly with GitHub. The mocks we use intercept requests the library makes so we can verify the parameters we use. In the example above, we were able to check that certain parameters were the only ones sent to a session method because we mocked out the session.
You may have noticed in the example above that we did not have to set up the mock object. There is a convenient helper written in tests/unit/helper.pyto do this for you.
Example - Testing the Release Object
Here’s a full example of how we test the Release object intests/unit/test_repos_release.py.
Our first step is to import the UnitHelper class fromtests/unit/helper.py and the Release object fromgithub3/repos/release.py.
from .helper import UnitHelper from github3.repos.release import Release
Then we construct our test class and indicate which class we will be testing (or describing).
class TestRelease(UnitHelper): described_class = Release
We can then use the GitHub API documentation about Releases to retrieve example release data. We then can use that as example data for our test like so:
class TestRelease(UnitHelper): described_class = Release example_data = { "url": releases_url("/1"), "html_url": "https://github.com/octocat/Hello-World/releases/v1.0.0", "assets_url": releases_url("/1/assets"), "upload_url": releases_url("/1/assets{?name}"), "id": 1, "tag_name": "v1.0.0", "target_commitish": "master", "name": "v1.0.0", "body": "Description of the release", "draft": False, "prerelease": False, "created_at": "2013-02-27T19:35:32Z", "published_at": "2013-02-27T19:35:32Z" }
The above code now will handle making clean and brand new instances of theRelease object with the example data and a faked out session. We can now construct our first test.
def test_delete(self): self.instance.delete() self.session.delete.assert_called_once_with( self.example_data['url'], headers={'Accept': 'application/vnd.github.manifold-preview'} )
Integration Tests
Integration testing is the phase in software testing in which individual software modules are combined and tested as a group.
The purpose of integration testing is to verify functional, performance, and reliability requirements placed on major design items.
In github3.py we use integration tests to ensure that when we make what should be a valid request to GitHub, it is in fact valid. For example, if we were testing how github3.py requests a user’s information, we would expect a request for a real user’s data to be valid. If the test fails we know either what the library is doing is wrong or the data requested does not exist.
An existing test that demonstrates integration testing can be found intests/integration/test_repos_release.py:
def test_iter_assets(self): """Test the ability to iterate over the assets of a release.""" cassette_name = self.cassette_name('iter_assets') with self.recorder.use_cassette(cassette_name): repository = self.gh.repository('sigmavirus24', 'github3.py') release = repository.release(76677) for asset in release.iter_assets(): assert isinstance(asset, github3.repos.release.Asset) assert asset is not None
In this test we use self.recorder to record our interaction with GitHub. We then proceed to make the request to GitHub that will exercise the code we wish to test. First we request a Repository object from GitHub and then using that we request a Release object. After receiving that release, we exercise the code that lists the assets of a Release. We verify that each asset is an instance of the Asset class and that at the end the assetvariable is not None. If asset was None, that would indicate that GitHub did not return any data and it did not exercise the code we are trying to test.
Betamax
Betamax is the library that we use to create the recorder above. It sets up the session object to intercept every request and corresponding response and save them to what it calls cassettes. After you record the interaction it never has to speak to the internet again for that request.
In github3.py there is a helper class (much like UnitHelper) intests/integration/helper.py which sets everything up for us.
Example - Testing the Release Object
Here’s an example of how we write an integration test for github3.py. The example can be found in tests/integration/test_repos_release.py.
Our first steps are the necessary imports.
import github3
from .helper import IntegrationHelper
Then we start writing our test right away.
class TestRelease(IntegrationHelper): def test_delete(self): """Test the ability to delete a release.""" self.token_login() cassette_name = self.cassette_name('delete') with self.recorder.use_cassette(cassette_name): repository = self.gh.repository('github3py', 'github3.py') release = repository.create_release( '0.8.0.pre', 'develop', '0.8.0 fake release', 'To be deleted' ) assert release is not None assert release.delete() is True
Every test has access to self.gh which is an instance of GitHub.IntegrationHelper provides a lot of methods that allow you to focus on what we are testing instead of setting up for the test. The first of those methods we see in use is self.token_login which handles authenticating with a token. It’s sister method is self.basic_login which handles authentication with basic credentials. Both of these methods will set up the authentication for you on self.gh.
The next convenience method we see is self.cassette_name. It constructs a cassette name for you based on the test class name and the string you provide it.
Every test also has access to self.recorder. This is the Betamax recorder that has been set up for you to record your interactions. The recorder is started when you write
with self.recorder.use_cassette(cassette_name): # …
Everything that talks to GitHub should be written inside of the context created by the context manager there. No requests to GitHub should be made outside of that context.
In that context, we then retrieve a repository and create a release for it. We want to be sure that we will be deleting something that exists so we assert that what we received back from GitHub is not None. Finally we calldelete and assert that it returns True.
When you write your new test and record a new cassette, be sure to add the new cassette file to the repository, like so:
git add tests/cassettes/Release_delete.json
Recording Cassettes that Require Authentication/Authorization
If you need to write a test that requires an Authorization (i.e., OAuth token) or Authentication (i.e., username and password), all you need to do is set environment variables when running py.test, e.g.,
GH_AUTH="abc123" py.test GH_USER="sigmavirus24" GH_PASSWORD="super-secure-password-plz-kthxbai" py.test
If you are concerned that your credentials will be saved, you need not worry. Betamax sanitizes information like that before saving the cassette. It never does hurt to double check though.