el fuego

LosTests Pub

Test result publisher for Docker and Drone CI plugin to collect test results

Drone creates a very convenient environment to build your software and run tests (as well as package, deploy, and whatever you want, but right now we aren't about it). The problem actually is that it can only report if the build was successful or not. But it can't analyse test results and report which exactly tests fail, what exactly failures and/or errors occured.

LosTests Pub does provide you with such functionality.

Configuring the publisher

Modify your Drone docker-compose.yml file as follows:

version: '2'

services:
  drone-server:
    image: drone/drone:1
    [...]

+ lostests-pub:
+   image: elfuego/lostests-pub:1
+
+   ports:
+     - 8001:80
+   restart: always
+   volumes:
+     - /var/lib/lostests-pub:/var/lib/lostests-pub/
+   environment:
+     - LOSTESTS_HOST=http://<lostests-pub host>:8001
+     - LOSTESTS_SECRET=123456</span>

LostTests Pub mounts a data volume to persist the sqlite database.

Configuring test result collector plugin

For now only one plugin is available: LosTests Unit which supports: JUnit (Java/Android), Unittest (pure Python, Django), pytest (Python), PHPUnit, Golang go test.

Modify your pipeline in .drone.yml so that build step contains this command (for Python/Django):

kind: pipeline
[...]
steps:
  - name: build
    [...]
    commands:
      [...]
      - python3 manage.py test --noinput --testrunner="xmlrunner.extra.djangotestrunner.XMLTestRunner"

or this command (for pure Python/unittest):

kind: pipeline
[...]
steps:
  - name: build
    [...]
    commands:
      [...]
      - echo "import unittest as u; import xmlrunner as x; s = u.TestLoader().discover('src', 'test*.py', None); exit(not x.XMLTestRunner().run(s).wasSuccessful())" | python3 -

or this command (for pure Python/pytest):

kind: pipeline
[...]
steps:
  - name: build
    [...]
    commands:
      [...]
      - py.test --junitxml=TEST-pytest.xml

or these commands (for Golang):

kind: pipeline
[...]
steps:
  - name: build
    image: golang
    commands:
      [...]
      - go get -u github.com/jstemmer/go-junit-report
      - go test -v -benchmem -bench . ./... 2>&1 | go-junit-report -set-exit-code >TEST-test.xml

to generate test results in XML format. After that you only need to add such a simple step as follows:

kind: pipeline
[...]
steps:
  [...]
  - name: report-tests
    image: elfuego/lostests-unit:1
    url: http://<lostests-pub host>:8001
    relpath: test-reports
    secret:
      from_secret: lostests_secret
    when:
      status: [ success, failure ]

Profit!

Consider to create a repository secret _lostestssecret with value specified for lostests-pub (123456 in our example).

Now you can add a badge to your README.md:

[![tests](http://<lostests-pub host>:8001/badge/<user-name>/<repo-name>/tests.svg)](http://<lostests-pub host>:8001/tests/<user-name>/<repo-name>/)

or directly go to the link in your browser:

http://<lostests-pub host>:8001/tests/<user-name>/<repo-name>/

Here you can see all possible LosTests Unit environment variables.

Screenshots

Test result badge (a view from Gitea page):

Screenshots of LosTests Pub itself:

A build view

A test suite with an error and a failure

A test suite with a skipped test

A standard error view

A standard output view

An error test case

A failed test case

A skipped test case

A successful test case

el fuego © 2018-2019