pytest-inline: An inline testing tool for Python

Abstract

We present pytest-inline, the first inline testingframework for Python. We recently proposed inline tests tomake it easier to test individual program statements, but thereis currently no framework-level support available for developersto write inline tests in Python. To fill this gap, we design andimplement pytest-inline as a pytest plugin, which is the mostpopular Python testing framework. In pytest-inline, a developercan write inline tests by assigning test inputs to variablesin the target statement and specifying the expected outputs.Then, pytest-inline runs each inline test and fails if the targetstatement’s output does not match the expected result. In thispaper, we describe the design of pytest-inline, the testing featuresthat it provides, and the intended use cases. Our evaluation ofpytest-inline on the inline tests we wrote for 80 target statementsfrom 31 open-source Python projects shows that using it to runinline tests incurs negligible overhead, at 0.012x. pytest-inline isopen-sourced, and a video demo of pytest-inline can be found athttps://www.youtube.com/watch?v=pZgiAxR_uJg.