|Justin M. Keyes 0190de9aab Merge #11307 'Lua: vim.validate()'||9 hours ago|
|benchmark||2 years ago|
|busted/outputHandlers||1 month ago|
|config||2 years ago|
|functional||9 hours ago|
|includes||3 months ago|
|symbolic/klee||7 months ago|
|unit||3 weeks ago|
|README.md||1 month ago|
|helpers.lua||2 weeks ago|
You can learn the key concepts of Lua in 15 minutes. Use any existing test as a template to start writing new tests.
Tests are run by
/cmake/RunTests.cmake file, using
busted (a Lua test-runner).
For some failures,
$NVIM_LOG_FILE) may provide insight.
Depending on the presence of binaries (e.g.,
xclip) some tests will be
ignored. You must compile with libintl to prevent
E319: The command is not
available in this version errors.
/test/functional: functional tests
/test/unit: unit tests
*.infiles which are transformed into
configure_fileCMake command: this is for acessing CMake variables in lua tests.
/test/includes: include-files for use by luajit
ffi.cdefC definitions parser: normally used to make macros not accessible via this mechanism accessible the other way.
/test/*/preload.lua: modules preloaded by busted
/test/**/helpers.lua: common utility functions for test code
/test/*/**/*_spec.lua: actual tests. Files that do not end with
_spec.luaare libraries like
/test/**/helpers.lua, except that they have some common topic.
/src/nvim/testdir: old tests (from Vim)
To run all tests (except “old” tests):
To run only unit tests:
To run only functional tests:
To run all legacy Vim tests:
To run a single legacy test set
TEST_FILE, for example:
TEST_FILE=test_syntax.res make oldtest
.resextension (instead of
.vim) is required.
$GDBto run tests under gdbserver. And if
$VALGRINDis set it will pass
--vgdb=yesto valgrind instead of starting gdbserver directly.
:h press-enterprompts. The default screen width is 50 columns. Commands that try to print lines longer than 50 columns in the command-line, e.g.
:edit very...long...path, will trigger the prompt. In this case, a shorter path or
:silent editshould be used.
If you can’t figure out what is going on, try to visualize the screen. Put this at the beginning of your test:
local Screen = require('test.functional.ui.screen') local screen = Screen.new() screen:attach()
screen:snapshot_util() at any position in your test. See the
comment at the top of
test/functional/ui/screen.lua for more.
Another filter method is by setting a pattern of test name to
it('foo api',function() ... end) it('bar api',function() ... end)
To run only test with filter name:
TEST_FILTER='foo.*api' make functionaltest
To run a specific unit test:
TEST_FILE=test/unit/foo.lua make unittest
To run a specific functional test:
TEST_FILE=test/functional/foo.lua make functionaltest
To repeat a test:
BUSTED_ARGS="--repeat=100 --no-keep-going" TEST_FILE=test/functional/foo_spec.lua make functionaltest
Tests can be “tagged” by adding
# before a token in the test description.
it('#foo bar baz', function() ... end) it('#foo another test', function() ... end)
To run only the tagged tests:
TEST_TAG=foo make functionaltest
TEST_FILEis not a pattern string like
TEST_FILTER. The given value to
TEST_FILEmust be a path to an existing file.
TEST_FILTERfilter tests by the string descriptions found in
types.h, so types used in the tested functions could be moved to it to avoid having to rewrite the declarations in the test files.
#defineconstants must be rewritten
enumso they can be “visible” to the tests.
if-else. If a functional test depends on some external factor (e.g. the existence of
$PATH), and you can’t mock or fake the dependency, then skip the test via
pending()if the external factor is missing. This ensures that the total test-count (success + fail + error + pending) is the same in all environments.
source([=[...]=])blocks may break Vim’s Lua syntax highlighting. Try
:syntax sync fromstartto fix it.
/test/functional are divided into groups
by the semantic component they are testing.
src/nvim/, because they are testing functions directly. E.g. unit-tests for
src/nvim/undo.cshould live in
test/functional/*/*_spec.luagroup that makes sense, before creating a new one.
make lint (and
make lualint) runs luacheck
on the test code.
If a luacheck warning must be ignored, specify the warning code. Example:
-- luacheck: ignore 621
Ignore the smallest applicable scope (e.g. inside a function, not at the top of the file).
Test behaviour is affected by environment variables. Currently supported (Functional, Unit, Benchmarks) (when Defined; when set to 1; when defined, treated as Integer; when defined, treated as String; when defined, treated as Number; !must be defined to function properly):
BUSTED_ARGS (F) (U): arguments forwarded to
GDB (F) (D): makes nvim instances to be run under
gdbserver. It will be
gdb build/bin/nvim, type
GDBSERVER_PORT (F) (I): overrides port used for
VALGRIND (F) (D): makes nvim instances to be run under
files are named
valgrind-%p.log in this case. Note that non-empty valgrind
log may fail tests. Valgrind arguments may be seen in
/test/functional/helpers.lua. May be used in conjunction with
VALGRIND_LOG (F) (S): overrides valgrind log file name used for
TEST_SKIP_FRAGILE (F) (D): makes test suite skip some fragile tests.
NVIM_PRG (F) (S): override path to Neovim executable (default
CC (U) (S): specifies which C compiler to use to preprocess files.
Currently only compilers with gcc-compatible arguments are supported.
NVIM_TEST_MAIN_CDEFS (U) (1): makes
ffi.cdef run in main process. This
raises a possibility of bugs due to conflicts in header definitions, despite
the counters, but greatly speeds up unit tests by not requiring
do parsing of big strings with C definitions.
NVIM_TEST_PRINT_I (U) (1): makes
cimport print preprocessed, but not yet
formatc headers. Used to debug
formatc. Printing is done
with the line numbers.
NVIM_TEST_PRINT_CDEF (U) (1): makes
cimport print final lines which will
be then passed to
ffi.cdef. Used to debug errors
ffi.cdef happens to
NVIM_TEST_PRINT_SYSCALLS (U) (1): makes it print to stderr when syscall
wrappers are called and what they returned. Used to debug code which makes
unit tests be executed in separate processes.
NVIM_TEST_RUN_FAILING_TESTS (U) (1): makes
itp run tests which are known
to fail (marked by setting third argument to
LOG_DIR (FU) (S!): specifies where to seek for valgrind and ASAN log files.
NVIM_TEST_CORE_* (FU) (S): a set of environment variables which specify
where to search for core files. Are supposed to be defined all at once.
NVIM_TEST_CORE_GLOB_DIRECTORY (FU) (S): directory where core files are
located. May be
.. This directory is then recursively searched for core
files. Note: this variable must be defined for any of the following to have
NVIM_TEST_CORE_GLOB_RE (FU) (S): regular expression which must be matched
by core files. E.g.
/core[^/]*$. May be absent, in which case any file is
considered to be matched.
NVIM_TEST_CORE_EXC_RE (FU) (S): regular expression which excludes certain
directories from searching for core files inside. E.g. use
^/%.deps$ to not
/.deps. If absent, nothing is excluded.
NVIM_TEST_CORE_DB_CMD (FU) (S): command to get backtrace out of the
gdb -n -batch -ex "thread apply all bt full"
"$_NVIM_TEST_APP" -c "$_NVIM_TEST_CORE". Defaults to the example command.
This debug command may use environment variables
_NVIM_TEST_APP (path to
application which is being debugged: normally either nvim or luajit) and
_NVIM_TEST_CORE (core file to get backtrace from).
NVIM_TEST_CORE_RANDOM_SKIP (FU) (D): makes
check_cores not check cores
after approximately 90% of the tests. Should be used when finding cores is
too hard for some reason. Normally (on OS X or when
NVIM_TEST_CORE_GLOB_DIRECTORY is defined and this variable is not) cores
are checked for after each test.
NVIM_TEST_RUN_TESTTEST (U) (1): allows running
test/unit/testtest_spec.lua used to check how testing infrastructure works.
NVIM_TEST_TRACE_LEVEL (U) (N): specifies unit tests tracing level:
0disables tracing (the fastest, but you get no data if tests crash and there no core dump was generated),
1leaves only C function calls and returns in the trace (faster than recording everything),
2records all function calls, returns and executed Lua source lines.
NVIM_TEST_TRACE_ON_ERROR (U) (1): makes unit tests yield trace on error in
addition to regular error message.
NVIM_TEST_MAXTRACE (U) (N): specifies maximum number of trace lines to
keep. Default is 1024.