- Running out of retries, or unexpected screen state should make the
test FAIL, not ERROR.
- Uses levels to report the location of the caller.
- Improve message with retry-failure (formatting).
Before:
[ RUN ] test: 103.53 ms ERR
test/functional/helpers.lua:388:
retry() attempts: 1
test/functional/ui/screen.lua:587: Row 1 did not match.
Expected:
|*X^ |
|{0:~ }|
|{0:~ }|
| |
Actual:
|*^ |
|{0:~ }|
|{0:~ }|
| |
To print the expect() call that would assert the current screen state, use
screen:snapshot_util(). In case of non-deterministic failures, use
screen:redraw_debug() to show all intermediate screen states.
stack traceback:
test/functional/helpers.lua:388: in function 'retry'
test/functional/test_spec.lua:24: in function <test/functional/test_spec.lua:23>
After:
[ RUN ] test: 105.22 ms FAIL
test/functional/test_spec.lua:24: stopping after 1 retry() attempts.
test/functional/test_spec.lua:25: Row 1 did not match.
Expected:
|*X^ |
|{0:~ }|
|{0:~ }|
| |
Actual:
|*^ |
|{0:~ }|
|{0:~ }|
| |
To print the expect() call that would assert the current screen state, use
screen:snapshot_util(). In case of non-deterministic failures, use
screen:redraw_debug() to show all intermediate screen states.
stack traceback:
test/functional/helpers.lua:389: in function 'retry'
test/functional/test_spec.lua:24: in function <test/functional/test_spec.lua:23>
- Rename `meth_pcall`.
- Make `pcall_err` raise an error if the function does not fail.
- Add `vim.pesc()` to treat a string as literal where a Lua pattern is
expected.
For debugging failures like:
test/functional/helpers.lua:240: test/functional/ui/screen.lua:898:
bad argument #1 to 'unpack' (table expected, got number)
test/functional/helpers.lua:240: test/functional/ui/screen.lua:708:
attempt to index local 'item' (a number value)
ref #10804
Regardless of the comment "Big timeout for ASAN/valgrind" it would use
10s by default already.
This changes it to use `load_adjust`, which itself is only computed on
CI now, and outside of any tests - since it has side effects when being
used the first time!
The failure seen on AppVeyor:
[ RUN ] jobs can get the pid value using getpid: ERR
test\functional\helpers.lua:167:
==============================================================================
got 0 messages (ignored 0), expected 1
stack traceback:
test\functional\helpers.lua:167: in function 'expect_msg_seq'
test/functional\core\job_spec.lua:288: in function <test/functional\core\job_spec.lua:281>
Log: https://ci.appveyor.com/project/neovim/neovim/builds/26537324/job/y1io66fbx399q7h6?fullLog=true#L6554
* ci: AppVeyor: set GCOV_ERROR_FILE
This prevents the warnings/errors to be spilled into test results,
causing them to fail them, e.g.:
[ FAILED ] C:/projects/neovim/test/functional\core\main_spec.lua @ 97: Command-line option -s errors out when trying to use nonexistent file with -s
C:/projects/neovim/test/functional\core\main_spec.lua:98: Expected objects to be the same.
Passed in:
(string) 'Cannot open for reading: "Xtest-functional-core-main-s.nonexistent": no such file or directory
profiling:C:\projects\neovim\build/src/nvim/CMakeFiles/nvim.dir/buffer.c.gcda:Data file mismatch - some data files may have been concurrently updated without locking support
'
Expected:
(string) 'Cannot open for reading: "Xtest-functional-core-main-s.nonexistent": no such file or directory
'
stack traceback:
C:/projects/neovim/test/functional\core\main_spec.lua:98: in function <C:/projects/neovim/test/functional\core\main_spec.lua:97>
For reference, the locking appears to have been reworked for gcc 9.1 [1].
1: https://github.com/gcc-mirror/gcc/commit/56621355b
helpers.clear: keep GCOV_ERROR_FILE in environment
* ci: AppVeyor: remove MINGW_64 config (used with cov now)
Also:
- run MINGW_64-gcov first, and with PRs, since it provides coverage.
The test.functional.helpers and test.unit.helpers modules now include
all of the public functions from test.helpers, so there is no need to
separately require('test.helpers').
This is where "pure functions" can live, which can be shared by Nvim and
test logic which may not have a running Nvim instance available.
If in the future we use Nvim itself as the Lua engine for tests, then
these functions could be moved directly onto the `vim` Lua module.
closes#6580
Automatically include all "global helper" util functions in the
unit.helpers and functional.helpers and modules. So tests don't need to
expicitly do:
local global_helpers = require('test.helpers')
Callers can instead specify `args_rm={'--headless'}`.
TODO: should `nvim_argv` have "--headless" by default? Need to inspect
some uses of spawn(nvim_argv) ...
ref #9280
Introduce the `vim.compat` module, to help environments with system Lua
5.2+ run the build/tests. Include the module implicitly in all tests.
ref #8677
legacy `vim` module:
beep
buffer
command
dict
eval
firstline
lastline
line
list
open
type
window
- window_split_tab_spec.lua: Put cursor at bottom of :terminal buffer so
that it follows output.
- inccommand_spec.lua: Increase timeout to allow 2nd retry.
- Timer tests are less reliable on Travis CI macOS 10.12/10.13.
ref #6829
ref e39dade80b
ref de13113dc1
ref https://github.com/neovim/neovim/pull/9095#issuecomment-429603452
> We don't guarantee that a X ms timer is triggered during Y ms sleep
> for any X<Y, though I would expect the load to be really bad for this
> to happen with X=10ms, Y=40ms.
Give embeders a chance to set up nvim, by processing a request before
startup. This allows an external UI to show messages and prompts from
--cmd and buffer loading (e.g. swap files)
closes#8362
Vim's code calls `call_shell` directly from `get_system_output_as_rettv`
whereas in Nvim this function has been rewritten to not call `call_shell` but to call
`os_system` via `do_os_system`, losing the support for profiling and verbose.
Changing the code to call `call_shell` from `get_system_output_as_rettv`
seems to be too complicated to be worth it on the current version of the
code. So this commit duplicates the relevant code.
Justin M. Keyes