|GitLab Bot f698fdbdcb Add latest changes from gitlab-org/gitlab@master||4 hours ago|
|bin||Fix bootable scenario arguments for OptionParser||2 years ago|
|knapsack||Add latest changes from gitlab-org/gitlab@master||2 months ago|
|load||Fix url property format and use conn. pool||8 months ago|
|qa||Add latest changes from gitlab-org/gitlab@master||4 hours ago|
|spec||Add latest changes from gitlab-org/gitlab@master||6 days ago|
|.gitignore||Add latest changes from gitlab-org/gitlab@master||4 days ago|
|.rspec||Add GitLab QA integrations tests to GitLab CE / EE||2 years ago|
|.rspec_parallel||Run tests in parallel via parallel_tests||5 months ago|
|Dockerfile||Add latest changes from gitlab-org/gitlab@master||5 days ago|
|Gemfile||Add latest changes from gitlab-org/gitlab@master||1 week ago|
|Gemfile.lock||Add latest changes from gitlab-org/gitlab@master||5 days ago|
|README.md||Replace GDK ref||3 months ago|
|Rakefile||Add script for MR with many commits||6 months ago|
|qa.rb||Add latest changes from gitlab-org/gitlab@master||2 days ago|
This directory contains end-to-end tests for GitLab. It includes the test framework and the tests themselves.
The tests can be found in
qa/specs/features (not to be confused with the unit
tests for the test framework, which are in
It is part of the GitLab QA project.
GitLab QA is an end-to-end tests suite for GitLab.
These are black-box and entirely click-driven end-to-end tests you can run against any existing instance.
We recently added a new CI job that is going to be triggered for every push
event in CE and EE projects. The job is called
qa:selectors and it will
verify coupling between page objects implemented as a part of GitLab QA
and corresponding views / partials / selectors in CE / EE.
qa:selectors job fails in your merge request, you are supposed to
fix page objects. You should also trigger end-to-end tests
package-and-qa-manual manual action, to test if everything works fine.
You can use GitLab QA to exercise tests on any live instance! If you don’t have an instance available you can follow the instructions below to use the GitLab Development Kit (GDK). This is the recommended option if you would like to contribute to the tests.
Once you have GDK running, switch to the
qa directory. E.g., if you setup
GDK to develop in the main
gitlab-ce repo, the GitLab source code will be
gitlab directory and so the end-to-end test code will be in
From there you can run the tests. For example, the
following call would login to the GDK instance and run all specs in
# Make sure to install the dependencies first with `bundle install` bundle exec bin/qa Test::Instance::All http://localhost:3000
Note: If you want to run tests requiring SSH against GDK, you will need to modify your GDK setup.
You can also supply specific tests to run as another parameter. For example, to run the repository-related specs, you can execute:
bundle exec bin/qa Test::Instance::All http://localhost:3000 -- qa/specs/features/browser_ui/3_create/repository
Since the arguments would be passed to
rspec, you could use all
options there. For example, passing
--backtrace and also line number:
bundle exec bin/qa Test::Instance::All http://localhost:3000 -- qa/specs/features/browser_ui/3_create/merge_request/create_merge_request_spec.rb:6 --backtrace
Note that the separator
-- is required; all subsequent options will be
ignored by the QA framework and passed to
Unless told otherwise, the QA tests will run as the default
root user seeded
by the GDK.
If you need to authenticate as a different user, you can provide the
GITLAB_PASSWORD environment variables:
GITLAB_USERNAME=jsmith GITLAB_PASSWORD=password bundle exec bin/qa Test::Instance::All https://gitlab.example.com
Some QA tests require logging in as an admin user. By default, the QA
tests will use the the same
root user seeded by the GDK.
If you need to authenticate with different admin credentials, you can
GITLAB_ADMIN_USERNAME=admin GITLAB_ADMIN_PASSWORD=myadminpassword GITLAB_USERNAME=jsmith GITLAB_PASSWORD=password bundle exec bin/qa Test::Instance::All https://gitlab.example.com
If your user doesn’t have permission to default sandbox group
gitlab-qa-sandbox, you could also use another sandbox group by giving
GITLAB_USERNAME=jsmith GITLAB_PASSWORD=password GITLAB_SANDBOX_NAME=jsmith-qa-sandbox bundle exec bin/qa Test::Instance::All https://gitlab.example.com
The environment variable
QA_COOKIES can be set to send additional cookies
on every request. This is necessary on gitlab.com to direct traffic to the
canary fleet. To do this set
To set multiple cookies, separate them with the
; character, for example:
Once you have made changes to the CE/EE repositories, you may want to build a
Docker image to test locally instead of waiting for the
gitlab-ee-qa nightly builds. To do that, you can run from the top
directory (one level up from this directory):
docker build -t gitlab/gitlab-ce-qa:nightly --file ./qa/Dockerfile ./
Tests can be put in quarantine by assigning
:quarantine metadata. This means
they will be skipped unless run with
--tag quarantine. This can be used for
tests that are expected to fail while a fix is in progress (similar to how
can be used).
bundle exec bin/qa Test::Instance::All http://localhost:3000 -- --tag quarantine
quarantine is used with other tags, tests will only be run if they have at
least one of the tags other than
quarantine. This is different from how RSpec
tags usually work, where all tags are inclusive.
For example, suppose one test has
:quarantine metadata, and
another test has
:quarantine metadata. If the tests are run with
--tag smoke --tag quarantine, only the first test will run. The test with
:ldap will not run even though it also has
Tests can be run with with a feature flag enabled by using the command-line
--enable-feature FEATURE_FLAG. For example, to enable the feature flag
that enforces Gitaly request limits, you would use the command:
bundle exec bin/qa Test::Instance::All http://localhost:3000 --enable-feature gitaly_enforce_requests_limits
This will instruct the QA framework to enable the
feature flag (via the API), run
all the tests in the
Test::Instance::All scenario, and then disable the
feature flag again.
Note also that the
-- separator isn’t used because
--enable-feature is a QA
framework option, not an