Execute the standard, non-selenium tests. The execution is
on the slower side: on my idle machine, KVM enabled it takes
a bit more than 2.5 minutes to execute it (executing tests with
4 threads parallel, 1/core, the default configuration). If the machine is
under load, it easily grows to over 10 minutes.
Added two backported patches for Django 5.2 to fix some tests that
would otherwise fail:
0001-Fix-test_strip_tags-test.patch: tag stripping tests failed due to
changed Python behavior
0001-fix-test_msgfmt_error_including_non_ascii-test.patch: tests were
updated to work with msgfmt 0.25
Most of the skipped tests require some specific database backend
(Postgres, MySQL, Oracle...) or are Selenium tests.
The output is very long (the suite contains way over 15k tests),
so I omit the example output here.
The current summary (for v5.2.9):
Ran 18121 tests in 140.891s
OK (skipped=1394, expected failures=5)
Signed-off-by: Gyorgy Sarvari <skandigraun@gmail.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
scrypt/fastjsonschema/pillow run into OOM on some qemu
architectures e.g. RISCV64. Raise the memory to be atleast 2G
for running these ptest images
Signed-off-by: Khem Raj <raj.khem@gmail.com>
In oe-core, PTESTS_PROBLEMS are also enabled, so even though
the recipe might have failures or the ptests do not run cleanly,
the BBCLASSEXTEND still works.
Enable the same behavior for meta-python.
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
These are test images to build all recipes in layer. Renaming them makes
them refect what they are. Moreover we can rename the ptest images to
match OE-Core naming conventions for meta-oe/meta-perl/meta-python
Signed-off-by: Khem Raj <raj.khem@gmail.com>
These were essentially duplicates of core-image-minimal, however
core-image-base is a better baseline for upper layers, so switched the
consumers of these images to use core-image-base
Signed-off-by: Khem Raj <raj.khem@gmail.com>
Similar to "core-image-ptest: Switch to BBCLASSEXTEND parallel execution"
Refactor meta-python-ptest-image.bb for an image per ptest in the form of
meta-python-ptest-image-XXX using class extensions.
* Add meta-python-ptest-all-image.bb depends on ALL meta-python-ptest-image-XXX in meta-python
* Add meta-python-ptest-fast-image.bb depends on meta-python ptests which take less than ~30s
This allows parallel execution of the tests.
The downside to this approach is the parsing time of an image generating
this number of class extensions is slow but making it easier to execute
and collect test data should outweigh that.
A useful advantage is that the dependencies of each ptest package are
checked/tested individually.
Add to local.conf:
IMAGE_CLASSES += "testimage"
All the meta-python-ptest-image-XXX can be built with:
$ bitbake meta-python-ptest-all-image
Subsequently, all the ptest images can be run with:
$ bitbake -c testimage meta-python-ptest-all-image
You will likely want to limit the number of parallel image tests:
BB_NUMBER_THREADS = "4"
based on the number of TAP devices you have defined or the number of simultaneous
QEMU sessions your server can support.
Results can be summarized with:
$ resulttool report tmp/log/oeqa/testresults.json
Signed-off-by: Tim Orling <tim.orling@konsulko.com>
Signed-off-by: Khem Raj <raj.khem@gmail.com>
