1094 Commits

Author SHA1 Message Date
Richard Purdie
1cb8768b28 oeqa/selftest: Fix broken symlink removal handling
The test above this removal correctly looks at symlinks however to
remove a symlink we should call unlink(), not remove(). This avoids
some build failures/tracebacks.

(From OE-Core rev: 830412641abe8c7a3ad5eb1709c8372c796a7760)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit dbdb6e73b0f52bc5f9429aca47802d51edbbc834)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-10-25 04:51:01 -10:00
Richard Purdie
5bad01b1be oeqa/selftest/wic: Improve assertTrue calls
assertTrue is a problematic call use in test cases since when it fails,
you just get an unhelpful "False is not True" message.

Replace some uses with assertIn/assertNotIn which will give more helpful results
and for the rest, add msg entries which given more helpful debugging.

For example, this patch would help debugging of #15176.

(From OE-Core rev: 52a2455ba7d91d404fc2c4568c805cf1fbe2b2ad)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 35d4c39e0df1a304f557471151a03d1e4b0f30c7)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-10-18 05:25:19 -10:00
Mikko Rapeli
c5d0496e16 oeqa/selftest/context.py: check git command return values
Don't ignore return values from the git command lines. If something goes
wrong, fail the test right away.

(From OE-Core rev: 776053f4af19e9c2de7ce725e6ec94312028fa7f)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit dfc178a70d6fa60e89d4716f05d68e2c72c6ecd3)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-10-18 05:25:19 -10:00
Alexander Kanavin
4284cd0c18 build-sysroots: target or native sysroot population need to be selected explicitly
Running them in parallel is prone to races as postinsts from target sysroots
rely on executables from native sysroots which may or may not be fully prepared
yet. This was observed for example here:

https://autobuilder.yoctoproject.org/typhoon/#/builders/146/builds/468/steps/12/logs/stdio
https://autobuilder.yoctoproject.org/typhoon/#/builders/147/builds/467/steps/12/logs/stdio

(From OE-Core rev: 93e864a23e3f82edc2b2fa2207b2bc4b56c230a9)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit 38d7a2e45b883cf999a86af05bcc0eaa875bb47c)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-10-18 05:25:19 -10:00
Mikko Rapeli
8acce65aa5 oeqa selftest context.py: remove warning from missing meta-selftest
It's not a warning but a handled case and layer gets added
automatically. Very few build configs have this layer enabled
by default.

(From OE-Core rev: 70d2ceea48ef6ec04675e4e6a9c48ebc5bb78f52)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
(cherry picked from commit 9a2493ea83f0b30578a819de05108502aaadc7f5)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-10-13 04:31:05 -10:00
Mikko Rapeli
0198d514d6 oeqa selftest context.py: whitespace fix
(From OE-Core rev: fa4204c34efeda543a108c942cd50a6d6322600e)

Signed-off-by: Mikko Rapeli <mikko.rapeli@linaro.org>
(cherry picked from commit 8aed3ca405cb3cc5d346eb8da8b3307d93f75d5b)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-10-13 04:31:05 -10:00
Anuj Mittal
9c34365a3d selftest/cases/glibc.py: switch to using NFS over TCP
This provides a more reliable test execution when running tests that
write a large buffer/file and significantly reduces the localedata test
failures.

(From OE-Core rev: e138b648a56a0146588dc52edd9b44929ed8a73e)

Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 97a7612e3959bc9c75116a4e696f47cc31aea75d)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-08-15 06:18:49 -10:00
Anuj Mittal
13a94b094b selftest/cases/glibc.py: increase the memory for testing
Some of the tests trigger OOM and fail. Increase the amount of memory
available so we dont run into these issues.

(From OE-Core rev: b4e7158aca35235c86afee2b68236e250488ef98)

Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 4d22dba482cb19ffcff5abee73f24526ea9d1c2a)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-08-15 06:18:49 -10:00
Anuj Mittal
07fc9ad254 selftest/cases/glibc.py: fix the override syntax
Fix the override so we actually pass the correct value to glibc.

(From OE-Core rev: 8ad6000c814631e1147fddd5d2462b39ea44b99e)

Signed-off-by: Anuj Mittal <anuj.mittal@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 38fd2120f0f48512091ddad6205ce19839eaf589)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-08-15 06:18:49 -10:00
Yoann Congal
29ef950125 oeqa/selftest/devtool: add unit test for "devtool add -b"
Fix [Yocto #15085]

Co-authored-by: Fawzi KHABER <fawzi.khaber@smile.fr>
(From OE-Core rev: 2dca7cba28a08d63e0ca88bb9897e01661e177ef)

Signed-off-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit d5eedf8ca689ccb433c2f5d0b324378f966dd627)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-08-01 06:17:28 -10:00
Fabien Mahot
5a3a4e3e49 oeqa/selftest/bbtests: add non-existent prefile/postfile tests
Fixes [YOCTO #10725]

(From OE-Core rev: 47966f06f0c221b00d773f8e9db20190a80d668a)

Signed-off-by: Fabien Mahot <fabien.mahot@smile.fr>
Reviewed-by: Yoann Congal <yoann.congal@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b0c33655fad5b2e7d96a45b6210527dfb766797b)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-07-04 05:32:29 -10:00
Richard Purdie
bb85801f7f selftest/reproducible: Allow native/cross reuse in test
We don't compare reproducibility of the native/cross components, only the
target ones. With the long build times of rust-native, the test now takes
crazy lengths of time so this tweak should allow us to reuse native/cross
artefacts from sstate whilst still testing the target output is
reproducible.

(From OE-Core rev: ea3d21235633dc5b01c6ce39e768d00d1da5d87a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit b494d83c639a877cefeb7cbab6d37195e492f059)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-06-14 04:14:25 -10:00
Thomas Roos
387963e269 oeqa: adding selftest-hello and use it to speed up tests
Adding a selftest version of hello world to run it in tests where no download is necessary.
Also using this in several tests to speed them up.
Using the -native version wherever possible will also speed up tests a lot.

[YOCTO #11142]

(From OE-Core rev: 95241ece5fcef53901b97f96ae961cea208ca31a)

Signed-off-by: Thomas Roos <throos@amazon.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
(cherry picked from commit c3f26b63934888df0e3cd563c1c2804eb78a368e)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-06-14 04:14:25 -10:00
Thomas Roos
c99e034431 oeqa/selftest/cases/devtool.py: skip all tests require folder a git repo
Devtool selftests require poky dir a git repo, when downloading poky as a tar,
this is not the case. Those tests will now skipped.

[YOCTO #12389]

(From OE-Core rev: 5b913494b472ef03435cabf984361b43c3d813f4)

Signed-off-by: Thomas Roos <throos@amazon.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 95a5bc130dc51ea9de95c64dbf0e9c7892415d50)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-06-14 04:14:25 -10:00
Alexander Kanavin
9fa1f1403f selftest/distrodata: clean up exception lists in recipe maintainers test
Specifically:
- add missing maintainer.inc entries for initramfs-module-*, systemd-machine-units and
target-sdk-provides-dummy and drop them from exception list.

- remove rust from exception list for unbuildable-by-default recipes as it is now buildable.

- add missing maintainer.inc entry for libx11-compose-data and cve-update-nvd2-native;
as they are also unbuildable by default, they needs to be in exception list as well.

(From OE-Core rev: d1ad59bb7a5360433403733de71b66ff3e1484ef)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Luca Ceresoli <luca.ceresoli@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit e9158b191c1cfc16f97abed6c05891aa84fe9463)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-05-18 04:29:33 -10:00
Richard Purdie
10f4547dff recipes: Default to https git protocol where possible
The recommendation from server maintainers is that the https protocol
is both faster and more reliable than the dedicated git protocol at this point.
Switch to it where possible.

(From OE-Core rev: 8f3669f81db8a58f8ed2faef76acab3499f59619)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
(cherry picked from commit 139102a73d4151f4748b4a861bd4ab28dda7dab7)
Signed-off-by: Steve Sakoman <steve@sakoman.com>
2023-05-18 04:29:33 -10:00
Martin Jansa
f135a26aa3 selftest: efibootpartition.py: fix QEMU_USE_KVM usage
* if you don't have QEMU_USE_KVM in local.conf it fails with:

  2023-03-12 18:06:29,591 - oe-selftest - DEBUG - Checking if qemux86-64 is not this MACHINE
  2023-03-12 18:06:29,594 - oe-selftest - INFO -  ... ERROR
  2023-03-12 18:06:29,594 - oe-selftest - INFO - Traceback (most recent call last):
    File "/OE/build/poky/meta/lib/oeqa/core/decorator/__init__.py", line 35, in wrapped_f
      return func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
    File "/OE/build/poky/meta/lib/oeqa/selftest/cases/efibootpartition.py", line 18, in test_boot_efi
      if oe.types.qemu_use_kvm(self.td['QEMU_USE_KVM'], self.td["TARGET_ARCH"]):
                               ~~~~~~~^^^^^^^^^^^^^^^^
  KeyError: 'QEMU_USE_KVM'

[YOCTO #12937]

(From OE-Core rev: 7c32ca2f91beb98769c89470b37f06cb4b99aebb)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-06 14:31:42 +01:00
Mark Asselstine
53a1ad0c88 oeqa/selftest/bblogging: uncomment python stdout checks
Since bitbake commit 81a58647b2f4fc0a2589b2978fc9d81b2bfe6aec
[bitbake: build: Make python output print to stdout when running with
-v (verbose)] we no longer need to comment out the python stdout
checks.

(From OE-Core rev: 67886a8473c511c8ab3db2e4587cc5a070979d11)

Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-05 17:26:53 +01:00
Martin Jansa
36f736bd39 selftest: runqemu: use better error message when asserts fail
* It was showing whole log and that the runqemu command failed, but not
  where the log file is, nor why it thinks the runqemu failed

[YOCTO #12937]

(From OE-Core rev: ad4b72e6524114a3bdee10cab71f194ea143cd24)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-03 11:39:59 +01:00
Martin Jansa
34aa323f18 selftest: runqemu: better check for ROOTFS: in the log
* it was searching for line like this:
  ROOTFS: [/OE/build/poky/build/build-st-2023-03-20-esdk-runqemu-patch1/runqemu.RunqemuTests.test_boot_machine_ext4/build-st/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64-20230320081121.rootfs.ext4]
  but with IMAGE_NAME_SUFFIX changed to something else than default ".rootfs"
  or with my pending changes the line looks like this:
  ROOTFS: [/OE/build/poky/build/build-st-2023-03-20-esdk-runqemu-patch2/runqemu.RunqemuTests.test_boot_machine_ext4/build-st/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs-20230320085744.ext4]
  and test was failing.

* Check for whole line starting with ROOTFS: and ending just with .ext4

[YOCTO #12937]

(From OE-Core rev: 7a0f622f23aff2c4eeca0606e7682931eb53287a)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-03 11:39:59 +01:00
Martin Jansa
edd4fe8cb3 selftest: imagefeatures.py: don't mix tabs and spaces for indentation
* introduced in:
  https://git.openembedded.org/openembedded-core/commit/?id=96d4486df6d870ef19e2055b026729e66bc118f3

(From OE-Core rev: 143f624116802579e7a6804c8d4b562f931880c8)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-03 11:39:59 +01:00
Frederic Martinsons
ad460bb6aa meta-selftest: provide a recipe for zvariant
This recipe is for showing a "real world" example of
a crate that depends on some git repositories.

Usually, this kind of crate is built within a global
workspace (here it is the zbus project) and so
doesn't need a Cargo.lock on its own.

For the sake of the demonstration, I had to tweak things
a little to be able to compile zvariant in standalone
(no relative path in dependency, no symlink to LICENSE
provide a Cargo.lock)

The use case where the crate had some git repository
in dependency is very common for "private" crate that
are not aimed to be published on crates.io.
When the project grow bigger, it is common to have
a bin and multiple lib developped in parallel, and these
libs are surely on a git repostitory.

A test case have been also added to check for:
  - the previous patch about git subpath parameter and devtool
  - the correctness of overriding dependencies (first patch of the
series)

(From OE-Core rev: 409e045f96f69877de6f36ed14c5c19a9cb74eaf)

Signed-off-by: Frederic Martinsons <frederic.martinsons@gmail.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-04-01 11:36:26 +01:00
Enrico Jörns
d59756912c oeqa: whitespace and indentation cleanups
(From OE-Core rev: 4922221d1259e2f78233f17bb901cdac5b9aa520)

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-31 23:30:36 +01:00
Enrico Jörns
4d2c6baba5 oeqa/selftest/cases/runqemu: update imports
'tempfile' and 'oeqa.utils.command.runCmd' are unused while 'os' is
actually used.

(From OE-Core rev: 50f766233e83528eebeca92877277374c7193530)

Signed-off-by: Enrico Jorns <ejo@pengutronix.de>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-31 23:30:36 +01:00
Romuald Jeanne
6815f71692 oeqa/selftest/imagefeatures: set a test for mutliubi in test_image_fstypes
Allow 'multiubi' fstype image.
Set 'MULTIUBI_BUILD' with two entries to allow configuration for two
different '.ubifs'.
Configure 'MKUBIFS_ARGS_*' and 'UBINIZE_ARGS_*' for both configurations.

(From OE-Core rev: 96d4486df6d870ef19e2055b026729e66bc118f3)

Signed-off-by: Romuald Jeanne <romuald.jeanne@st.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-30 23:26:04 +01:00
Martin Jansa
953007b68b selftest: eSDK rename to esdk
* I was hit by oe-selftest -r eSDK.oeSDKExtSelfTest.test_install_libraries_headers
  running all tests except only this selected one:

  poky $ oe-selftest -v -r eSDK.oeSDKExtSelfTest.test_install_libraries_headers -K -B /OE/build/poky/build-eSDK
  2023-03-13 14:00:52,955 - oe-selftest - DEBUG - Selected tests with -r: ['eSDK.oeSDKExtSelfTest.test_install_libraries_headers']
  2023-03-13 14:00:55,531 - oe-selftest - INFO - Changing cwd to /OE/build/poky/build
  ..
  2023-03-13 14:00:58,128 - oe-selftest - INFO - test_archiver_allows_to_filter_on_recipe_name (archiver.Archiver.test_archiver_allows_to_filter_on_recipe_name)

  this is caused by _built_modules_dict(modules) function
  which filters out eSDK.oeSDKExtSelfTest.test_install_libraries_headers
  based on the regexp and then it runs all loaded tests, because
  modules are empty

  the initial regexp and comment from 2017:
     https://git.openembedded.org/openembedded-core/commit/?id=80db3d999ae26d298d9d5418a32b11a4f27af9d5

     # Assumption: package and module names do not contain upper case
     # characters, whereas class names do
     m = re.match(r'^([^A-Z]+)(?:\.([A-Z][^.]*)(?:\.([^.]+))?)?$', module)

  might still be valid, but it was loosened in 2018 to accept upper case in module:

     https://git.openembedded.org/openembedded-core/commit/?id=1ecf48fd286a77078451b67879a44f9c9dc7a894
     Some test cases (eSDK.oeSDK*, runtime_test/*) does not match with current regex, fix it accept all.

  Then skipping the not matching modules was added later in 2018:

     https://git.openembedded.org/openembedded-core/commit/?id=f2042bf3638ed4edfb167e7f7d4be6da60997ead

  and regexp was updated again in 2020 not to accept upper case in modules:

     https://git.openembedded.org/openembedded-core/commit/?id=ad81ea90a815389e45ff302a85151724c71f71c3
     oeqa/core/loader: refine regex to find module
     test case in format <module name>.<class name>.<test case name>
     this is clear when test cases is only 3 item deep.

     but confused when it is 4 item deep, eg,
     oelib.types.TestList.test_list_nosep

  I'm afraid that changing this regexp again to accept eSDK will break
  someone's favorite test case, renaming eSDK looks much safer option

  There is only 1 such case in poky:
  $ oe-selftest --list-modules | grep INFO.- | sed 's/^.*INFO - //g' | grep -v '^[a-z_\.]*$'
  Listing all available test modules:
  eSDK

  Most modules are just a-z (52x), then oelib.<foo> (6x) and 7 modules with underscore '_'.

(From OE-Core rev: 173ef4397b5be19f72385b1a0d892a1fa8979d53)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-30 23:26:04 +01:00
Fawzi KHABER
3ae3669468 oeqa/selftest/cases/package.py: adding unittest for package rename conflicts
This Unittest tries to rename a package, using an already used name and
fails on do_package.

Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr>
(From OE-Core rev: c3fe173d5196506d89aa464ba56aabcf581a60db)

Signed-off-by: Fawzi KHABER <fawzi.khaber@smile.fr>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-30 12:32:30 +01:00
Geoffrey GIRY
81740facf4 cve-check: Fix false negative version issue
NVD DB store version and update in the same value, separated by '_'.
The proposed patch check if the version from NVD DB contains a "_",
ie 9.2.0_p1 is convert to 9.2.0p1 before version comparison.

[YOCTO #14127]

Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr>
(From OE-Core rev: 7d00f6ec578084a0a0e5caf36241d53036d996c4)

Signed-off-by: Geoffrey GIRY <geoffrey.giry@smile.fr>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-30 12:30:38 +01:00
Jose Quaresma
2ba13d9447 oeqa/selftest/reproducible: Split different packages from missing packages output
When we provide sstate_targets recipes they are reported as missing
which is not quite correct because they are proveided to cut build/debugging time.

(From OE-Core rev: fd70a564d6946fa460638dd04ce2daecf4566cf3)

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-28 22:28:45 +01:00
Jose Quaresma
f5f465ff57 oeqa/selftest: OESelftestTestContext: convert relative to full path when newbuilddir is provided
Relative paths in BBLAYERS only works when the new build dir are on the same
ascending directory node:
 .
 ├── build
 ├── build-st

It works because they share the same ascending relative directory node.

So use the full path when the argument newbuilddir is provided
to make the oe-selftest work everywere regardless of the location chosen.

(From OE-Core rev: 2e022c1977bc1006c00a87e08a2dca5b69db4801)

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-28 22:28:45 +01:00
Jose Quaresma
4147b7e76d oeqs/selftest: OESelftestTestContext: replace the os.environ after subprocess.check_output
No function change but is needed to simplify the next patch.

(From OE-Core rev: 193d8f0d6383f54c61f90d58eeb8f41580c2360a)

Signed-off-by: Jose Quaresma <jose.quaresma@foundries.io>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-28 22:28:45 +01:00
Martin Jansa
248f6f01cf selftest: imagefeatures.py: respect IMAGE_LINK_NAME for debugfs and manifest as well
* these cases were correctly respecting IMAGE_LINK_NAME in most tests

* the only exception was relatively wide glob for manifest:
  "test-empty-image-*.manifest"
* and even wider glob for -dbg:
  "*-dbg.rootfs.tar.bz2"

* replace them with the exact filename we expect for given image

* be aware that gzip won't accept the symlink in IMAGE_LINK_NAME causing:
  2023-03-13 08:58:23,845 - oe-selftest - INFO -  ... FAIL
  2023-03-13 08:58:23,845 - oe-selftest - INFO - Traceback (most recent call last):
    File "/OE/build/poky/meta/lib/oeqa/selftest/cases/imagefeatures.py", line 124, in test_bmap
      self.assertTrue(runCmd('gzip -t %s' % gzip_path))
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/OE/build/poky/meta/lib/oeqa/utils/commands.py", line 214, in runCmd
      raise AssertionError("Command '%s' returned non-zero exit status %d:\n%s" % (command, result.status, exc_output))
  AssertionError: Command 'gzip -t /OE/build/poky/build/build-st-2023-03-12-todo-patch2/imagefeatures.ImageFeatures.test_bmap/build-st/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.ext4.bmap.gz' returned non-zero exit status 1:
  gzip: skipping: /OE/build/poky/build/build-st-2023-03-12-todo-patch2/imagefeatures.ImageFeatures.test_bmap/build-st/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs.ext4.bmap.gz is a symbolic link

* and "*-dbg.rootfs.tar.bz2" doesn't work if IMAGE_NAME_SUFFIX
  is changed to anything else than ".rootfs" or moved into
  IMAGE_LINK_NAME (like I plan in future changes where this will
  need to be updated again).

  Luckily we can use the symlink which currently doesn't have
  IMAGE_NAME_SUFFIX nor the DATETIME, so we don't need to search
  for it with glob, e.g. currently:

  core-image-minimal-qemux86-64-dbg-20230313112546-dbg.rootfs.tar.bz2
  core-image-minimal-qemux86-64-dbg.tar.bz2 -> core-image-minimal-qemux86-64-dbg-20230313112546-dbg.rootfs.tar.bz2

[YOCTO #12937]

(From OE-Core rev: 39285e981343930e41afe4eb8f2db675a85d54c2)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 13:53:29 +00:00
Martin Jansa
752d8ab3ed selftest: minidebuginfo.py respect IMAGE_LINK_NAME
* use IMAGE_LINK_NAME instead of hardcoding
  core-image-minimal-${MACHINE} assumption

[YOCTO #12937]

(From OE-Core rev: 9f8ffe22d9bb7cd93b9bc9ece917a48b27ab22d3)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 13:53:29 +00:00
Martin Jansa
2e7d36b542 selftest: gdbserver.py: respect IMAGE_LINK_NAME
* use IMAGE_LINK_NAME instead of hardcoding
  core-image-minimal-${MACHINE} assumption

[YOCTO #12937]

(From OE-Core rev: 3ee65ecdf841d74e2767d620e7f4a62e3a8863dc)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 13:53:29 +00:00
Martin Jansa
923a9e11b1 selftest: fitimage.py: respect INITRAMFS_IMAGE_NAME and KERNEL_FIT_LINK_NAME
* use these variables instead of assuming:
  "fitImage-its-%s-%s-%s" % (image_type, machine, machine)
  notice that these files have no extension and the -machine suffix
  is duplicated (once from core-image-minimal and once from
  kernel-fitimage.bbclass:kernel_do_deploy:append
  through KERNEL_FIT_LINK_NAME which is MACHINE by default:

  KERNEL_FIT_NAME ?= "${KERNEL_ARTIFACT_NAME}"
  KERNEL_FIT_LINK_NAME ?= "${KERNEL_ARTIFACT_LINK_NAME}"

  KERNEL_ARTIFACT_NAME ?= "${PKGE}-${PKGV}-${PKGR}-${MACHINE}${IMAGE_VERSION_SUFFIX}"
  KERNEL_ARTIFACT_LINK_NAME ?= "${MACHINE}"

[YOCTO #12937]

(From OE-Core rev: d6e52428ca25d9b6ec2a52406f2d1859d05867d3)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 13:53:29 +00:00
Martin Jansa
a2b8fec289 selftest: runqemu: respect IMAGE_LINK_NAME
* don't assume that every built image is named:

  <recipe>-<machine>.<suffix>

[YOCTO #12937]

(From OE-Core rev: d3c1c983d062f0731610f4a6984126ec6dab8caf)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 13:53:29 +00:00
Martin Jansa
e1334b120e selftest: wic: respect IMAGE_LINK_NAME also in test_rawcopy_plugin_qemu
* this one is more tricky, because the test_rawcopy_plugin.wks.in file
  is used while building core-image-minimal-mtdutils, but the image filename
  inside wks.in is from core-image-minimal, so we cannot just let bitbake
  expand IMAGE_LINK_NAME, use separate variable set in the same config fragment
  IMAGE_LINK_NAME_CORE_IMAGE_MINIMAL

[YOCTO #12937]

(From OE-Core rev: 8864ee825fdc52fe7a8cf93876c81c639fba66f4)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 13:53:29 +00:00
Martin Jansa
af9f67606c selftest: wic: respect IMAGE_LINK_NAME
* use IMAGE_LINK_NAME instead of hardcoding
  core-image-minimal-${MACHINE} assumption

[YOCTO #12937]

(From OE-Core rev: d6151aeb1e8f6162113b1123fa0f162536e48a44)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 13:53:29 +00:00
Martin Jansa
9dd62390e2 selftest: devtool: set BB_HASHSERVE_UPSTREAM when setting SSTATE_MIRROR
* with my build/conf/local.conf:
  SSTATE_DIR = "/OE/build/poky/build/sstate-cache"
  these devtool tests will first set own SSTATE_DIR and the original one set as SSTATE_MIRROR:

2023-03-11 11:51:46,837 - oe-selftest - INFO - test_devtool_update_recipe_append (devtool.DevtoolUpdateTests.test_devtool_update_recipe_append)
2023-03-11 11:51:46,846 - oe-selftest - DEBUG - Appending to: /OE/build/poky/build/build-st-2023-03-11-patch2/devtool.DevtoolUpdateTests.test_devtool_update_recipe_append/build-st/conf/selftest.inc
SSTATE_DIR = "/OE/build/poky/build/build-st-2023-03-11-patch2/devtool.DevtoolUpdateTests.test_devtool_update_recipe_append/build-st/sstate_devtool"
SSTATE_MIRRORS += "file://.* file:////OE/build/poky/build/sstate-cache/PATH"

* but that unfortunately leads to a warning from sanity.bbclass
  about SSTATE_MIRRORS without matching BB_HASHSERVE, because
  BB_HASHSERVE is set to "auto" by default

  these tests failing with:

2023-03-11 11:55:39,610 - oe-selftest - INFO - ======================================================================
2023-03-11 11:55:39,610 - oe-selftest - INFO - FAIL: test_devtool_update_recipe_append_git (devtool.DevtoolUpdateTests.test_devtool_update_recipe_append_git)
2023-03-11 11:55:39,610 - oe-selftest - INFO - ----------------------------------------------------------------------
2023-03-11 11:55:39,611 - oe-selftest - INFO - Traceback (most recent call last):
  File "/OE/build/poky/meta/lib/oeqa/selftest/cases/devtool.py", line 1118, in test_devtool_update_recipe_append_git
    self.assertNotIn('WARNING:', result.output)
AssertionError: 'WARNING:' unexpectedly found in 'NOTE: Starting bitbake server...\nWARNING: You are using a local hash equivalence server but have configured an sstate mirror. This will likely mean no sstate will match from the mirror. You may wish to disable the hash equivalence use (BB_HASHSERVE), or use a hash equivalence server alongside the sstate mirror.\nLoading cache...done.\nLoaded 0 entries from dependency cache.\nParsing recipes...done.\nParsing of 947 .bb files complete (0 cached, 947 parsed). 1764 targets, 52 skipped, 0 masked, 0 errors.\n\nSummary: There was 1 WARNING message.\nINFO: Updating SRCREV in recipe mtd-utils-selftest_git.bb\nNOTE: Writing append file /tmp/devtoolqa1m2lh02v/layer/recipes-devtools/mtd/mtd-utils-selftest_git.bbappend'
----------------------------------------------------------------------

* just setting BB_HASHSERVE to empty doesn't work, because then we
  would need to disable OEEquivHash as well as it fails with:

  ERROR: OEEquivHash requires BB_HASHSERVE to be set

(From OE-Core rev: 96d4392ee9c5c3674e5c4c4512f527a2ca6765e4)

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-22 13:53:29 +00:00
Richard Purdie
2ba8593b95 oeqa/selftest/sstate: Split classes to allow more parallelism
By splitting the single class into multiple classes, it allows more parallelism
in the execution. These tests don't have interdependencies that benefit
from sstate reuse or anything like that so this makes sense to improve overall
test execution time if/where possible.

(From OE-Core rev: 9dbc2f4ebbde47b1c997948a9690ffb8eb29b552)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-13 17:35:22 +00:00
Richard Purdie
2420625414 oeqa/selftest/sstate: Move common code to base class
Move the other common shares test functions to the base class to
improve the code structure.

(From OE-Core rev: de3e6f85c5537a3571ffbe2326b73f2c2526bce2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-13 17:35:22 +00:00
Richard Purdie
a4c484b8cb oeqa/selftest/sstate: Merge sstate test class with tests themselves
Having this base class as a separate file is just confusing. Merge with
the rest of the test code.

(From OE-Core rev: 977522a3b063225e22e2fd04b8265a4595606db2)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-13 17:35:22 +00:00
Richard Purdie
4eb546cf1c selftest/recipetool: Stop test corrupting tinfoil class
Running recipetool.RecipetoolTests.test_recipetool_handle_license_vars
followed by wic.Wic2.test_biosplusefi_plugin_qemu would show a failure of:

  File "/media/build/poky/meta/lib/oeqa/utils/commands.py", line 351, in runqemu
    qemu = oeqa.targetcontrol.QemuTarget(recipedata, targetlogger, image_fstype)
  File "/media/build/poky/meta/lib/oeqa/targetcontrol.py", line 116, in __init__
    use_kvm = oe.types.qemu_use_kvm(d.getVar('QEMU_USE_KVM'), d.getVar('TARGET_ARCH'))
  File "/media/build/poky/meta/lib/oe/types.py", line 182, in qemu_use_kvm
    if kvm and boolean(kvm):
  File "/media/build/poky/meta/lib/oe/types.py", line 128, in boolean
    raise ValueError("Invalid boolean value '%s'" % value)
ValueError: Invalid boolean value '/media/build/poky/meta/files/common-licenses'

which made no sense until you realise the recipetool test is corrupting
the tinfoil class. Work on a copy instead to avoid this.

(From OE-Core rev: 7fe76fe17f67c1bbd108d02836692fed20d24771)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-13 17:35:22 +00:00
Alexis Lothoré
c31c140746 oeqa/selftest/resulttool: add test for error propagation in test name filtering
(From OE-Core rev: 17d825c88499c6f7e3ffe22fae099871d0850abc)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-12 23:39:13 +00:00
Alexis Lothoré
754d270ab8 oeqa/selftest/resulttool: fix fake data used for testing
resulttool searches for "status" field, not "STATUS", in results.
This fix is more to avoid confusion than fixing anything, since the updated
tests are about regression.can_be_compared, which does not check for "status"

(From OE-Core rev: df342c81d02dea3f677966602089d4a972bb09bd)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-12 23:39:13 +00:00
Alexis Lothoré
e3b005d8eb oeqa/selftest/resulttool: fix ptest filtering tests
There are multiple issues with the tests for the ptest fixup mechanism
introduced in c45d58f003e8d8b323169ca9d479dc49c43a9974:
- the feature does not impact regression.can_be_compared but directly the
  comparison step in regression.regression_common
- the "status" field was wrong ("STATUS"), which prevents tests comparison
- there is a typo babeltrace2 ptest result name

(From OE-Core rev: 69205e2e2117fc469fcf3ae4b2174482db1d6297)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-12 23:39:12 +00:00
Alexander Kanavin
03e900a748 selftest/overlayfs: enable systemd via INIT_MANAGER
Latest systemd version needs the full set of settings enabled
by that (if also r/o rootfs is enabled), and refuses to boot
otherwise.

(From OE-Core rev: 16f7661000a50e58d512a5d47794c5f8be2c49e6)

Signed-off-by: Alexander Kanavin <alex@linutronix.de>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-11 00:08:40 +00:00
Michael Halstead
45d0ab66fd selftest/runtime_test/virgl: Disable for all Rocky Linux
RHEL compatible kernels do not support vgem so we disable virgl headless
testing.

(From OE-Core rev: 78bc1ba2361249ec845d1b9e28d429a37dd83910)

Signed-off-by: Michael Halstead <mhalstead@linuxfoundation.org>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-09 23:51:29 +00:00
Ross Burton
f3c5db2465 oeqa/selftest/imagefeatures: add test for man-db
Add a test to verify that manpages are built with api-documentation
enabled, and apropos works on the target (so mandb has been ran).

(From OE-Core rev: c085c335319bb98e09177c09feb1241a8560b606)

Signed-off-by: Ross Burton <ross.burton@arm.com>
Signed-off-by: Alexandre Belloni <alexandre.belloni@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-03-08 16:41:55 +00:00
Alexis Lothoré
9b8665bb96 oeqa/selftest/resulttool: add test for metadata filtering on regression
Multiple filters have been added to resulttool to ensure that test
results are compared only to relevant older tests results

Add some unit tests about added filters:
- tests should only be compared when machine matches
- tests run as oeselftest should only be compared when
  OESELFTEST_METADATA matches
- many ptests should match despite non-static names
- runtime tests should match with matching tests regarding ltp tests
  content

(From OE-Core rev: b84302ef56f2516742a496aef43b89d4c3decd37)

Signed-off-by: Alexis Lothoré <alexis.lothore@bootlin.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
2023-02-28 07:53:54 +00:00