From 152bbdb7cfefab89132549ea83fb0b859c290e8a Mon Sep 17 00:00:00 2001 From: Ahmad Fatoum Date: Fri, 4 Jun 2021 10:46:55 +0200 Subject: MIPS: qemu-malta: generate swapped image as part of multi-image build Having to manually swap the words in the MIPS Malta image for QEMU little endian emulation is annoying. Have the multi-image build for Malta generate a second .swapped image that can be readily used if needed. Signed-off-by: Ahmad Fatoum Link: https://lore.barebox.org/20210604084704.17410-5-a.fatoum@pengutronix.de Signed-off-by: Sascha Hauer --- Documentation/boards/mips/qemu-malta.rst | 16 ++++------------ 1 file changed, 4 insertions(+), 12 deletions(-) (limited to 'Documentation/boards') diff --git a/Documentation/boards/mips/qemu-malta.rst b/Documentation/boards/mips/qemu-malta.rst index e188ae8c64..fd37d5edb2 100644 --- a/Documentation/boards/mips/qemu-malta.rst +++ b/Documentation/boards/mips/qemu-malta.rst @@ -10,31 +10,23 @@ QEMU run string: qemu-system-mips -nodefaults -M malta -m 256 \ -device VGA -serial stdio -monitor null \ - -bios barebox-flash-image + -bios ./images/barebox-qemu-malta.img Little-endian mode ------------------ -Running little-endian Malta is a bit tricky. In little-endian mode the 32bit words in the boot flash image are swapped, a neat trick which allows bi-endian firmware. -You have to swap words of ``zbarebox.bin`` image, e.g.: - -.. code-block:: sh - - echo arch/mips/pbl/zbarebox.bin \ - | cpio --create \ - | cpio --extract --swap --unconditional - -QEMU run string: +The barebox build generates a second ``./images/barebox-qemu-malta.img.swapped`` +image that can be used in this case, e.g.: .. code-block:: sh qemu-system-mipsel -nodefaults -M malta -m 256 \ -device VGA -serial stdio -monitor null \ - -bios barebox-flash-image + -bios ./images/barebox-qemu-malta.img.swapped Using GXemul -- cgit v1.2.3 From ff10a9966637f65efe2c82f65ad2b76e34f3774c Mon Sep 17 00:00:00 2001 From: Ahmad Fatoum Date: Fri, 4 Jun 2021 10:46:57 +0200 Subject: Documentation: boards: RISC-V: update TinyEMU support Adding the tinyemu files as separate files and including them has the nice benefit that temu users can just use these files directly instead of copy pasting. While at it, update the section as input, as support was added meanwhile. Signed-off-by: Ahmad Fatoum Link: https://lore.barebox.org/20210604084704.17410-7-a.fatoum@pengutronix.de Signed-off-by: Sascha Hauer --- Documentation/boards/riscv.rst | 20 ++++++++++---------- Documentation/boards/riscv/barebox-virt32.cfg | 7 +++++++ Documentation/boards/riscv/barebox-virt64.cfg | 7 +++++++ 3 files changed, 24 insertions(+), 10 deletions(-) create mode 100644 Documentation/boards/riscv/barebox-virt32.cfg create mode 100644 Documentation/boards/riscv/barebox-virt64.cfg (limited to 'Documentation/boards') diff --git a/Documentation/boards/riscv.rst b/Documentation/boards/riscv.rst index 59cdc00a99..955a6fbb6b 100644 --- a/Documentation/boards/riscv.rst +++ b/Documentation/boards/riscv.rst @@ -41,25 +41,25 @@ TinyEMU ------- TinyEMU can emulate a qemu-virt like machine with a RISC-V 32-, 64- -and 128-bit CPU. It can run barebox with this sample configuration:: +and 128-bit CPU. It can run 32-bit barebox with this sample configuration: - /* temu barebox-virt64.cfg */ - { - version: 1, - machine: "riscv64", - memory_size: 256, - bios: "bbl64.bin", - kernel: "./images/barebox-dt-2nd.img", - } +.. literalinclude:: riscv/barebox-virt32.cfg + +as well as 64-bit barebox with this configuration: + +.. literalinclude:: riscv/barebox-virt64.cfg ``barebox-dt-2nd.img`` can be generated like with Qemu. Graphical -output is also supported, but virtio input support is still missing. +output and input are also supported. To activate add:: display0: { device: "simplefb", width: 800, height: 600 }, + input_device: "virtio", into the config file. +See https://barebox.org/jsbarebox/?graphic=1 for a live example. + Erizo ----- diff --git a/Documentation/boards/riscv/barebox-virt32.cfg b/Documentation/boards/riscv/barebox-virt32.cfg new file mode 100644 index 0000000000..5f0eb34eee --- /dev/null +++ b/Documentation/boards/riscv/barebox-virt32.cfg @@ -0,0 +1,7 @@ +{ + version: 1, + machine: "riscv32", + memory_size: 256, + bios: "bbl32.bin", + kernel: "images/barebox-dt-2nd.img", +} diff --git a/Documentation/boards/riscv/barebox-virt64.cfg b/Documentation/boards/riscv/barebox-virt64.cfg new file mode 100644 index 0000000000..45e1cd8308 --- /dev/null +++ b/Documentation/boards/riscv/barebox-virt64.cfg @@ -0,0 +1,7 @@ +{ + version: 1, + machine: "riscv64", + memory_size: 256, + bios: "bbl64.bin", + kernel: "images/barebox-dt-2nd.img", +} -- cgit v1.2.3 From 7271ce78d0fccd530a9b67d52f2248a9e6fd9f2f Mon Sep 17 00:00:00 2001 From: Ahmad Fatoum Date: Fri, 4 Jun 2021 10:47:02 +0200 Subject: test: add emulate.pl, a runner for barebox on emulated targets This script is a wrapper around tuxmake, qemu and labgrid-pytest. It parses the same YAML files, labgrid-pytest uses and instructs tuxmake to build and collect the needed images. By default, it will start an interactive emulator session, but with --test, it can also run labgrid-pytest instead. The script has some knowledge of QEMU options to make common tasks like passing images straight-forward. Script is written with both manual use and CI in mind. Signed-off-by: Ahmad Fatoum Link: https://lore.barebox.org/20210604084704.17410-12-a.fatoum@pengutronix.de Signed-off-by: Sascha Hauer --- Documentation/boards/emulated.rst | 75 ++++++ test/emulate.pl | 528 ++++++++++++++++++++++++++++++++++++++ test/kconfig/base.cfg | 0 test/kconfig/full.cfg | 0 4 files changed, 603 insertions(+) create mode 100644 Documentation/boards/emulated.rst create mode 100755 test/emulate.pl create mode 100644 test/kconfig/base.cfg create mode 100644 test/kconfig/full.cfg (limited to 'Documentation/boards') diff --git a/Documentation/boards/emulated.rst b/Documentation/boards/emulated.rst new file mode 100644 index 0000000000..584883d6ef --- /dev/null +++ b/Documentation/boards/emulated.rst @@ -0,0 +1,75 @@ +Emulated targets +================ + +Some targets barebox runs on are virtualized by emulators like QEMU, which +allows basic testing of barebox functionality without the actual hardware. + +Generic DT image +---------------- + +Supported for ARM and RISC-V. It generates a barebox image that can +be booted with the Linux kernel booting convention, which makes +it suitable to be passed as argument to the QEMU ``-kernel`` option +(as well as booted just like Linux from barebox or other bootloaders). + +The device tree can be passed externally via QEMU's ``-dtb`` option, but +also the QEMU internal device tree can be used. + +The latter can be very useful with :ref:`virtio_sect`, because QEMU will +fix up the virtio mmio regions into the device tree and barebox will +discover the devices automatically, analogously to what it does with +VirtIO over PCI. + +test/emulate.pl +--------------- + +The ``emulate.pl`` script shipped with barebox can be used to easily +start VMs. It reads a number of YAML files in ``test/$ARCH``, which +describe some virtualized targets that barebox is known to run on. + +Controlled by command line options, these targets are built with +tuxmake if available and loaded into the emulator for either interactive +use or for automated testing with Labgrid ``QEMUDriver``. + +.. _tuxmake: https://pypi.org/project/tuxmake/ +.. _Labgrid: https://labgrid.org + +Install dependencies for interactive use:: + + cpan YAML::XS # or use e.g. libyaml-libyaml-perl on Debian + pip3 install tuxmake # optional + +Example usage:: + + # Switch to barebox source directory + cd barebox + + # emulate x86 VM runnig the EFI payload from efi_defconfig + ARCH=x86 ./test/emulate.pl efi_defconfig + + # build all MIPS targets known to emulate.pl and exit + ARCH=mips ./test/emulate.pl --no-emulate + +The script can also be used with a precompiled barebox tree:: + + # Switch to build directory + export KBUILD_OUTPUT=build + + # run a barebox image built outside tuxmake on an ARM virt machine + ARCH=arm ./test/emulate.pl virt@vexpress_defconfig --no-tuxmake + + # run tests instead of starting emulator interactively + ARCH=arm ./test/emulate.pl virt@vexpress_defconfig --no-tuxmake --test + +``emulate.pl`` also has some knowledge on paravirtualized devices:: + + # Run target and pass a block device (here /dev/virtioblk0) + ARCH=riscv ./test/emulate.pl --blk=rootfs.ext4 virt64_defconfig + +Needed command line options can be passed directly to the +emulator/``pytest`` as well by placing them behind ``--``:: + + # appends -device ? to the command line. Add -n to see the final result + ARCH=riscv ./test/emulate.pl virt64_defconfig -- -device ? + +For a complete listing of options run ``./test/emulate.pl -h``. diff --git a/test/emulate.pl b/test/emulate.pl new file mode 100755 index 0000000000..78c2815884 --- /dev/null +++ b/test/emulate.pl @@ -0,0 +1,528 @@ +#!/usr/bin/env perl +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2021 Ahmad Fatoum + +use strict; +use warnings; + +use Cwd; +use File::Basename; +use File::Spec; +use File::Temp 'tempdir'; +use Getopt::Long; +use List::Util 'first'; +use Pod::Usage; +use YAML::XS 'LoadFile'; + +my @QEMU_INTERACTIVE_OPTS = qw(-serial mon:stdio -trace file=/dev/null); + +my %targets; + +for my $arch (glob dirname(__FILE__) . "/*/") { + for my $cfg (glob "$arch/*.yaml") { + my $linkdest = readlink $cfg // ''; + + my $yaml = LoadFile($cfg); + + defined $yaml && exists $yaml->{targets} && exists $yaml->{targets}{main} + or die "Invalid structure for $cfg\n"; + + my $path = File::Spec->abs2rel($cfg, getcwd); + $cfg = basename($cfg); + $cfg =~ s/\.yaml$//; + $linkdest =~ s{^.*?([^/]+)\.yaml$}{$1}; + + $targets{basename $arch}{$cfg} = $yaml->{targets}{main}; + $targets{basename $arch}{$cfg}{path} = $path; + $targets{basename $arch}{$cfg}{tools} = $yaml->{tools}; + $targets{basename $arch}{$cfg}{images} = $yaml->{images}; + $targets{basename $arch}{$cfg}{alias} = $linkdest if $linkdest && $linkdest ne $cfg; + } +} + +my %arch_aliases = (arm64 => 'arm', x86_64 => 'x86', i386 => 'x86', um => 'sandbox'); + +my ($dryrun, $help, @blks, $rng, $list, $shell, $runtime, @kconfig_add, $artifacts); +my ($tuxmake, $emulate, $clean, $extraconsoles, $test, $verbosity) = (1, 1, 1, 0, 0, 1); +my ($kconfig_base, $kconfig_full) = (1, 0); + +my @OPTS; + +if (defined (my $idx = first { $ARGV[$_] eq '--' } 0 .. @ARGV - 1)) { + @OPTS = splice @ARGV, 1 + $idx; +} + +my @emulate_only = qw/blk console rng/; + +GetOptions( + 'help|?|h' => \$help, + 'dryrun|n' => \$dryrun, + 'list|l' => \$list, + 'tuxmake!' => \$tuxmake, + 'verbosity=i' => \$verbosity, + 'artifacts=s' => \$artifacts, + 'runtime=s' => \$runtime, + 'blk=s@' => \@blks, + 'console+' => \$extraconsoles, + 'rng' => \$rng, + 'emulate!' => \$emulate, + 'clean!' => \$clean, + 'shell' => \$shell, + 'kconfig-base!' => \$kconfig_base, + 'kconfig-full!' => \$kconfig_full, + 'kconfig-add|K=s@' => \@kconfig_add, + 'test' => \$test, +) or pod2usage(2); +pod2usage(1) if $help; + +if ($test && (@blks || $extraconsoles || $rng)) { + die "Virtual devices on command-line not supported with --test\n"; +} + +my @ARCH = split /\s/, $ENV{ARCH} // ''; +@ARCH = @ARCH ? @ARCH : keys %targets; + +my $success = 0; + +for my $arch (@ARCH) { + my @targets = @ARGV ? @ARGV : keys %{$targets{$arch}}; + @targets != 1 && !$tuxmake + and die "can't use --no-tuxmake with more than one config\n"; + + unless (defined $targets{$arch}) { + die "Unknown ARCH=$arch. Supported values:\n", + join(', ', keys %targets), "\n"; + } + + for my $config (@targets) { + $arch = $arch_aliases{$arch} // $arch; + + $config = fileparse($config, ".yaml"); + + unless (defined $targets{$arch}{$config}) { + next; + } + + if ($list) { + print "ARCH=$arch $config\n"; + $success += 1; + next; + } + + if (defined $targets{$arch}{$config}{alias}) { + next if grep { /^$targets{$arch}{$config}{alias}$/ } @targets; + $config = $targets{$arch}{$config}{alias}; + redo; + } + + print "# $config\n" if $dryrun; + $success += process($arch, $config); + } +} + +$success > 0 + or die "No suitable config found. $0 -l to list all built-in.\n"; + +sub process { + my ($ARCH, $defconfig, %keys) = @_; + my $target = $targets{$ARCH}{$defconfig}; + + if (!exists ($target->{runner}{tuxmake_arch})) { + $target->{runner}{tuxmake_arch} = $ARCH; + } + + my $dir; + + $dir = tempdir("bareboxbuild-XXXXXXXXXX", TMPDIR => 1, CLEANUP => $clean); + report('mkdir', $dir); + + my %opts = ( + target => $target, builddir => $tuxmake ? $dir : getcwd, + defconfig => $defconfig, + extra_opts => [map { s/\{config\}/$defconfig/gr } @OPTS], + ); + + build(%opts) if $tuxmake; + + while (my ($k, $v) = each %{$target->{runner}{download}}) { + if ($v =~ m{^[/.]}) { + symlink_force($v, "$dir/$k"); + } else { + vsystem('curl', '-L', '--create-dirs', '-o', "$dir/$k", $v) == 0 + or die "Failed to download resource `$v': $?\n"; + } + + symlink_force("$dir/$k", "$k") unless $tuxmake; + } + + if ($shell) { + pushd($dir); + system($ENV{SHELL} // "/bin/sh"); + popd(); + } + + return 1 unless $emulate; + + if ($tuxmake) { + $ENV{KBUILD_OUTPUT} = $dir; + print "export KBUILD_OUTPUT=$ENV{KBUILD_OUTPUT}\n" if $dryrun; + } + + my $success = $test ? test(%opts) : emulate(%opts); + + report("rm", "-rd", $dir) if $clean && $tuxmake; + + print "\n\n" if $dryrun; + return $success; +} + +sub build { + my %args = @_; + my ($runner, $dir) = ($args{target}{runner}, $args{builddir}); + + $args{defconfig} =~ s/[^@]+@//g; + + my @TUXMAKE_ARGS = ('-a', $runner->{tuxmake_arch}, '-k', $args{defconfig}); + + if (defined $runner->{kconfig_add}) { + for my $cfg (@{$runner->{kconfig_add}}) { + push @TUXMAKE_ARGS, "--kconfig-add=$cfg" + } + } + + push @TUXMAKE_ARGS, "--kconfig-add=test/kconfig/base.cfg" if $kconfig_base || $kconfig_full; + push @TUXMAKE_ARGS, "--kconfig-add=test/kconfig/full.cfg" if $kconfig_full; + + for (@kconfig_add) { + push @TUXMAKE_ARGS, "--kconfig-add=$_"; + } + + push @TUXMAKE_ARGS, "--runtime=$runtime" if $runtime; + + push @TUXMAKE_ARGS, '-q' if $verbosity == 0; + push @TUXMAKE_ARGS, '-v' if $verbosity == 2; + + vsystem('tuxmake', @TUXMAKE_ARGS, '-b', $dir, '-o', + $artifacts // "$dir/artifacts", 'default') == 0 + or die "Error building: $?\n"; +} + +sub emulate { + my %args = @_; + my %target = %{$args{target}}; + my @OPTS = @{$args{extra_opts}}; + + if (defined $target{drivers}{QEMUDriver}) { + my %qemu = %{$target{drivers}{QEMUDriver}}; + my ($kernel, $bios, $dtb); + my $qemu_virtio; + my $i; + + $kernel = abs_configpath($qemu{kernel}, \%args); + $bios = abs_configpath($qemu{bios}, \%args); + $dtb = abs_configpath($qemu{dtb}, \%args); + + my @cmdline = ($target{tools}{$qemu{qemu_bin}}, + '-M', $qemu{machine}, '-cpu', $qemu{cpu}, '-m', $qemu{memory}); + + push @cmdline, '-kernel', $kernel if defined $kernel; + push @cmdline, '-bios', $bios if defined $bios; + push @cmdline, '-dtb', $dtb if defined $dtb; + + push @cmdline, @QEMU_INTERACTIVE_OPTS; + for (split /\s/, $qemu{extra_args}) { + push @cmdline, $_; + } + + if (has_feature(\%target, 'virtio-pci')) { + $qemu_virtio = 'pci,disable-legacy=on,disable-modern=off'; + push @{$target{features}}, 'pci'; + push @{$target{features}}, 'virtio'; + } elsif (has_feature(\%target, 'virtio-mmio')) { + $qemu_virtio = 'device'; + push @{$target{features}}, 'virtio'; + } + + $i = 0; + for my $blk (@blks) { + if (has_feature(\%target, 'virtio')) { + $blk = rel2abs($blk); + push @cmdline, + '-drive', "if=none,file=$blk,format=raw,id=hd$i", + '-device', "virtio-blk-$qemu_virtio,drive=hd$i"; + } else { + die "--blk unsupported for target\n"; + } + } + + # note that barebox doesn't yet support multiple virtio consoles + if ($extraconsoles) { + $i = 0; + + if (defined $qemu_virtio) { + push @cmdline, + '-device', "virtio-serial-$qemu_virtio", + '-chardev', "pty,id=virtcon$i", + '-device', "virtconsole,chardev=virtcon$i,name=console.virtcon$i"; + + $i++; + } + + if ($i < $extraconsoles) { + # ns16550 serial driver only works with x86 so far + if (has_feature(\%target, 'pci')) { + for (; $i < $extraconsoles; $i++) { + push @cmdline, + '-chardev', "pty,id=pcicon$i", + '-device', "pci-serial,chardev=pcicon$i"; + } + } else { + warn "barebox currently supports only a single extra virtio console\n"; + } + } + } + + if (defined $rng) { + if (has_feature(\%target, 'virtio')) { + push @cmdline, '-device', "virtio-rng-$qemu_virtio"; + } else { + die "--rng unsupported for target\n"; + } + } + + pushd($args{builddir}) if $tuxmake; + + vsystem(@cmdline, @OPTS) == 0 or die "Error running emulator: $?\n"; + + } elsif (defined $target{drivers}{TinyEMUDriver}) { + my %temu = %{$target{drivers}{TinyEMUDriver}}; + my $TEMU_CFG; + my $i = 0; + + if (exists $temu{config}) { + $temu{config} = rel2abs($temu{config}); + open(my $fh, '<', "$temu{config}") + or die "Could not open file '$temu{config}': $!"; + $TEMU_CFG = do { local $/; <$fh> }; + } + + print ("$temu{config}\n"); + + defined $TEMU_CFG or die "Unknown tinyemu-config\n"; + + open(my $fh, '>', "$args{builddir}/tinyemu.cfg") + or die "Could not create file 'tinyemu.cfg': $!"; + print $fh $TEMU_CFG; + print "cat >'tinyemu.cfg' <{features}} +} + +sub report { + print join(' ', map { /\s/ ? "'$_'" : $_ } @_), "\n" if $dryrun; + 0; +} + +sub vsystem { + if ($dryrun) { + return report(@_); + } else { + my $ret = system @_; + warn "vsystem: $!\n" if $ret == -1; + return $ret >> 8; + } +} + +sub rel2abs { + File::Spec->rel2abs(@_) +} + +sub abs_configpath { + my ($path, $args) = @_; + my $LG_BUILDDIR; + + return unless defined $path; + $path = $args->{target}{images}{$path}; + return unless defined $path; + + if (exists $ENV{KBUILD_OUTPUT}) { + $LG_BUILDDIR = $ENV{KBUILD_OUTPUT}; + } elsif (-d 'build') { + $LG_BUILDDIR = 'build'; + } else { + $LG_BUILDDIR = getcwd(); + } + + $path =~ s/\$LG_BUILDDIR\b/$LG_BUILDDIR/g; + + return rel2abs($path, $args->{builddir}) +} + +sub symlink_force { + unlink($_[1]); + symlink($_[0], $_[1]); +} + +my @oldcwd; + +sub pushd { + my ($old, $new) = (getcwd, shift); + report ("pushd", $new); + push @oldcwd, $old; + chdir $new; + return $old; +}; + +sub popd { + report("popd"); + chdir pop(@oldcwd) +}; + +__END__ + +=head1 NAME + +emulate.pl - Build and run barebox under an emulator + +=head1 SYNOPSIS + +[ARCH=arch] emulate.pl [options] [defconfigs...] [--] [qemu/pytest-opts] + +=head1 OPTIONS + +Must be run from barebox source directory. If building out-of-tree, +either set C or ensure the out-of-tree directory +can be reached from a C symlink in the current working +directory. + +=over 8 + +=item B<-?>, B<-h>, B<--help> + +Print this help message and exit + +=item B<-n>, B<--dryrun> + +Print commands and exit + +=item B<-l>, B<--list> + +Filter input with list of known targets + +=item B<--verbosity>=%u + +Specify output verbosity level for both tuxmake and pytest. Default value is 1. + +=item B<--no-tuxmake> + +Don't rerun tuxkmake. Assumes current working directory is finished build directory + +=item B<--artifacts>=%s + +Destination directory for the tuxmake artifacts. By default, this is +the artifacts/ subdirectory in the temporary build directory and is +not persisted (unless --no-clean is specified). + +=item B<--blk>=%s + +Pass block device to emulated barebox. Can be specified more than once + +=item B<--console> + +Pass one Virt I/O console to emulated barebox. + +=item B<--rng> + +instantiate Virt I/O random number generator + +=item B<--no-emulate> + +Don't emulate anything and exit directly after build + +=item B<--no-clean> + +Don't delete temporary working directory after + +=item B<--shell> + +Open shell in temporary working directory, after build, but before emulation + +=item B<--test> + +Instead of starting emulator interactively, run it under labgrid-pytest with +the in-tree python tests. + +=item B<--runtime>=%s + +Runtime to use for the tuxmake builds. By default, builds are +run natively on the build host. +Supported: null, podman-local, podman, docker, docker-local. + +=item B<--no-kconfig-base> + +Don't apply test/kconfig/base.cfg. This may lead to more tests being +skipped. + +=item B<--kconfig-full> + +Applies test/kconfig/full.cfg on top of base.cfg. This enables as much as +possible to avoid skipping tests for disabled functionality. + +=item B<--kconfig_add>=%s, B<-K>=%s + +Extra kconfig fragments, merged on top of the defconfig and Kconfig +fragments described by the YAML. In tree configuration fragment +(e.g. `test/kconfig/virtio-pci.config`), path to local file, URL, +`CONFIG_*=[y|m|n]`, or `# CONFIG_* is not set` are supported. +Can be specified multiple times, and will be merged in the order given. + +=back + +=cut diff --git a/test/kconfig/base.cfg b/test/kconfig/base.cfg new file mode 100644 index 0000000000..e69de29bb2 diff --git a/test/kconfig/full.cfg b/test/kconfig/full.cfg new file mode 100644 index 0000000000..e69de29bb2 -- cgit v1.2.3