UTAH is very simple in terms of testing reliably and being able to reproduce a test run in a consistent manner. A command line specifies the test environment, type of machine to test on and a runlist. Then wait for it to finish and analyze the results.
UTAH can drive testing on whichever machine it is running or on a different one, by provisioning it with a particular provisioning method.
WARNING: Your machine will be modified, not to interfere with your current set-up or virtualized environment, so UTAH will create a new user called utah with sudo privileges to run the tests and have an inventory of existing VMs that you could be interested in for running tests.
There are some dependencies that need to be met in order to test with VMs, in a nutshell:
sudo apt-add-repository -y ppa:utah/stable
sudo apt-get update
sudo apt-get install utah
This will install the provisioning code as well as the test runner. To install just the test runner, install the utah-client package.
usage: run_utah_phablet.py [-h] [-s SERIAL] [--skip-install] [-r REVISION]
[--current] [--ubuntu-bootstrap] [--developer-mode]
[--channel CHANNEL] [--preserve] [--skip-network]
[-n NETWORK_FILE] [--skip-utah] [-b BRANCH]
[-e ENV] [--ppa PPA] [--results-dir RESULTS_DIR]
[--pull PULL] [-l RUNLIST] [-d DISCONNECTED]
[--device DEVICE] [--from-host] [--whoopsie]
Provisions a device for UTAH testing.
optional arguments:
-h, --help show this help message and exit
-s SERIAL, --serial SERIAL
Android serial ID of device
--skip-install Skip running the phablet-flash install.
-r REVISION, --revision REVISION
series/revision to install
--current Use the "current" image rather than "pending".
--ubuntu-bootstrap Use the system-images.ubuntu.com images.
--developer-mode Allow write access system-image partitions.
--channel CHANNEL Use an alternative channel for system-image.
--preserve Preserve the systems user data when flashing.
--skip-network Skip setting up wifi settings on device.
-n NETWORK_FILE, --network-file NETWORK_FILE
Specify network manager config file for wifi. If none
is specified we'll try and use the default wifi
configuration found on host system.
--skip-utah Skip install the utah-client on device.
-b BRANCH, --branch BRANCH
Install UTAH client on device from branch.
-e ENV, --env ENV Environment variables to set for utah-client.
--ppa PPA Specify an alternative PPA for utah.
default=ppa:utah/stable
--results-dir RESULTS_DIR
Directory to store results in on the host.
default=/tmp
--pull PULL A file or directory to copy into the hosts's results-
dir after UTAH has been run. This option can be
specified multiple times.
-l RUNLIST, --runlist RUNLIST
The utah runlist to execute
-d DISCONNECTED, --disconnected DISCONNECTED
Uses the supplied script to disconnect USB while UTAH
is executed.
--device DEVICE The Android device type.
--from-host Executes from the host (the runlist must talk to the
target using ADB.
--whoopsie Run whoopsie-upload-all after the test to to ensure
complete .crash files are uploaded.
usage: run_utah_tests.py [-h] [-m MACHINETYPE] [-v VARIANT]
[--skip-provisioning] [-s SERIES] [-t TYPE] [-a ARCH]
[-n] [-d] [-j] [-f FILES] [-o OUTDIR] [--dumplogs]
[--outputpreseed] [-i IMAGE] [-p PRESEED] [-b BOOT]
[--rewrite {all,minimal,casperonly,none}] [-k KERNEL]
[-r INITRD] [--name NAME] [-e EMULATOR] [-x XML]
[-g GIGABYTES] [--diskbus DISKBUS] [-l LOGPATH]
runlist
Provision a machine and run a runlist there.
positional arguments:
runlist URLs of runlist files to run
optional arguments:
-h, --help show this help message and exit
-m MACHINETYPE, --machinetype MACHINETYPE
Type of machine to provision (physical, virtual)
(Default is virtual)
-v VARIANT, --variant VARIANT
Variant of architecture, i.e., armel, armhf
--skip-provisioning Reuse a system that is already provisioned (name
argument must be passed)
-s SERIES, --series SERIES
Series to use for installation (lucid, precise,
quantal, raring, saucy) (Default is raring)
-t TYPE, --type TYPE Install type to use for installation (desktop, server,
mini, alternate) (Default is mini)
-a ARCH, --arch ARCH Architecture to use for installation (i386, amd64,
arm) (Default is amd64)
-n, --no-destroy Preserve VM after tests have run
-d, --debug Enable debug logging
-j, --json Enable json logging (default is YAML)
-f FILES, --files FILES
File or directory to copy from test system
-o OUTDIR, --outdir OUTDIR
Directory to store locally copied files (Default is
/var/log/utah/machine-name)
--dumplogs Write client output logs to standard out
--outputpreseed Copy preseed to logs directory and list as log file in
output
-i IMAGE, --image IMAGE
Image/ISO file to use for installation
-p PRESEED, --preseed PRESEED
Preseed file to use for installation
-b BOOT, --boot BOOT Boot arguments for initial installation
--rewrite {all,minimal,casperonly,none}
Set level of automatic configuration rewriting
(Default is all)
-k KERNEL, --kernel KERNEL
Kernel file to use for installation
-r INITRD, --initrd INITRD
InitRD file to use for installation
--name NAME Name of machine to provision
-e EMULATOR, --emulator EMULATOR
Emulator to use (kvm and qemu are supported, kvm will
be favored if available)
-x XML, --xml XML XML VM definition file (Default is /etc/utah/default-
vm.xml)
-g GIGABYTES, --gigabytes GIGABYTES
Size in gigabytes of virtual disk, specify more than
once for multiple disks (Default is [8])
--diskbus DISKBUS Disk bus to use for customvm installation (virtio,
sata, ide) (Default is virtio)
-l LOGPATH, --logpath LOGPATH
Directory used to write log files to
For example:
Provision a VM using a precise server image with i386 architecture and run the given runlist
run_utah_tests.py -s precise -t server -a i386 \
/usr/share/utah/client/examples/master.run