iOS

Getting started

In this guide will demonstrate how to use Emcee. We will use two MacOS machines to run unit and UI tests from a sample project. You can also use a single machine to try out Emcee to see if it works for your project. In this case, a single machine will act as a queue and a worker simultaneously. Alternatively, you can scale this guide to as many machines as you have.

If you encounter any issues while proceeding through the guide, please open an issue or reach out via https://t.me/emcee_ios.

Table of contents

1. Setting up machines
2. Building the sample project
3. Running tests using Emcee
   1. Tests without a host application
   2. Tests with a host application
   3. XCUI tests
4. Advanced Emcee configuration

Setting up machines

You will need to grant SSH access to your machines.

Expand to see how to set up your machines.

We will be using two machines: ios-build-machine77 and ios-build-machine78.

• ios-build-machine77 will be a worker and a queue - it will provide workers with tests to execute and execute some of those tests.
• ios-build-machine78 will be a worker - it will only execute tests.

Both machines are set up with a standard non-administrator user emcee and a qwerty password.

Install Xcode and sudo xcode-select --switch /Applications/Xcode.app on all of your machines.

We will use Xcode 14.3 (14E222b) and the iOS 16.4 simulator runtime bundled with this Xcode. If you want to use a specific version of simulator runtime, proceed to Xcode -> Preferences... -> Components -> Simulators and install the runtime on all the worker machines, where you want the tests to execute with the specific runtime version.

Emcee uses ssh to deploy itself to the machines specified as queue and workers. Enable SSH in your System Preferences -> Sharing -> Remote Login. To open this pane execute:

$ open "x-apple.systempreferences:com.apple.preferences.sharing?Services_RemoteLogin"

Now make sure that machines are accessible by ssh. For example:

ssh emcee@ios-build-machine77

If your machines are not accessible by DNS, use their IP addresses instead. You can check IP address in System Preferences -> Sharing. Please note IP addresses may change over time. To open this pane execute:

$ open "x-apple.systempreferences:com.apple.preferences.sharing"

Building the sample project

In this step, we will build a sample project that features different types of tests. Xcode and xcodebuild will produce build artifacts in derived data.

Expand to see how to build the sample project for testing purposes.

You can run this step from either machine. Clone the sample project:

cd ~
git clone https://github.com/avito-tech/Emcee.git
cd Emcee/Samples/EmceeSample

To build the project, create a simulator:

xcrun simctl create '16.4' 'iPhone 14' 'iOS16.4'

Now run xcodebuild:

xcodebuild build-for-testing \
	-project EmceeSample.xcodeproj \
	-destination "platform=iOS Simulator,name=16.4,OS=16.4" \
	-scheme AllTests \
	-derivedDataPath derivedData

Xcodebuild will place the build products in:

derivedData/Build/Products/Debug-iphonesimulator

Running tests using Emcee

Now that the machines are ready, and the project is built, download Emcee on the same machine where you built the project by running:

curl -L https://github.com/avito-tech/Emcee/releases/download/20.2.0/emceeFree.artifact.zip -o emceeFree.artifact.zip && unzip emceeFree.artifact.zip emceeFree && xattr -c emceeFree && xattr -c EmceeTestsInspector

With Emcee installed it is finally time to run the tests. The sample project includes 3 test types:

• Tests that don't require a host application
• Tests that require a host application
• XCUI tests

Tests without a host application

Let's first run tests that don't require a host application. We will be using the runTests command:

./emceeFree runTests \
	--queue "ssh://emcee:qwerty@ios-build-machine77" \
	--worker "ssh://emcee:qwerty@ios-build-machine77" \
	--worker "ssh://emcee:qwerty@ios-build-machine78" \
	--device "com.apple.CoreSimulator.SimDeviceType.iPhone-14" \
	--runtime "com.apple.CoreSimulator.SimRuntime.iOS-16-4" \
	--test-bundle derivedData/Build/Products/Debug-iphonesimulator/EmceeSampleTestsWithoutHost.xctest \
	--junit tests_without_host_junit.xml

Here is what these options stand for:

• --queue - is a URL of a machine that will serve workers with tests
• --worker - is a URL of a machine that will execute tests that it queries from the queue
• --device and --runtime - are options that specify which simulators will run the tests
• --test-bundle - is a path to the xctest bundle
• --junit - is a path to the JUnit xml that will contain the result of the test run

You can find more about all options accepted by runTests and how to specify them using Emcee runTests -h.

After the test finishes, Emcee will create a tests_without_host_junit.xml file. The JUnit report contains four testcase entries matching the four test methods from the EmceeSampleTestsWithoutHost.xctest test bundle.

<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="xctest" tests="4" failures="1">
    <testsuite name="EmceeSampleTestsWithoutHost" tests="4" failures="1">
        <testcase classname="EmceeSampleTestsWithoutHost" name="test_0___from_tests_without_host___that_always_succeeds" timestamp="2021-12-29T01:15:16+03:00" time="0.012196063995361328"></testcase>
        <testcase classname="EmceeSampleTestsWithoutHost" name="test_1___from_tests_without_host___that_always_succeeds" timestamp="2021-12-29T01:15:27+03:00" time="0.02156198024749756"></testcase>
        <testcase classname="EmceeSampleTestsWithoutHost" name="test_2___from_tests_without_host___that_always_succeeds" timestamp="2021-12-29T01:15:36+03:00" time="0.021990060806274414"></testcase>
        <testcase classname="EmceeSampleTestsWithoutHost" name="test___from_tests_without_host___that_always_fails" timestamp="2021-12-29T01:15:31+03:00" time="0.1255110502243042">
            <failure message="failed - Failure from tests without host">/Users/emcee/Emcee/SampleProject/EmceeSampleTestsWithoutHost/EmceeSampleTestsWithoutHost.swift:17</failure>
            <failure message="failed - Failure from tests without host">/Users/emcee/Emcee/SampleProject/EmceeSampleTestsWithoutHost/EmceeSampleTestsWithoutHost.swift:17</failure>
        </testcase>
    </testsuite>
</testsuites>

For a more sophisticated test reporting mechanism such as Allure, check out the Plugins documentation.

Tests with a host application

Now let's try running tests that require a host application. Host application path is specified using the --app option. For example:

./emceeFree runTests \
    --queue "ssh://emcee:qwerty@ios-build-machine77" \
    --worker "ssh://emcee:qwerty@ios-build-machine77" \
    --worker "ssh://emcee:qwerty@ios-build-machine78" \
    --device "com.apple.CoreSimulator.SimDeviceType.iPhone-14" \
    --runtime "com.apple.CoreSimulator.SimRuntime.iOS-16-4" \
    --app derivedData/Build/Products/Debug-iphonesimulator/EmceeSample.app \
    --test-bundle derivedData/Build/Products/Debug-iphonesimulator/EmceeSample.app/PlugIns/EmceeSampleHostedTests.xctest \
    --junit tests_with_host_junit.xml

To get a visual confirmation that Emcee is running the tests, you can open the Simulator app on the worker machines:

open "$(xcode-select -p)"/Applications/Simulator.app

XCUI tests

Finally, we will run XCUI tests by adding a --runner option and changing the --test-bundle option to the XCUI test bundle:

./emceeFree runTests \
    --queue "ssh://emcee:qwerty@ios-build-machine77" \
    --worker "ssh://emcee:qwerty@ios-build-machine77" \
    --worker "ssh://emcee:qwerty@ios-build-machine78" \
    --device "com.apple.CoreSimulator.SimDeviceType.iPhone-14" \
    --runtime "com.apple.CoreSimulator.SimRuntime.iOS-16-4" \
    --runner derivedData/Build/Products/Debug-iphonesimulator/EmceeSampleUITests-Runner.app \
    --app derivedData/Build/Products/Debug-iphonesimulator/EmceeSample.app \
    --test-bundle derivedData/Build/Products/Debug-iphonesimulator/EmceeSampleUITests-Runner.app/PlugIns/EmceeSampleUITests.xctest \
    --junit ui_tests_junit.xml

This is how the test run will look: