Remote

Remote - Control your iPhone from Xcode

“Remote” is a plugin for Xcode that allows you to control an iPhone
from a window on your Mac during development. Originally created to avoid
having to pick up a device during testing you can record “macros”
of device touches and replay them. It will also compare the resulting screen
output against a snapshot for end-to-end testing. The Macro log is an
editable WebView that can be modified at will. Finally, you can now
record and save all display output into a quicktime movie.

Update: This former Xcode plugin has been re-organised into a
Swift Package for use in other apps. To use Remote, simply add this
project as a Swift package and the RemoteCapture target to your app.
It will connect automatically using the hostname of your desktop. You
also need to be running an application containg the RemoteUI target
such as InjectionIII or the
HotReloading daemon with
“Remote Control” enabled to receive connections and render your device’s
screen. By default, adding the package to a project will have it try to
connect to a process running the remote UI. The package manifest
compiles your hostname into the package so it should be able to
connect from a device.

(The gif shows the recording of a macro, saving it and then playing it
back - testing the screen is as expected in the snapshot.)

To use, download the zipped source for this plugin, build and restart
Xcode. You can then patch your project’s main.m to include the Remote
client header using “Product/Remote/Patch App” or it can load from a
bundle on the fly if using the simulator. When using a device check that
the correct IP address has been patched into main.m so the device can connect.
To use with Swift, add an empty main.m to your project so it can be patched.

The display shadowing window will not display by default. Use the
Menu item “Product/Remote/Load” to have it appear. Thereafter, touches
on the shadow display or device applied and recorded. To save a macro
or a sequence of touches, enter a name into the textfield towards the
bottom of the touch display and click the save button. You can then
replay the macro either by loading it using the pulldown menu
at the top of the touch display and clicking replay or directly from
the “Product/Remote/” Menu.

For end-to-end testing, include a snapshot in a macro by clicking
the “Snapshot” button. On replay, the macro will pause until the screen
matches the snapshot within the specified tolerance or it will timeout
asking if you would like to update the snapshot or the tolerance used
(remember to save the updated macro.) The units of tolerance are the
number of bytes the screen image differs after the run length encoding
of simular pixel values.

Macro entries logged/replayed:

• Hardware <hw.machine> - device type from sysctlbyname()

• Device <screen width> <screen height> <snapshot scale> <device scale>

• Begin <wait time> <x> <y> [<x2> <y2>] - touch(s) start

• Moved/Ended <ditto> touches moved/ended - two touches maximum

• Expect timeout:<seconds> tolerance:<bytes different>… <snapshot>

Implementation Classes

UI:

• RMPluginController - interface between the Remote display and Xcode

• RMWindowController - macro re-player and overall nib controller

• RMMacroManager - controls display, saving and loading of macros

Internal (connected by protocol RMDeviceDelegate):

• RMImageView - subclass of NSImageView for event capture/device display

• RMDeviceController - interface between remote display and device

• RemoteCapture.h - #imported into application’s main.m to connect to Xcode

RemoteCapture.h requires a patched main.m to be compiled using ARC.

Limitations

Remote uses [UIWindow.layer renderInContext:] so most activity on the device is
captured including the keyboard but excepting video replay and openGL layers.
UIAlertView prompts are also not captured as they seem to render outside the window
hierarchy. Finally, UIDatePickers are not rendered correctly at all although they will
respond to events. To preserve network bandwidth to the device animations are not played.

Remote performs better with swipe events if you make an initial tap on the device.

Please note: for some reason it takes about a minute before Xcode will accept remote
connections on it’s server socket (firewall complications?). After restarting Xcode
you may need to wait a while before being able to use the Remote plugin from a device.

Thanks

AVFoundation code for video capture adapted from:
https://github.com/acj/TimeLapseBuilder-Swift

MIT license. Please see the LICENSE file for the particulars.