Introduction to UI Unit Testing with UI Automation

Enter the Microsoft UI Automation framework. While recently delving into Windows Presentation Foundation, I came across the UI Automation framework. I thought that it sounded interesting and decided to dive a little deeper. I discovered that the framework provides access to the UI elements on the users desktop, allowing for manipulation and programmatic interaction with the user interface. As I continued learning more about the UI Automation framework, I began to understand how useful it would be in creating automated unit tests against the user interface. This article aims to provide an introduction to the UI Automation framework and how it can be used to create automated unit tests against the user interface. Although the framework can also be used to manipulate Win32 and Windows Forms applications, I will be testing two sample WPF applications. These unit tests are written using Microsofts Unit Testing Framework, but the concepts presented are applicable to tests written using other testing frameworks, including NUnit, Rhino Mocks, and NMock. Introduction to UI Unit Testing with UI Automation - Testing a simple UI Application

To introduce some of the major classes and concepts from the UI Automation Framework, I am going to use the small application below. It contains two checkboxes and two buttons. The first button should be enabled when the first checkbox is checked. The second button should be disabled when the second checkbox is checked.

Before I begin writing any unit tests for the user interface, I need to add references to the necessary assemblies. The two assemblies that I need are UIAutomationClient.dll and UIAutomationTypes.dll, available as of .NET Framework 3.0. The System.Windows.Automation namespace contains all of the objects I will be using. The primary object in the UI Automation Framework is the aptly named AutomationElement. The AutomationElement class provides access to common properties of UI elements as well as any child elements. Each piece of the user interface, starting with the desktop, is represented as an AutomationElement. This means that each window, menu bar, button, list box item, and so on is an AutomationElement. Even a buttons text is represented as an AutomationElement. With every object on the desktop being represented as an AutomationElement, how do we find the one element that we actually want to manipulate? Since everything is contained within the desktop, we start there by using the static field AutomationElement.RootElement. We can then use the FindFirst and FindAll methods to locate specific UI elements. The FindFirst and FindAll methods take two parameters: a TreeScope value and a Condition object. The TreeScope enumeration specifies the scope(s) to search for elements from the current

AutomationElement. To prevent searching every UI element, we will use TreeScope.Children to search only the immediate children of the desktop. TreeScope.Descendants would search the immediate children, their children, and so forth. The Condition object is used to filter the results available within the specified scopes. The most powerful Condition is the PropertyCondition. PropertyCondition allows us to check a property of the AutomationElement against a specific value. The property is an AutomationProperty object and can be obtained from static fields of AutomationElement. To combine multiple PropertyConditions, the Boolean conditions, AndCondition, OrCondition, and NotCondition, can be used. Putting all of this information to use, we will obtain the AutomationElement for our application from the desktop.
// Retrieve the application window Condition windowNameCond = new PropertyCondition(AutomationElement.NameProperty, "Simple App"); AutomationElement appWindow = AutomationElement.RootElement.FindFirst(TreeScope.Children, windowNameCond); Assert.IsNotNull(appWindow, "An element with the name 'Simple App' could not be found.");

The Name property exemplifies how the UI Automation framework is designed to work against varying applications. For window elements, the title maps to the Name property. A WPF button maps the Content property to the Name property, whereas a Win32 button would map its Caption property to Name. By encapsulating these differences between frameworks, developers can use the UI Automation framework seamlessly between WPF, Windows Forms, and Win32 applications. We used the Name property to find our application. In an application with a large number of elements, relying on the Name property could cause some confusion when trying to locate a specific element. To help automation clients find an element, WPF provides the attached string property AutomationProperties.AutomationId. By setting the AutomationId (in XAML or programmatically), we can easily find the element we want. The next code block demonstrates this as we obtain the AutomationElements for the checkboxes and buttons.
// Retrieve the checkbox and button elements by AutomationId AutomationElement checkBox1 = appWindow.FindFirst(TreeScope.Children, new PropertyCondition(AutomationElement.AutomationIdProperty, "CheckBox1")); Assert.IsNotNull(checkBox1, "An element with AutomationID 'CheckBox1' could not be found."); AutomationElement checkBox2 = appWindow.FindFirst(TreeScope.Children, new PropertyCondition(AutomationElement.AutomationIdProperty, "CheckBox2")); Assert.IsNotNull(checkBox2, "An element with AutomationID 'CheckBox2' could not be found."); AutomationElement button1 = appWindow.FindFirst(TreeScope.Children, new PropertyCondition(AutomationElement.AutomationIdProperty, "Button1")); Assert.IsNotNull(button1, "An element with AutomationID 'Button1' could not be found."); AutomationElement button2 = appWindow.FindFirst(TreeScope.Children, new PropertyCondition(AutomationElement.AutomationIdProperty, "Button2")); Assert.IsNotNull(button2, "An element with AutomationID 'Button2' could not be found.");

Now that we have our AutomationElements, how can we manipulate them? Using the Current property or GetCurrentPropertyValue method of AutomationElement only provides us read-only access to common properties of the elements. The UI Automation framework provides an answer with control patterns. A control pattern defines the behaviors and properties that are specific to a type of control. For example, a button can support the Invoke pattern and a list box can support the Scroll pattern. An element can support multiple control patterns, and the patterns supported can change depending on the current state of the element. A text box would only support the Scroll pattern if the scroll bars are, or could become, visible. In this next code block, we retrieve TogglePatterns for the checkboxes.

// Retrieve Toggle patterns for checkboxes TogglePattern checkBox1Pattern = null; try { checkBox1Pattern = checkBox1.GetCurrentPattern( TogglePattern.Pattern) as TogglePattern; } catch (InvalidOperationException) { Assert.Fail("CheckBox1 does not support the Toggle pattern"); } catch (ElementNotAvailableException) { Assert.Fail("CheckBox1 is no longer available"); } ...

With the TogglePatterns obtained, we can now manipulate the checkboxes and examine properties that are specific to toggle controls. Using the Current property of the TogglePattern objects, we can check the current ToggleState value. The TogglePattern object provides the Toggle method; this method allows us to simulate a user changing the state of the checkbox. The final code block demonstrates using the TogglePatterns and verifying the enabled states of the buttons.
// Initialize the checkboxes to be unchecked while (checkBox1Pattern.Current.ToggleState != ToggleState.Off) checkBox1Pattern.Toggle(); while (checkBox2Pattern.Current.ToggleState != ToggleState.Off) checkBox2Pattern.Toggle(); // Verify that the buttons are enabled/disabled properly Assert.AreEqual<bool>(false, (bool)button1.GetCurrentPropertyValue(AutomationElement.IsEnabledProperty), "When CheckBox1 is unchecked, Button1 should be disabled."); Assert.AreEqual<bool>(true, (bool)button2.GetCurrentPropertyValue(AutomationElement.IsEnabledProperty), "When CheckBox2 is unchecked, Button2 should be enabled."); // Change the checkboxes to be checked and verify the button states again checkBox1Pattern.Toggle(); checkBox2Pattern.Toggle(); Assert.AreEqual<bool>(true, (bool)button1.GetCurrentPropertyValue( AutomationElement.IsEnabledProperty), "When CheckBox1 is checked, Button1 should be enabled."); Assert.AreEqual<bool>(false, (bool)button2.GetCurrentPropertyValue( AutomationElement.IsEnabledProperty), "When CheckBox2 is checked, Button2 should be disabled.");

Introduction to UI Unit Testing with UI Automation - Testing a NotePad-Like Application

With the basics out of the way, I have an application that provides more complexity. The application shown below is a simple text editor. It contains a single text box control, a menu bar, and a formatting toolbar. The File menu contains a Close command. The Edit menu contains commands for Cut, Copy, Paste, Undo, and Redo. The Format menu contains a checkable menu item for turning word wrap on and off.

I created five different unit tests to test different aspects of the user interface for this application. To aid in reducing the amount of code and assertions in each unit test, I created some helper functions to handle finding elements and retrieving control patterns. I also created the following method to run during the initialization of each test. During initialization, the application is located by AutomationId. If it is not found, the application is started. After starting, the application window is repositioned and resized using the Transform pattern. The RuntimeId that is stored for later use in one of the tests is an identifier that is unique across all elements on the desktop.
private string SampleText = "lorem ipsum"; private string[] SampleLongText = new string[]{ "Lorem ipsum dolor sit amet, consectetuer adipiscing elit. ", ... "Duis eleifend ipsum quis tellus."}; private AutomationElement AppElement; private int[] AppRuntimeId; [TestInitialize()] public void TestInitialize() { // Check to see if the application is already running AppElement = AutomationElement.RootElement.FindFirst(TreeScope.Children, new PropertyCondition(AutomationElement.AutomationIdProperty, "TextApp")); if (AppElement == null) { // Start the application Process.Start(@"../../../TextApp/bin/Debug/TextApp.exe"); Thread.Sleep(1000); // Retrieve the application window by the automation id AppElement = FindElement(AutomationElement.RootElement, TreeScope.Children, AutomationElement.AutomationIdProperty, "TextApp"); // Position and resize the application window TransformPattern transformPattern = GetPattern(AppElement, TransformPattern.Pattern) as TransformPattern; transformPattern.Move(0, 0); transformPattern.Resize(400, 250); Assert.AreEqual<Rect>(new Rect(0, 0, 400, 250), AppElement.Current.BoundingRectangle);

// Store the RuntimeId for the application window for later validation AppRuntimeId = (int[])AppElement.GetCurrentPropertyValue( AutomationElement.RuntimeIdProperty); } }

The first unit test validates the formatting toolbar buttons and combo boxes. The buttons use the Invoke pattern, while the individual items in the combo boxes support the Selection Item pattern. For the text control, the Text pattern provides formatting and styling information. First, we need to set the controls to an initial state. Then, we will toggle the buttons and change the values of each combo box. We will check the state of the Text pattern to validate that the bindings between the formatting controls and the text control are working properly.
[TestMethod] public void FormattingToolbarTest() { // Retrieve toolbar elements AutomationElement boldElement = FindElement(AppElement, AutomationElement.AutomationIdProperty, "Bold"); ... AutomationElement fontFamilyElement = FindElement(AppElement, AutomationElement.AutomationIdProperty, "FontName"); AutomationElement arialElement = FindElement(fontFamilyElement, AutomationElement.NameProperty, "Arial"); ... // Retrieve the text element AutomationElement textBoxElement = FindElement(AppElement, AutomationElement.AutomationIdProperty, "TextBox"); // Retrieve Toggle patterns for the toolbar buttons TogglePattern boldPattern = GetPattern(boldElement, TogglePattern.Pattern) as TogglePattern; ... // Retrieve SelectionItem patterns for the combo box items SelectionItemPattern arialPattern = GetPattern(arialElement, SelectionItemPattern.Pattern) as SelectionItemPattern; ... // Retrieve Text pattern for the text control TextPattern textPattern = GetPattern(textBoxElement, TextPattern.Pattern) as TextPattern; // Set the formatting options to initial states while (boldPattern.Current.ToggleState != ToggleState.Off) boldPattern.Toggle(); ... arialPattern.Select(); ... // Verify that the text control reflects the current formatting options Assert.AreEqual<int>(400, (int)textPattern.DocumentRange.GetAttributeValue( TextPattern.FontWeightAttribute)); Assert.IsFalse((bool)textPattern.DocumentRange.GetAttributeValue( TextPattern.IsItalicAttribute)); Assert.AreEqual<TextDecorationLineStyle>(TextDecorationLineStyle.None, (TextDecorationLineStyle)textPattern.DocumentRange.GetAttributeValue(TextPattern.UnderlineStyleAttribute)); Assert.AreEqual<string>("arial", textPattern.DocumentRange.GetAttributeValue( TextPattern.FontNameAttribute).ToString().ToLower()); Assert.AreEqual<double>(12 * 72 / 96, (double)textPattern.DocumentRange.GetAttributeValue( TextPattern.FontSizeAttribute)); Assert.AreEqual<Color>( Color.FromArgb(0, Color.Black.R, Color.Black.G, Color.Black.B), Color.FromArgb((int)textPattern.DocumentRange.GetAttributeValue(TextPattern.ForegroundColorAttribute))); // Change all of the formatting options

boldPattern.Toggle(); ... // Verify that the text control reflects the current formatting options Assert.AreEqual<int>(700, (int)textPattern.DocumentRange.GetAttributeValue(TextPattern.FontWeightAttribute)); ... }

Introduction to UI Unit Testing with UI Automation - Entering Text The next unit test simply tests entering text. Modifying the text is accomplished either by using keyboard input or the Value pattern. Not all text controls support Value pattern, so both methods should be handled. After this unit test, we will use SendKeys for all text changes.
[TestMethod()] public void EnteringTextTest() { // Retrieve the text element AutomationElement textBoxElement = FindElement(AppElement, AutomationElement.AutomationIdProperty, "TextBox"); // Retrieve Text and Value (if available) patterns for the text control TextPattern textPattern = GetPattern(textBoxElement, TextPattern.Pattern) as TextPattern; object objPattern; if (!textBoxElement.TryGetCurrentPattern(ValuePattern.Pattern, out objPattern)) { Assert.Inconclusive("TextBox does not support the Value pattern."); objPattern = null; } ValuePattern valuePattern = objPattern as ValuePattern; // Make sure that the text control has focus before modifying the text textBoxElement.SetFocus(); if (valuePattern == null) { // Value pattern is not supported, so use SendKeys to modify the text Thread.Sleep(100); SendKeys.SendWait("^{HOME}"); SendKeys.SendWait("^+{END}"); SendKeys.SendWait("{DEL}"); } else { // Since Value pattern is supported, // use the SetValue method to modify the text valuePattern.SetValue(""); } // Verify that the text control is empty Assert.AreEqual<string>("", textPattern.DocumentRange.GetText(-1)); if (valuePattern == null) { // Value pattern is not supported, so use SendKeys to modify the text Thread.Sleep(100); SendKeys.SendWait(SampleText); } else

{ // Since Value pattern is supported, // use the SetValue method to modify the text valuePattern.SetValue(SampleText); } // Verify that the text in the text control matches our sample text Assert.AreEqual<string>(SampleText, textPattern.DocumentRange.GetText(-1)); }

Introduction to UI Unit Testing with UI Automation - Verifying Commands

The Edit menu contains commands that can alter the text, so the next unit test verifies the functionality of these commands. When retrieving the elements for the menu items, the menu needs to be expanded so that these items are generated and available. Menu items can support different control patterns; in this case, we need to use the Expand Collapse pattern to expand and collapse the Edit menu; the menu items under the Edit menu support the Invoke pattern.
[TestMethod()] public void EditMenuTest() { // Retrieve Edit menu element AutomationElement menuElement = FindElement(AppElement, AutomationElement.AutomationIdProperty, "Edit"); // Expand the menu to generate menu items ExpandCollapsePattern menuPattern = GetPattern(menuElement,ExpandCollapsePattern.Pattern) as ExpandCollapsePattern; menuPattern.Expand(); AutomationElement cutElement = FindElement(menuElement, AutomationElement.AutomationIdProperty, "Cut"); ... menuPattern.Collapse(); // Retrieve the text element AutomationElement textBoxElement = FindElement(AppElement, AutomationElement.AutomationIdProperty, "TextBox"); // Retrieve Invoke patterns for the Edit menu items InvokePattern cutPattern = GetPattern(cutElement, InvokePattern.Pattern) as InvokePattern; ... // Retrieve Text pattern for the text control TextPattern textPattern = GetPattern(textBoxElement, TextPattern.Pattern) as TextPattern; // Make sure that the text control has focus before modifying the text textBoxElement.SetFocus(); // Set the initial text in the control Thread.Sleep(100); SendKeys.SendWait("^{HOME}"); SendKeys.SendWait("^+{END}"); SendKeys.SendWait("{DEL}"); SendKeys.SendWait(SampleText); Assert.AreEqual<string>(SampleText, textPattern.DocumentRange.GetText(-1)); // Validate that the Undo and Redo menu items work properly Thread.Sleep(100); undoPattern.Invoke();

Assert.AreEqual<string>("", textPattern.DocumentRange.GetText(-1)); redoPattern.Invoke(); Assert.AreEqual<string>(SampleText, textPattern.DocumentRange.GetText(-1).ToLower)); // Validate that the Copy and Paste menu items work properly Thread.Sleep(100); SendKeys.SendWait("^{END}"); SendKeys.SendWait("+{LEFT}+{LEFT}+{LEFT}"); copyPattern.Invoke(); SendKeys.SendWait("^{HOME}"); pastePattern.Invoke(); Assert.AreEqual<string>(SampleText.Substring(SampleText.Length - 3) + SampleText, textPattern.DocumentRange.GetText(-1).ToLower()); // Validate that the Cut and Paste menu items work properly Thread.Sleep(100); SendKeys.SendWait("^{HOME}"); SendKeys.SendWait("+{RIGHT}+{RIGHT}+{RIGHT}"); cutPattern.Invoke(); SendKeys.SendWait("^{END}"); pastePattern.Invoke(); Assert.AreEqual<string>(SampleText + SampleText.Substring( SampleText.Length - 3), textPattern.DocumentRange.GetText(-1).ToLower()); }

Introduction to UI Unit Testing with UI Automation - Testing a more complex command

The next unit test focuses on the Word Wrap menu item. It is retrieved in the same manner as the Edit menu items. In this case, the menu item is checkable; so, it supports the Toggle pattern rather than the Invoke pattern. The Word Wrap function affects the text controls ability to scroll. The text control in this application is set to automatically display scroll bars when necessary, so it supports the Scroll pattern. This pattern will enable us to determine if the scroll bars are available for use by the text control. With Word Wrap off, the text control should only scroll horizontally; with it on, the text control should only scroll vertically. Moving the caret in the text control changes the position of the thumbs in the scroll bars. We can verify their positions by using the HorizontalScrollPercent and VerticalScrollPercent properties. The thumb position is represented as a percentage of the total scroll bar; so, the leftmost or topmost position would be 0%, and the rightmost or bottommost position would be 100%. This next block of code demonstrates this by moving the caret to the beginning and end of the text.
[TestMethod()] public void WordWrapTest() { // Retrieve Format menu element AutomationElement menuElement = FindElement(AppElement, AutomationElement.AutomationIdProperty, "Format"); // Expand the menu to generate menu items ExpandCollapsePattern menuPattern = GetPattern(menuElement, ExpandCollapsePattern.Pattern) as ExpandCollapsePattern; menuPattern.Expand(); AutomationElement wordWrapElement = FindElement(menuElement, AutomationElement.AutomationIdProperty, "WordWrap"); menuPattern.Collapse();

// Retrieve the text element AutomationElement textBoxElement = FindElement(AppElement, AutomationElement.AutomationIdProperty, "TextBox"); // Retrieve Toggle pattern for the Word Wrap menu item TogglePattern wordWrapPattern = GetPattern(wordWrapElement, TogglePattern.Pattern) as TogglePattern; // Retrieve Scroll pattern for the text control ScrollPattern scrollPattern = GetPattern(textBoxElement, ScrollPattern.Pattern) as ScrollPattern; // Make sure that the text control has focus before modifying the text textBoxElement.SetFocus(); // Make sure that the Word Wrap menu item is unchecked while (wordWrapPattern.Current.ToggleState != ToggleState.Off) wordWrapPattern.Toggle(); // Delete all the text in the text control Thread.Sleep(100); SendKeys.SendWait("^{HOME}"); SendKeys.SendWait("^+{END}"); SendKeys.SendWait("{DEL}"); // Validate that, with no text, the text control is not scrollable Assert.IsFalse(scrollPattern.Current.HorizontallyScrollable); Assert.IsFalse(scrollPattern.Current.VerticallyScrollable); // Enter a large amount of text into the text control Thread.Sleep(100); for (int i = 0; i < SampleLongText.Length; i++) { SendKeys.SendWait(SampleLongText[i]); } // Validate that, with a lot of text, the text control // can scroll horizontally, but not vertically Assert.IsTrue(scrollPattern.Current.HorizontallyScrollable); Assert.IsFalse(scrollPattern.Current.VerticallyScrollable); // Turn word wrap on, and validate that now // the text control can only scroll vertically wordWrapPattern.Toggle(); Assert.IsFalse(scrollPattern.Current.HorizontallyScrollable); Assert.IsTrue(scrollPattern.Current.VerticallyScrollable); // Move the caret to the beginning and verify // that the thumb is at the top of the vertical scroll bar Thread.Sleep(100); SendKeys.SendWait("^{HOME}"); Assert.AreEqual<double>(0, Math.Round(scrollPattern.Current.VerticalScrollPercent, 0)); // Move the caret to the beginning and verify that // the thumb is at the bottom of the vertical scroll bar Thread.Sleep(100); SendKeys.SendWait("^{END}"); Assert.AreEqual<double>(100, Math.Round(scrollPattern.Current.VerticalScrollPercent, 0)); }

Introduction to UI Unit Testing with UI Automation - Testing with Events

The final test method breaks with the synchronous, self-contained nature of unit tests by introducing events. The UI Automation framework supplies events related to the control patterns, like InvokedEvent or WindowOpenedEvent, as well as more general events such as focus or property changes. To demonstrate how events can be used, we will watch for our application to be closed through the WindowClosedEvent. First, we will need a class variable for our event handler. Next, within the test method, we call the static Automation.AddAutomationEventHandler. With this method, we inform the UI Automation framework that we are listening for the Window Closed event for our application and specify our event handler. Finally, we close the application using the Window pattern. The event handler receives the generic AutomationEventArgs object that contains the identifier for what event occurred. Once we know that the Window Closed event occurred, we cast the event arguments to a WindowClosedEventArgs object. This specific object provides the RuntimeId for the window that was just closed. Additional code verifies that the RuntimeId matches the one for our application and that the RuntimeId does not match the RuntimeIds for any child windows of the desktop. Finally, we need to unsubscribe from any automation events once our tests have completed. After each test completes running, I call Automation.RemoveAllEventHandlers in the TestCleanup method. Other removal methods allow for specifying specific events and automation elements; since our application is closed at this point, RemoveAllEventHandlers is the safest method.
private AutomationEventHandler UIAutomationEventHandler; [TestMethod()] public void CloseApplicationTest() { // Listen for the WindowClosed event on the application window Automation.AddAutomationEventHandler( WindowPattern.WindowClosedEvent, AppElement, TreeScope.Element, UIAutomationEventHandler = new AutomationEventHandler( OnUIAutomationEvent)); // Use the Window pattern to close the application window WindowPattern windowPattern = GetPattern(AppElement, WindowPattern.Pattern) as WindowPattern; windowPattern.Close(); } private void OnUIAutomationEvent(object sender, AutomationEventArgs e) { // Verify that the WindowClosed event occurred if (e.EventId == WindowPattern.WindowClosedEvent) { // Cast the AutomationEventArgs paramater to WindowClosedEventArgs // now that we know it is the WindowClosed event WindowClosedEventArgs args = e as WindowClosedEventArgs; // Validate the RuntimeId for the window that // closed against the RuntimeId // for the application we have been testing Assert.AreEqual<int[]>(AppRuntimeId, args.GetRuntimeId()); // Retrieve all children of the desktop AutomationElementCollection windows = AutomationElement.RootElement.FindAll(TreeScope.Children, Condition.TrueCondition);

// Verify that none of the desktop children have the same RuntimeId // as the application we have been testing and just closed foreach (AutomationElement window in windows) { if (Automation.Compare(args.GetRuntimeId(), (int[])window.GetCurrentPropertyValue( AutomationElement.RuntimeIdProperty))) { Assert.Fail(); } } } else Assert.Fail(); } [TestCleanup()] public void TestCleanup() { // Unsubscribe from all event handlers now // that the automation test is complete Automation.RemoveAllEventHandlers(); }

Introduction to UI Unit Testing with UI Automation - Summary I hope that this article has provided a good introduction to how the UI Automation framework can be used for the automated testing of user interfaces. When I first began learning about the framework, I thought that I would find it complicated and too much of a burden to ever use in real projects. I quickly realized that the key to understanding the framework is to look at applications from a users perspective rather than the point of view of the developer. The control patterns match how a user interacts with and views the system. The properties of automation elements and control patterns support how a user thinks of an application; they are not meant to expose every property that can be set by a developer. The capabilities of the UI Automation framework extend far beyond the sample applications and tests in this article. There are a number of other control patterns to explore. Using the Text pattern with rich text controls can be quite powerful. Microsoft has created a framework that simplifies the manipulation of applications user interfaces. In doing so, developers now have another tool available to create solid and well tested applications. A good tool for examining applications, automation properties, and control patterns is UI Spy from Microsoft. It was removed from the Windows SDK for Windows Server 2008 and .NET Framework 3.5, but it is still available in the Windows SDK Update for Windows Vista.