Escolar Documentos
Profissional Documentos
Cultura Documentos
PUBLISHED BY Enovation Press, Corp. A Division of Software Solutions, Corp PO Box 46462 Madison, WI 53744 Copyright @2010 by Rigoberto Garcia All rights reserved. No part of the contents of this book may be reproduced or transmitted in any form or by any means without written expressed permission of the publisher.
978-1-105-99506-4
Information has been obtained by Publisher from sources believed to be reliable. However, because of the possibility of human or mechanical errors by our sources, Publisher, or others, do not guarantee the accuracy, or completeness of any information included in this work and are not responsible for any errors or omissions or the results obtained from the use of such information Enovation Press, Inc. does not make any representations or warranties as to the accuracy, adequacy, or completeness of any information contained in this Work, and is not responsible for any errors or omissions.
subsequent chapters. For this reason, you should work through the chapters in sequence, although the information on each chapter could be used stand alone also. I have organized each chapter into results-oriented topics. Topics include all the relevant and supporting information you need to master Automated UI Testing Using Visual Studio 2010, and System Center 2012 Virtual Machine Manager. All Content is based on framework 3.5 or Framework 4.0 and it cover heavily virtualization technology and SharePoint 2010 UI Testing and Server Core (Longhorn) technology. Activities or Labs found in this book, will allow you to apply this information to practical hands-on examples. At times I will use several business scenarios. These business scenarios immerse you in a comfortable environment where you are presented with situations that require you to make real world decisions based on the principles you have learned in the course of this book. Within these simulations, you will meet characters, have access to clear and timely feedback, and have the ability to review and change previous decisions. These business simulations are available on the support website at http://northamerica.softwaresolutioncorp.com/kb and can be downloaded from any computer, tablet or smart device. The course manual can be also found at the site and PowerPoint presentations. .
As a Review Tool
Any method of instruction is only as effective as the time and effort you are willing to investing it. In addition, some of the information that you learn in class may not be important to you immediately, but it may become important later on. For this reason, we encourage you to spend some time reviewing the topics and activities after the course. For an additional challenge when reviewing activities, try the Why To Do It column before looking at the How To Do It column.
As a Reference
The organization and layout of the book make it easy to use as a learning tool and as an after class reference. You can use this book as a first source for definitions of terms, background information on given topics, and summaries of procedures.
Table of Contents
Chapter 1....................................................................................................................................... 17 Why Automated Testing? ............................................................................................................. 17 Introduction............................................................................................................................... 18 Manual software testing ........................................................................................................... 18 Automated software testing tool .............................................................................................. 18 Improves Accuracy................................................................................................................. 19 Increases Test Coverage ........................................................................................................ 19 Does What Manual Testing Cannot ....................................................................................... 19 Helps Developers and Testers ............................................................................................... 19 Improves Team Morale ......................................................................................................... 19 UI Automation for Testing ......................................................................................................... 20 Implementing UI Automation................................................................................................ 21 UI Automation in a Client .......................................................................................................... 21 Programmatic Access ........................................................................................................ 22 Key Properties for Test Automation ................................................................................. 22 AutomationID.................................................................................................................... 22 ControlType....................................................................................................................... 23 NameProperty................................................................................................................... 23 Implementing UI Automation in a Test Application ......................................................... 23 Chapter 2....................................................................................................................................... 25 Application Lifecycle Management............................................................................................... 25 Scenario #1:....................................................................................................................... 27 The Client Tier ........................................................................................................................ 28 The Application Tier ............................................................................................................... 28 The Data Tier ......................................................................................................................... 29 Automatic Compiling ............................................................................................................. 29 Application Lifecycle Management ....................................................................................... 30 Microsoft Test Manager: .................................................................................................. 30
Visual Studio Premium: ..................................................................................................... 30 Visual Studio Ultimate ...................................................................................................... 30 Overview of Virtual Lab for Application Lifecycle Management .............................................. 32 What are Virtual Environments ............................................................................................. 34 Simple Virtual Environments ................................................................................................. 35 Complex Virtual Environments .............................................................................................. 36 Targeted Virtual Environments ............................................................................................. 36 Virtual Environments ................................................................................................................ 37 Virtual machines (Hyper-V or VMware) ................................................................................ 38 Virtual machine ................................................................................................................. 38 Host ................................................................................................................................... 38 Snapshot ........................................................................................................................... 39 System Center Virtual Machine Manager (SCVMM) ............................................................. 39 Template ........................................................................................................................... 39 Host group......................................................................................................................... 39 Library server .................................................................................................................... 39 Library share ..................................................................................................................... 39 Lab Management ................................................................................................................... 40 Virtual environment .......................................................................................................... 40 Physical environment........................................................................................................ 40 Lab Center ......................................................................................................................... 40 Team project library.......................................................................................................... 40 Creating and managing environments .................................................................................. 41 Testing ................................................................................................................................... 42 Build deployment .................................................................................................................. 43 Lab Management ................................................................................................................... 44 Chapter 3....................................................................................................................................... 45 ANATOMY of a Coded UI Test ....................................................................................................... 45 LESSON 1: Overview of Coded UI Test ...................................................................................... 47 Contents of a Coded UI Test.................................................................................................. 47 UIMap.Designer.cs ............................................................................................................ 48 Declarations section .............................................................................................................. 48 UIMap class............................................................................................................................ 48
UIMap methods ..................................................................................................................... 49 UIMap properties .................................................................................................................. 50 UIMap.cs ................................................................................................................................ 51 UIMap class............................................................................................................................ 51 CodedUITest1.cs .................................................................................................................... 52 CodedUITest1 class ................................................................................................................ 52 CodedUITest1 properties....................................................................................................... 52 CodedUITest1 methods ......................................................................................................... 52 UIMap.uitest .......................................................................................................................... 54 Chapter 4....................................................................................................................................... 56 Automated Test ............................................................................................................................ 56 Creating Automated Tests......................................................................................................... 57 Automated Testing Best Activities ........................................................................................ 58 Manual to Automated Test Conversion ................................................................................ 59 Testing Methods........................................................................................................................ 61 Use Cases or User Stories ................................................................................................. 61 Speed Up Testing User Interface ...................................................................................... 61 Improving Code Quality Using Unit Tests ......................................................................... 62 Test How Your Application Performs Under Heavy Loads ............................................... 62 Test the Functionality of the Application......................................................................... 62 Improving Database Code Quality Using Database Unit Tests: ........................................ 62 Converting Manual Test Cases to Automated Tests to Run As Part of a Test Plan .......... 62 Running Automated Tests from a Test Plan.............................................................................. 63 Best Practice Checklist ........................................................................................................... 64 Determine what roles you need in your environment ..................................................... 64 Create a physical environment .............................................................................................. 64 Create a test plan .............................................................................................................. 64 Associating automated test with a Test Case ........................................................................... 66 Testing Scenarios ................................................................................................................... 68 Coded UI Tests ....................................................................................................................... 68 Unit Tests ............................................................................................................................... 68 Database Unit Tests ............................................................................................................... 68 Load tests ............................................................................................................................... 68
10
Web Performance Tests ........................................................................................................ 68 Generic Tests ......................................................................................................................... 68 LESSON 4: Testing the Application ............................................................................................ 69 Planning your testing strategy ............................................................................................... 70 Planning your testing effort .............................................................................................. 70 Creating and managing tests ............................................................................................ 70 Setting up how to run tests or collect data remotely ....................................................... 70 Running Your Tests ........................................................................................................... 70 Submitting bugs ..................................................................................................................... 70 Tracking software quality.................................................................................................. 71 ................................................................................................................................................... 71 LAB: Automated Test Case ........................................................................................................ 72 Create and Configure Test Projects for Automated Tests ........................................................ 76 ............................................................................................................................................... 76 LAB 1: Create a Test Project When You Add a Test ............................................................... 77 ............................................................................................................................................... 77 LAB 2: Create a test project as you generate unit tests ........................................................ 78 ............................................................................................................................................... 78 LAB 3: Create a test project by using the Add New Project dialog box ................................ 79 LAB 4: Configure Test Project Creation ................................................................................. 80 Chapter 5....................................................................................................................................... 82 Tracking Software Quality ............................................................................................................. 82 Test Dashboard (Agile) .......................................................................................................... 83 SharePoint 2010 Developers Dashboards ............................................................................. 84 Counters in developer dashboard ......................................................................................... 84 Pre-Process counters ........................................................................................................ 84 Client Counters ................................................................................................................. 85 System Counter ................................................................................................................. 85 How to use Monitor in Code?................................................................................................ 85 Developer Dash Board in a custom Page .............................................................................. 86 STSADM Utility ....................................................................................................................... 86 PowerShell Commands .......................................................................................................... 87 LAB 1: Create the Portal in SharePoint and then fix up the missing documents ...................... 88
11
Integrate portal site with team project ................................................................................. 89 Download process template.................................................................................................. 90 Add Product and Iteration backlog........................................................................................ 90 Bind Iteration backlog workbook to the team project .......................................................... 91 Bind Product backlog workbook to the team project ........................................................... 93 Chapter 6....................................................................................................................................... 94 Create a Coded UI Test ................................................................................................................. 94 Introduction............................................................................................................................... 95 LAB: Create a coded UI test....................................................................................................... 97 Coding UI Test ........................................................................................................................... 98 LAB: Generating a coded UI test by recording the application under test ............................... 99 Chapter 7..................................................................................................................................... 104 UI Controls and Validation Code Using the Coded UI Test Builder ............................................ 104 Chapter 8..................................................................................................................................... 109 Data-Driven Coded UI Test ......................................................................................................... 109 Configuring the UI Test DataSource ........................................................................................ 111 LAB: Create the file and add it to your test project. ............................................................... 116 Action Recording ................................................................................................................. 117 LAB: Generate a Coded UI Test from an Action Recording .................................................... 118 Chapter 9..................................................................................................................................... 121 Actions and Properties in a Coded UI ......................................................................................... 121 Accessing Actions of UI Test Control ...................................................................................... 122 Chapter 10................................................................................................................................... 125 What to do if your UI Changes .................................................................................................... 125 LAB: Creating, Editing and Maintaining a Coded UI Test ........................................................ 126 Chapter 11................................................................................................................................... 135 Extending Coded UI Tests and Action Recordings to Support Microsoft Excel .......................... 135 Architecture Overview ............................................................................................................ 136 Create A Test Case ............................................................................................................... 137 Create The CodedUI Watin Test .......................................................................................... 137 Testing Using Virtual Environments ........................................................................................ 139 Running Tests .......................................................................................................................... 141 ................................................................................................................................................ 143
12
Database Unit Tests ................................................................................................................ 144 Chapter 12................................................................................................................................... 145 Running Automated Tests........................................................................................................... 145 Create Test Cases from Automated Tests Using TCM.exe ......................................................... 165 Introduction............................................................................................................................. 166 Common Parameters for the Procedures ............................................................................... 167 LAB: Create Test Cases from an Assembly of Automated Tests ............................................. 168 ................................................................................................................................................. 172 EXERCISE: ................................................................................................................................. 173
13
14
15
16
Chapter 1
Why Automated Testing?
17
Introduction
Every software development group tests its products, yet delivered software always has defects. Test engineers strive to catch them before the product is released but they always creep in and they often reappear, even with the best manual testing processes. Automated software testing is the best way to increase the effectiveness, efficiency and coverage of your software testing.
The Question is: What makes automated software testing so important to these successful companies? Automated Software Testing Saves Time and Money Software tests have to be repeated often during development cycles to ensure quality. Each time source code is modified software tests should be repeated. For every release of the software it may be tested on all supported operating systems and hardware configurations. Physically repeating these tests is costly and time consuming. Once created, automated tests can be run over and over again at no extra cost and they are much faster than manual tests. Automated software testing can decrease the time to run repetitive tests from days to hours.
18
Improves Accuracy
Even the most meticulous tester will make mistakes during monotonous manual testing. Automated tests perform the same steps precisely every time they are executed and never forget to record detailed results thru reporting strategies and monitoring webparts.
19
development team time to spend on more challenging and rewarding projects. Team members increase their skill sets and self-assurance and, in turn, pass those gains on to their organization.
Together they are required to implement UI Automation for it to be useful as an automated test tool. UI Automation providers are applications such as Microsoft Word, Excel, and other third-party applications or controls based on the Microsoft Windows operating system. UI Automation clients include automated test scripts and assistive technology applications. There are multiple topics to consider in UI Automation for Testing, such as: UI Automation in a Provider UI Automation in a Client Related Tools and Technologies Security Related Topics For a UI to be automated, a developer of an application or control must look at what actions an end-user can perform on the UI object using standard keyboard and mouse interaction. Once these key actions have been identified, the corresponding UI Automation control patterns (that is, the control patterns that mirror the functionality and behavior of the UI element) should be implemented on the control. For example, user interaction with a combo box control
20
(such as the run dialog) typically involves expanding and collapsing the combo box to hide or display a list of items, selecting an item from that list, or adding a new value via keyboard input. Note With other accessibility models, developers must gather information directly from individual buttons, menus, or other controls. Unfortunately, every control type comes in dozens of minor variations. In other words, even though ten variations of a pushbutton may all work the same way and perform the same function, they must all be treated as unique controls. There is no way to know that these controls are functionally equivalent. Control patterns were developed to represent these common control behaviors.
Implementing UI Automation
As mentioned earlier, without the unified model provided by UI Automation, test tools and developers are required to know framework-specific information in order to expose properties and behaviors of controls in that framework. Since there can be several different UI frameworks present at any single time within Windows operating systems, including Win32, Windows Forms, and Windows Presentation Foundation (WPF), it can be a daunting task to test multiple applications with controls that seem similar. For example, the following table outlines the framework-specific property names required to retrieve the name (or text) associated with a button control and shows the single equivalent UI Automation property. UI Automation Control Type Button Button Image UI Framework Windows Presentation Foundation Win32 HTML Framework Specific Property Content Caption alt UI Automation Property NameProperty NameProperty NameProperty
UI Automation providers are responsible for mapping the framework-specific properties of their controls to the equivalent UI Automation properties.
UI Automation in a Client
The goal of many automated test tools and scenarios is the consistent and repeatable manipulation of the user interface. This can involve unit testing specific controls through to the recording and playback of test scripts that iterate through a series of generic actions on a group of controls. A complication that arises from automated applications is the difficulty synchronizing a test with a dynamic target. For example, a list box control, such as one contained in the Windows Task Manager, that displays a list of currently running applications. Since the items in the list box are dynamically updated outside the control of the test application, attempting to repeat
21
the selection of a specific item in the list box with any consistency is impossible. Similar issues can also arise when attempting to repeat simple focus changes in a UI that is outside the control of the test application. Programmatic Access Programmatic access provides the ability to imitate, through code, any interaction and experience exposed by traditional mouse and keyboard input. UI Automation enables programmatic access through five components:
The UI Automation tree facilitates navigation through the structure of the UI. The tree is built from the collection of hWnd's. For more information, see UI Automation Tree Overview Automation elements are individual components in the UI. These can often be more granular than an hWnd. For more information, see UI Automation Control Types Overview. Automation properties provide specific information about UI elements. For more information, see UI Automation Properties Overview. Control patterns define a particular aspect of a control's functionality; they can consist of property, method, event, and structure information. For more information, see UI Automation Control Patterns Overview . Automation events provide event notifications and information. For more information, see UI Automation Events Overview.
Key Properties for Test Automation The ability to uniquely identify and subsequently locate any control within the UI provides the basis for automated test applications to operate on that UI. There are several Microsoft UI Automation properties used by clients and providers that assist in this. AutomationID Uniquely identifies an automation element from its siblings. AutomationIdProperty is not localized, unlike a property such as NameProperty that is typically localized if a product gets shipped in multiple languages. See Use the AutomationID Property.
Note AutomationIdProperty does not guarantee a unique identity throughout the automation tree. For example, an application may contain a menu control with multiple top-level menu items that, in turn, have multiple child menu items. These secondary
22
menu items may be identified by a generic scheme such as "Item1, Item 2, Item3, etc.", allowing duplicate identifiers for children across top-level menu items. ControlType Identifies the type of control represented by an automation element. Significant information can be inferred from knowledge of the control type. NameProperty This is a text string that identifies or explains a control. NameProperty should be used with caution since it can be localized. Implementing UI Automation in a Test Application Add the UI Automation References. The UI Automation dll's necessary for UI Automation clients are listed here. UIAutomationClient.dll provides access to the UI Automation client-side APIs. UIAutomationClientSideProvider.dll provides the ability to automate Win32 controls. See UI Automation Support for Standard Controls. UIAutomationTypes.dll provides access to the specific types defined in UI Automation. This namespace contains everything UI Automation clients need to use the capabilities of UI Automation except text handling. This namespace contains everything a UI Automation clients need to use the capabilities of UI Automation text handling. Automated test scripts locate UI Automation elements that represent controls of interest within the automation tree.
System.Windows.Automation
Query the UI using a Condition statement. This is typically where the language-neutral AutomationIdProperty is used.
23
tool such as UISpy.exe(UI Spy) that is able to itemized the UI automation properties of a control
Use the TreeWalker class to traverse the entire UI Automation tree or a subset thereof. Track focus. Use the hWnd of the control. Use screen location, such as the location of the mouse cursor.
Control patterns expose common behaviors for functionally similar controls. After locating the controls that require testing, automated test scripts obtain the control patterns of interest from those UI Automation elements. For example, the InvokePattern control pattern for typical button functionality or the WindowPattern control pattern for window functionality. Automated test scripts can now control any UI of interest from a UI framework using the information and functionality exposed by the UI Automation control patterns.
Automate the UI
24
Chapter 2
25
LESSON 1:
Application Lifecycle Management (ALM) is a continuous process of managing the life of an application through governance, development and maintenance. ALM is the marriage of business management to software engineering made possible by tools that facilitate and integrate requirements management, architecture, coding, testing, tracking, and release management. A developer can apply proven Activities to manage application's lifecycle by using the suite of tools in Visual Studio Premium and Visual Studio Ultimate in combination with Visual Studio Team Foundation Server. Visual Studio allows the developer to better understand customer needs and more effectively design, implement, and deploy code.
26
Scenario #1: A quality assurance team, task with testing application to determine their production viability, they checked-in code, builds, and test results. QA Team might adopt Activities such as check-in code, builds and test results so that customers value more application and to certify the performance and more reliability of the solution. A QA Test Unit might use the following tools to achieve results:
Plan and track your project: Enact processes and monitor their quality based on customer requirements into working software. Design functionality: Create new or use existing assets Architectural Diagrams: architectural diagrams help communicate critical information about the software solution. Write, unit test, debug, analyze, and profiling: Prototypes for Customer approvals Version control: to manage your source code and other files. Build: verify that requirements have been fulfilled in each build. Test: performance and stress tests. Deploy into virtual environments: Advanced testing.
27
Visual Studio Team Foundation works in three-tier architecture: The Client Tier The application tier The data tier
28
Automatic Compiling
With Team Foundation Build, you can create build definitions to automate compiling applications, running associated tests, performing code analysis, releasing continuous builds, and publishing build reports. To build an application, you create a build definition to specify what projects to build, what triggers a build to run, what automated tests to run, and where to deploy the output. This information is stored in the data warehouse, from which it is retrieved when a build runs. After the build runs, data about the build results is stored back in the warehouse, where it is available to view through build reports. The following illustration shows the three main phases of building an application: Three phases of building an application
29
simplifies solution development, Lowers risk and increases ROI return by providing tools for every stage of the lifecycle, from design and development through test and deployment. can design your application in UML,
30
Analyzes your existing software assets. defines software architecture Validate that design when your team checks in and builds with Team Foundation Server
The following table shows how these tools can be used throughout the application lifecycle. Application Lifecycle Plan and Track Design Develop Test Build Team Foundation Server Microsoft Test Manager Visual Studio Premium Visual Studio Ultimate
31
LESSON 3:
Overview of Virtual Lab for Application Lifecycle Management
32
Lab Management is an extension of Microsoft Test Manager that helps you to optimize the use of Microsoft Hyper-V technology to manage and use virtual machines in testing, building, and developing applications in Visual Studio 2010 or Visual Studio 2012. Visual Studio Lab Management is integrated with System Center 2012 Virtual Machine Manager (SCVMM) to enable you manage multiple physical computers that host virtual machines and to manage the storage of virtual machines, virtual machine templates, and other configuration files in SCVMM library servers.
33
34
can store the environment in a snapshot and then restore the environment to its original state so that it can be used again. Reduce the time required to create and configure machines for testing an application. Using Lab Management stored virtual machines and templates; you can quickly deploy customized environments that recreate the configuration of customer's production environments. Run multiple copies of a test or development at the same time Using Lab Management stored virtual environments; you can deploy multiple copies of a virtual environment at the same time without the computer name conflicts that can limit the use of unmanaged virtual machines in a domain. Enable members of a team to create and manage virtual environments without requiring system administrator privileges. You can use Lab Management permissions to create role-based self- service groups of team members who can deploy, create and manage virtual environments without requiring system administrator privileges. You can also combine Lab Management permissions with Visual Studio Team Foundation Server and SCVMM permissions to meet the specific requirements of your organization.
35
36
LESSON 3:
Virtual Environments
You must understand some key concepts before you start to use Lab Management:
37
38
Snapshot A snapshot saves the state of your virtual machine at a specific point in time. You can revert your virtual machine to this snapshot whenever you want and then use this virtual machine from this known state when the snapshot was taken.
39
Lab Management
Virtual environment A collection of virtual machines that run on Hyper-V hosts, are managed by System Center Virtual Machine Manager, and which Lab Management manages as a logical unit. Each virtual machine that you add to the environment is used for a role in your application. For example, you might select a role of Web server for a virtual machine. Physical environment A collection of physical machines, each of which has a test agent installed. If you have virtual machines running on a non-Hyper-V platform, you can model them as physical machines in Lab Management. Lab Center The activity area in Microsoft Test Manager where you manage and operate the virtual machines and environments that are deployed on the SCVMM host groups of your team project. Team project library The activity area in Test Manager where you manage the stored virtual machines
40
A Testing Engineer can use either physical or virtual machines in creating and managing environments, testing, or build deployment.
41
Testing
Using Microsoft Test Manager, you can run tests on both virtual as well as physical environments. The capabilities are exactly the same here and this means that you can run automated and manual tests, and do rich diagnostic data collection from all sorts of environments Hyper-V, non-Hyper-V, and physical.
42
Build deployment
You can easily automate the build-deployment-testing workflow on Hyper-V virtual environments using the template shipped in TFS 2010 (labdefaulttemplate.xaml). For physical environments, such a template is not available out of the box. However, since this automation is based on Windows Workflow Foundation 4.0, you can customize the template (or create your own) using the activities in the toolbox, to automate build deployment and testing on physical machines. This means you can do the same for non-Hyper-V environments as well, however note that the virtual environment specific actions (restore to snapshot/start/stop environment) will not be available for physical and non-Hyper-V environments.
43
Lab Management
Lab Management works with both virtual environments hosted on Hyper-V and physical environments, but the capabilities of each type of environment are different. Virtual environments on non-Hyper-V hosts are treated the same as physical environments. The following table summarizes the different capabilities for each type of environment. Capability Virtual (Hyper-V) Physical Virtual (VMware)
Testing Run unit tests Run manual tests Run coded UI and other automated tests File rich bugs using diagnostic adapters Build deployment Automatic build-deploy-test workflow Environments creation and management Create environment from VM template Start/stop/snapshot environment Connect using Environment Viewer Clone environment using network isolation Supported Supported Supported Supported Supported Supported Supported Customization required Supported Supported Supported Customization required
Supported
44
Chapter 3
ANATOMY of a Coded UI Test
45
46
When you create a Coded UI Test in a testing project, several files are added to your solution. In this topic, we will use an example Coded UI Test to explore these files.
UIMap.cs CodedUITest1.cs
Yes Yes
UIMap.uitest
No
47
UIMap.Designer.cs This file contains code that is automatically created by the Coded UI Test Builder when a test is created. This file is re-created every time that a test changes, so that it is not a file in which you can add or modify code.
Declarations section
This section includes the following declarations for a Windows UI. C# using System; using System.CodeDom.Compiler; using System.Collections.Generic; using System.Drawing; using System.Text.RegularExpressions; using System.Windows.Input; using Microsoft.VisualStudio.TestTools.UITest.Extension; using Microsoft.VisualStudio.TestTools.UITesting; using Microsoft.VisualStudio.TestTools.UITesting.WinControls; using Microsoft.VisualStudio.TestTools.UnitTesting; using Keyboard = Microsoft.VisualStudio.TestTools.UITesting.Keyboard; using Mouse = Microsoft.VisualStudio.TestTools.UITesting.Mouse; using MouseButtons = System.Windows.Forms.MouseButtons; The Microsoft.VisualStudio.TestTools.UITesting.WinControls namespace is included for a Windows user interface (UI). For a Web page UI, the namespace would be Microsoft.VisualStudio.TestTools.UITesting.HtmlControls; for a Windows Presentation Foundation UI, the namespace would be Microsoft.VisualStudio.TestTools.UITesting.WpfControls.
UIMap class
The next section of the file is the UIMap class. [GeneratedCode("Coded UITest Builder", "10.0.21221.0")] public partial class UIMap The class code starts with a GeneratedCodeAttribute that is applied to the class, which is declared as a partial class. You will notice that the attribute is also applied to every class in this file. The other file that can contain more code for this class is UIMap.cs, which is discussed later. The generated UIMap class includes code for each method that was specified when the test was recorded.
48
public void LaunchCalculator() public void AddItems() public void VerifyTotal() public void CleanUp() This part of the UIMap class also includes the generated code for each property that is required by the methods. public virtual LaunchCalculatorParams LaunchCalculatorParams public virtual AddItemsParams AddItemsParams public virtual VerifyTotalExpectedValues VerifyTotalExpectedValues public virtual CalculateItemsParams CalculateItemsParams public virtual VerifyMathAppTotalExpectedValues VerifyMathAppTotalExpectedValues public UIStartMenuWindow UIStartMenuWindow public UIRunWindow UIRunWindow public UICalculatorWindow UICalculatorWindow public UIStartWindow UIStartWindow public UIMathApplicationWindow UIMathApplicationWindow
UIMap methods
Each method has a structure that resembles the AddItems() method. This is explained in more detail under the code, which is presented together with line breaks to add clarity. /// <summary> /// AddItems - Use 'AddItemsParams' to pass parameters into this method. /// </summary> public void AddItems() { #region Variable Declarations WinControl uICalculatorDialog = this.UICalculatorWindow.UICalculatorDialog; WinEdit uIItemEdit = this.UICalculatorWindow.UIItemWindow.UIItemEdit; #endregion // Type '{NumPad7}' in 'Calculator' Dialog Keyboard.SendKeys(uICalculatorDialog, this.AddItemsParams.UICalculatorDialogSendKeys, ModifierKeys.None); // Type '{Add}{NumPad2}{Enter}' in 'Unknown Name' text box
49
Keyboard.SendKeys(uIItemEdit, this.AddItemsParams.UIItemEditSendKeys, ModifierKeys.None); } The summary comment for each method definition tells which class to use for parameter values for that method. In this case, it is the AddItemsParams class, which is defined later in the UIMap.cs file, and which is also the value type that is returned by the AddItemsParams property. At the top of the method code is a Variable Declarations region that defines local variables for the UI objects that will be used by the method. In this method, both UIItemWindow and UIItemEdit are properties that are accessed by using the UICalculatorWindow class, which is defined later in the UIMap.cs file. Next are lines that send text from the keyboard to the Calculator application by using properties of the AddItemsParams object. The VerifyTotal() method has a very similar structure, and includes the following assertion code. // Verify that 'Unknown Name' text box's property 'Text' equals '9. ' Assert.AreEqual( this.VerifyTotalExpectedValues.UIItemEditText, uIItemEdit.Text); The text box name is listed as unknown because the developer of the Windows Calculator application did not provide a publicly available name for the control. The Assert.AreEqual method fails when the actual value is not equal to the expected value, which would cause the test to fail. Also notice that the expected value includes a decimal point that is followed by a space. If you ever have to modify the functionality of this particular test, you must allow for that decimal point and the space.
UIMap properties
The code for each property is also very standard throughout the class. The following code for the AddItemsParams property is used in the AddItems() method. public virtual AddItemsParams AddItemsParams { get { if ((this.mAddItemsParams == null)) { this.mAddItemsParams = new AddItemsParams(); }
50
return this.mAddItemsParams; } } Notice that the property uses a private local variable that is named mAddItemsParams to hold the value before it returns it. The property name and the class name for the object it returns are the same. The class is defined later in the UIMap.cs file. Each class that is returned by a property is structured similarly. The following is the AddItemsParams class. /// <summary> /// Parameters to be passed into 'AddItems' /// </summary> [GeneratedCode("Coded UITest Builder", "10.0.21221.0")] public class AddItemsParams { #region Fields /// <summary> /// Type '{NumPad7}' in 'Calculator' Dialog /// </summary> public string UICalculatorDialogSendKeys = "{NumPad7}"; /// <summary> /// Type '{Add}{NumPad2}{Enter}' in 'Unknown Name' text box /// </summary> public string UIItemEditSendKeys = "{Add}{NumPad2}{Enter}"; #endregion } As with all classes in the UIMap.cs file, this class starts with the GeneratedCodeAttribute. In this small class is a Fields region that defines the strings to use as parameters for the Keyboard.SendKeys method that is used in the UIMap.AddItems() method that was discussed earlier. You can write code to replace the values in these string fields before the method in which these parameters are used is called.
UIMap.cs
By default, this file contains a partial UIMap class that has no methods or properties.
UIMap class
51
This is where you can create custom code to extend the functionality of the UIMap class. The code that you create in this file will not be regenerated by the Coded UI Test Builder every time that a test is modified. All parts of the UIMap can use the methods and properties from any other part of the UIMap class.
CodedUITest1.cs
This file is generated by the Coded UI Test Builder, but is not re-created every time that the test is modified, so that you can modify the code in this file. The name of the file is generated from the name that you specified for the test when you created it.
CodedUITest1 class
By default, this file contains the definition for only one class. [CodedUITest] public class CodedUITest1 The T:Microsoft.VisualStudio.TestTools.UITesting.CodedUITestAttribute is automatically applied to the class, which allows the testing framework to recognize it as a testing extension. Also notice that this is not a partial class. All class code is contained in this file.
CodedUITest1 properties
The class contains two default properties that are located at the bottom of the file. They must not be modified. /// <summary> /// Gets or sets the test context which provides /// information about and functionality for the current test run. ///</summary> public TestContext TestContext public UIMap UIMap
CodedUITest1 methods
By default, the class contains only one method. public void CodedUITestMethod1()
52
This method calls each UIMap method that you specified when you recorded your test, which is described in the section on the UIMap Class. A region that is titled Additional test attributes, if uncommented, contains two optional methods. // Use TestInitialize to run code before running each test [TestInitialize()] public void MyTestInitialize() { // To generate code for this test, select "Generate Code for Coded // UI Test" from the shortcut menu and select one of the menu items. // For more information on generated code, see // http://go.microsoft.com/fwlink/?LinkId=179463 // You could move this line from the CodedUITestMethod1() method this.UIMap.LaunchCalculator(); } // Use TestCleanup to run code after each test has run [TestCleanup()] public void MyTestCleanup() { // To generate code for this test, select "Generate Code for Coded // UI Test" from the shortcut menu and select one of the menu items. // For more information on generated code, see // http://go.microsoft.com/fwlink/?LinkId=179463 // You could move this line from the CodedUITestMethod1() method this.UIMap.CloseCalculator(); } The MyTestInitialize() method has the TestInitializeAttribute applied to it, which tells the testing framework to call this method before any other test methods. Similarly, the MyTestCleanup() method has the TestCleanupAttribute applied to it, which tells the testing framework to call this method after all other test methods have been called. Use of these methods is optional. For this test, the UIMap.LaunchCalculator() method could be called from MyTestInitialize() and the UIMap.CloseCalculator() method could be called from MyTestCleanup() instead of from CodedUITest1Method1(). If you add more methods to this class by using the CodedUITestAttribute, the testing framework will call each method as part of the test.
53
UIMap.uitest
This is an XML file that represents the structure of the coded UI test recording and all its parts. These include the actions and the classes in addition to the methods and properties of those classes. The UIMap.Designer.cs file contains the code that is generated by the Coded UI Builder to reproduce the structure of the test and provides the connection to the testing framework. The UIMap.uitest file is not directly editable. However, you can use the Coded UI Builder to modify the test, which automatically modifies the UIMap.uitest file and the UIMap.Designer.cs file.
54
55
Chapter 4
Automated Test
56
57
58
59
Used as part of load tests to test server responses for Web applications by sending and receiving http requests. Test functionality using API calls or command line tools for the application under test.
Yes
Yes
Not recommended because Web Performance tests are primarily intended to be used with load tests. Yes
Depending on what is need to be accomplish with UI testing will determine which types of automated tests you need to create. When you have created these tests, then you can decide if you want to report the results as part of a test plan. The tasks in the table below can help you determine your automated testing needs.
60
Testing Methods
Use Cases or User Stories After reviewing the users' needs for your application, you can create test cases to cover testing these needs. You can connect each test case to the relevant requirements, use cases or user stories, depending on your project management methodology. You can then associate an automated test with your test case. This links your automated tests to your requirements, use cases, or user stories. How to: Add Requirements or User Stories to Your Test Plan Creating a Test Plan Using Requirements or User Stories How to: Associate an Automated Test with a Test Case Speed Up Testing User Interface You can create automated user interface tests for your application. These tests can be used to check that there has been no regression or change in the user interface from coding changes. Testing the User Interface with Automated UI Tests
61
Improving Code Quality Using Unit Tests You can automatically generate unit tests from your existing code, or you can create unit tests manually. Unit tests will help improve the quality of your code by ensuring that changes you make do not cause issues in existing functionality. Verifying Code by Using Unit Tests Test How Your Application Performs Under Heavy Loads You can create either unit tests or Web Performance tests and use these in load tests. With a load test, you can create a heavy load on your application by simulating multiple users. Then you can measure the performance of your application and check it can still run successfully with a large number of users accessing it. Testing Application Performance and Stress
Test the Functionality of the Application If you can access the functionality of your application using API calls or a command line interface, you can create automated tests to drive your application without using the user interface. Creating an Automated Test That Runs an Executable Using Generic Tests Improving Database Code Quality Using Database Unit Tests: You can create database unit tests to verify that the stored procedures, functions and triggers in the database are working correctly. These tests can be run to check that changes to the database or database code do not affect existing database code. Verifying Database Code by Using Unit Tests
Converting Manual Test Cases to Automated Tests to Run As Part of a Test Plan You can convert manual test cases to automated tests by adding the details for the automated test to the test case using Visual Studio. Or you can create a test case and add an automated test if you want to run these tests as part of a test plan using Microsoft Test Manager. How to: Associate an Automated Test with a Test Case Defining Your Testing Effort Using Test Plans
Managing Your Automated Tests You can organize automated tests by marking test methods with a test category attribute. You can then use these groupings to select the tests that you want to run. Or you can convert a test
62
case into an automated test and manage these tests using test plans and suites. You can also use test lists to manage your automated tests. Defining Test Categories to Group Your Tests Organizing Test Cases Using Test Suites Defining Test Lists to Group Your Tests
When the tests are run as part of a test plan, you select test settings and environments for your test plan as shown in the following illustration. You configure a test plan using the Properties view in the Plan section of the Testing Center.
63
64
on these test results for the test and configuration pairings, as shown in the following illustration. Create automated test settings for your test plan with a set of roles that match your physical environment. Add the test settings and environment to your test plan. Create Test Settings for Automated Tests as Part of a Test Plan
Create automated tests using Microsoft Visual Studio 2010. These tests might be unit tests or coded UI tests. Creating and Running Unit Tests for Existing Code Testing the User Interface with Automated UI Tests
65
66
Then you can associate the test method with your test case as shown in the following illustration.
If you need to create a new test case, then you can do this from the Test View window as described in the following procedure Create a Test Case From an Automated test. You must then add this test case to a test plan to be able to run it.
67
Note You can view the information from Microsoft Test Manager, but you cannot modify it.
Testing Scenarios
The following table shows the types of automated tests that you can create and which types of tests can be run as part of a test plan: Type of test Description Tests the user interface by performing UI actions. Tests code at the method level. Tests a stored procedure, function or trigger in a database. Tests application performance and stress using unit tests, Web Performance tests or coded UI tests. Used as part of load tests to test server responses for Web applications by sending and receiving http requests. Test functionality using API calls or command line tools for the application under test. Run from a test plan by associating a test method with a test case Yes Yes Yes Not recommended because you cannot view the test as it runs.
Not recommended because Web Performance tests are primarily intended to be used with load tests. Yes
Use the following procedures to associate an automated test with a test case that you can then run as part of your test plan:
Add the automated test to your existing test case Create a test case from an automated test Set up your test plan to use your build Create your test settings and environment to run your tests Run the automated test using Microsoft Test Manager View and update the test results
68
When you have defined all these, you are ready for testing. When requirements or user stories, or features are ready to be tested, you can run your tests for each configuration that you specified. This plan enables you to measure your progress as you run your tests and report on how much testing remains. You can run manual tests from Microsoft Test Manager using the Microsoft Test Runner. You can also run automated tests from Microsoft Test Manager if the automation is associated with a test case. Results from running these tests will be associated with a test plan.
69
In addition, you can run automated tests from Visual Studio that are not associated with a plan. You can select to run the tests individually, as part of a check-in policy, or based on test categories. They can also be run as part of a build created by using Team Foundation Build, and from the command line. Because the testing tools are integrated with the other parts of Visual Studio Ultimate, you can save your test results to a database, generate trend and historical reports, and compare different kinds of data. For example, you might use the data to see how many and which bugs were found by your tests.
Submitting bugs
You can submit bugs with the detailed information that you collect when you run your tests using Test Runner. You can also submit bugs from Visual Studio or Microsoft Test Manager.
70
EXERCISE:
1. To open the solution that contains the automated test, click File and point to Open and then click Project/Solution. The Open Project dialog box is displayed. 2. Select your solution, and then click Open. Note You must open your solution to be able to associate the automation with the test case. 3. If you know the work item of the test case, in Team Explorer, on the Team menu, click Go To Work Item. You can also right-click the Work Items node for your team project. 4. In the Go to Work Item dialog box, in the ID box, type the ID number of your test case. 5. Click OK to open the work item. Note You can also use a pre-existing query in the Team Queries or My Queries node to find the work item if you do not know the ID. For more information about how to use queries in Team Explorer, see Finding Bugs, Tasks, and Other Work Items. 6. To add the automated test, click the Associated Automation tab. 7. To find the automated test name, click the ellipsis (). The Choose Test dialog box is displayed. All the tests in the solution are shown in the list together with their associated test projects. Note If a test case already has an automated test associated with it, you must first remove this association before you can add a different automated test. Click Remove association to remove the existing automation. 8. Select the test, and then click OK. Note The value in Automation Status is automatically changed to Automated 9. To save the changes to the test case, click Save Work Item. The next part of the process is to set up your test plan to use your build.
71
1.
2.
3.
4. 5.
6.
72
EXERCISE 2:
1. Open Microsoft Test Manager. Note To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010. 2. To select a test plan, click the down-arrow on the center group switcher and then click Testing Center. 3. On the center group menu bar, click Plan. 4. To set up your test plan to run the automated tests, click Properties and then click the drop-down arrow to the right of Filter for builds. The dialog box that shows build definition and quality is displayed. 5. To select the build definition that is used to build your automated tests, click Build definition. 6. Each build can be given a specific value to reflect the quality of the build. To select the quality of the builds you want to be able to view, click Build quality. 7. To save your changes, click Set build filter. 8. To select the most recent build to use with this test plan that includes the latest changes to the automated test, you must first click Save to save the plan and then click Modify. The Assign Build activity is displayed. You can compare your current build with a build you plan to take. The associated items list shows the changes to work items between the builds. You can then assign the latest build to take and use for testing with this plan. For more information about how to select a build, see Determining Which Builds Have Bug Fixes, New Features, or Requirements. 9. To close the Assign Build activity and return to the test plan properties, click the Close icon. 10. To save these changes for this test plan, click Save in the toolbar.
73
EXERCISE 3:
To run your automated tests, you must use a physical or virtual environment. You cannot run automated tests using Microsoft Test Manager without an environment. You must create an environment that contains the roles in your test settings and then use this environment in your test plan. For more information about how to create your environment and roles and test settings, see Setting Up Test Machines to Run Tests or Collect Data. Note If you want to run automated tests that interact with the desktop, you must set up your agent to run as a process instead of a service. For more information, see How to: Set Up Your Test Agent to Run Tests that Interact with the Desktop. 1. To create a physical environment, follow the steps in this topic: Creating a Physical Environment to Use for Testing. 2. If you are using Visual Studio Lab Management, you can create a virtual environment. To create a virtual environment, follow the steps in this topic: How to: Create an Environment from Virtual Machines or Templates. If you want more information about how to create virtual environments, see Virtual Environments Concepts and Guidelines. 3. To create your test settings, follow the steps in this topic: Create Test Settings for Automated Tests as Part of a Test Plan.
74
EXERCISE 4:
1. Open Microsoft Test Manager. Note To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010. 2. To run the automated test, click the down-arrow on the center group switcher and then click Testing Center. 3. On the center group menu bar, click Test. 4. (Optional) To override the build, the test settings or the environment to use for running the automated tests that you select in this test plan, right-click the test and then click Run with options. For example, if you want to run on a staging environment instead of your standard testing environment then you might select a different environment. From the Run options dialog box, you can change these settings, and then click Run to run the selected test. 5. To run the automated test without changing any options, right-click the test and then click Run. The Analyze Test Runs activity is displayed. It shows the progress of the test run that contains this test. The test result is displayed. It shows the details from the test run, the attachments for data collected for this test result, and the test results history for that test. You can close this view to return to the test run. 1. To save these changes for this test run, click Save in the toolbar.
75
Create a test project when you generate a unit test. When you generate unit tests, you can create a new test project into which the tests will be added, or add them to an existing test project. Using the Add New Project dialog box. With this method, you use a dialog box to choose the programming language of your test project. Configure test project creation. Set the programming language for new test projects, and whether one or more tests should be added to new test projects by default.
76
77
78
LAB 3: Create a test project by using the Add New Project dialog box
To create a test project by using the Add New Project dialog box 1. In Solution Explorer, right-click the solution. 2. Point to Add and then click New Project. The Add New Project dialog box is displayed. The Project types pane contains a Test Projects node and nodes for specific programming languages. 3. If you want your new test project to have a specific programming language, expand the node of that language. If not, expand Test Projects. Note Test projects are not available in every language. 4. If you expanded a language node, click Test. If you expanded the Test Projects node, click Test Documents. 5. Under Templates, click Test Project. 6. (Optional) Change the name or location of the project that you are about to add. 7. Click OK. The test project is added to your solution.
79
80
81
Chapter 5
Tracking Software Quality
82
As a Tester you build out dashboards to show our progress on our software test projects, we quickly enjoy the luxury of Dashboard Views that utilize Windows SharePoint 2010 technology. This new web part connects to a URL on the TFS application site tier called tsfredirect.aspx that then will load a report from the report site. A tester can use Visual Studio Test Professional 2010 or Visual Studio 2010 Ultimate to help you track the quality of the applications that you are testing using Microsoft Test Manager. You can check which builds have bug fixes, new features, or new requirements or user stories added. You can also view recommendations about which tests might have to be rerun, based on the effect of code changes that have been made to your application if you are using test impact analysis when you run your tests. You can track bugs found in your application under test. You can assess your software quality, based on these bugs. You can triage these bugs to assess priorities, assign the bugs and determine whether the bugs can be fixed. It is also possible that the bugs need more information, or the bug is actually intended behavior for the application. In addition, you can view reports on the readiness of your test cases, view the progress that you are making in your test plan, and run queries to review specific data for your team project that relates to requirements or user stories, test cases and bugs. If your team uses a project portal, you can view the predefined reports on the Test Dashboard. You can access the project portal from the Track view in the Testing Center for Microsoft Test Manager. For more information about the Test Dashboard, see Test Dashboard (Agile).
83
Agile teams can quickly find important information about their team projects by using dashboards. Dashboards show project data, support investigation, and help teams perform common tasks more quickly.
84
Client Counters Page Size System Counter Machine's available memory Hard Page faults Processor Utilization Heap size of managed memory
using (new SPMonitoredScope("MyTestScope")) { //Code which needs to be monitored goes here } Another approach implement ISPScopedPerformanceMonitor Interface in your custom class and use it. Once the developer dash board is turned on the information regarding execution of queries, function calls used in rendering the page; to enable the developer dashboard through the object model the Tester must create a new application to trigger the following code: SPWebService cs = SPWebService.ContentService; cs.DeveloperDashboardSettings.DisplayLevel = SPDeveloperDashboardLevel.On; cs.DeveloperDashboardSettings.Update();
85
SPDeveloperDashboardLevel has three set of dashboard levels by default: 1. ON - Developer dash board will always be active on the SharePoint farm
3. ON Demand - these options lets the end user activate and deactivate the developer dash board based on the choice of the developer/tester/end-user.
The last one is the most preferred option as usually people would not like the developer dashboard taking extra screen space on the SharePoint 2010 pages. When the On Demand option is set through the code then a small icon appears across the right corner of the SharePoint site pages and using that the Developer Dashboard can be set to on or off.
STSADM Utility
The following code example shows how a Testing Developer might enable the Developer Dashboard through STSADM Utility: Stsadm o setproperty pn developer-dashboard pv ondemand (or on or off)
86
Developer Dashboard is an excellent feature for Automated UI Test in SharePoint 2010 Development. Though Developer Dashboards cannot trace the events of a custom Sandboxed solution but still provides ample information to boost the performance of SharePoint pages.
PowerShell Commands
PowerShell Commands for Dashboards in SharePoint 2010 Dashboard Developer. $DevDashboardSetting = [Microsoft.SharePoint.Administration.SPWebservice}::ContentService.DeveloperDashbo ardSettings; $DevDashboardSettings.DisplayLevel = OnDemand; $DevDashboardSettings.RequiredPermissions = EmptyMask; $DevDashboardSettings.TraceEnable = $true; $DevDashboardSettings.Update()
87
LAB 1: Create the Portal in SharePoint and then fix up the missing documents
EXERCISE 1: Create new portal site
1. 2. Go to the SharePoint site of your project collection)
3. Click on the Site Actions at the left side of the screen and choose the option Site Settings
4. In the site settings, choose the Sites and workspaces option 5. Create a new site 6. Enter the values for the Title, the description, the site address. And choose for the TFS2010 Agile Dashboard as template. 7. Create the site, by clicking on the Create button
88
5. Check the Enable team project portal checkbox 6. Click on Configure URL.. . 7. You will get a new dialog as below 8. Enter the url to the TFS server in the web application ComboBox 9. And specify the relative site path: sites/<project collection>/<site name>
89
10. Confirm with OK 11. Check in the Project Portal Settings dialog the checkbox "Reports and dashboards refer to data for this team project" 12. Confirm the settings with OK (this takes a while...) 13. When you now browse to the portal, you will see that the dashboards are now showing up with the data for the current team project. We will fix that in the next step.
90
1. Go to the Team Explorer in Visual Studio 2. Make sure the team project is in the list of team projects, and expand the team project 3. Right click the Documents node, and choose New Document Library
5. Right click the Shared Documents node and choose Upload Document 6. Go the the file location where you stored the process template from step 3 and then navigate to the subdirectory "Agile Process Template 5.0\MSF for Agile Software Development v5.0\Windows SharePoint Services\Shared Documents\Project Management" 7. Select in the Open Dialog the files "Iteration Backlog" and "Product Backlog", and click Open
91
comes from the table created in the previous step. As you collect these properties, copy them into notepad so they can be used in later steps.
How to retrieve the value? Switch to the Design ribbon and select the Table Name value in the Properties portion of the ribbon In the Visual Studio Team Explorer, right click your Team Project and select Properties. Select the URL value and copy the GUID (long value with lots of characters) at the end of the URL In the Properties dialog, select the Name field and copy the value In the Properties dialog, select the Server Name field and copy the value [UPDATE] I have found that this is not correct: you need to specify the instance of your SQL Server. The value is used to create a connection to the TFS cube.
7. Switch back to the Iteration Backlog workbook. 8. Click the Office button and select Prepare Properties. 9. Click the Document Properties Server drop down and select Advanced Properties. 10. Switch to the Custom tab and add the following properties using the values you collected above. Variable name [Table name]_ASServerName [Table name]_ASDatabase [Table name]_TeamProjectName [Table name]_TeamProjectId 11. Click OK to close the properties dialog. 12. It is possible that the Estimated Work (Hours) is showing the #REF! value. To resolve that change the formula with: =SUMIFS([Table name][Original Estimate]; [Table name][Iteration Path];CurrentIteration&"*";[Table name][Area Path];AreaPath&"*";[Table name][Work Item Type]; "Task") For example Value [TFS server name] tfs_warehouse [Team Project name] [Project GUID]
92
=SUMIFS(VSTS_ab392b55_6647_439a_bae4_8c66e908bc0d[Original Estimate]; VSTS_ab392b55_6647_439a_bae4_8c66e908bc0d[Iteration Path];CurrentIteration&"*";VSTS_ab392b55_6647_439a_bae4_8c66e908bc0d[Area Path];AreaPath&"*";VSTS_ab392b55_6647_439a_bae4_8c66e908bc0d[Work Item Type]; "Task") 13. Also the Total Remaining Work in the Individual Capacity table may contain #REF! values. To resolve that change the formula with: =SUMIFS([Table name][Remaining Work]; [Table name][Iteration Path];CurrentIteration&"*";[Table name][Area Path];AreaPath&"*";[Table name][Assigned To];[Team Member];[Table name][Work Item Type]; "Task") For example =SUMIFS(VSTS_ab392b55_6647_439a_bae4_8c66e908bc0d[Remaining Work]; VSTS_ab392b55_6647_439a_bae4_8c66e908bc0d[Iteration Path];CurrentIteration&"*";VSTS_ab392b55_6647_439a_bae4_8c66e908bc0d[Area Path];AreaPath&"*";VSTS_ab392b55_6647_439a_bae4_8c66e908bc0d[Assigned To];[Team Member];VSTS_ab392b55_6647_439a_bae4_8c66e908bc0d[Work Item Type]; "Task") 14. Save and close the workbook.
93
Chapter 6
Create a Coded UI Test
94
Introduction
Using Visual Studio Premium or Visual Studio Ultimate, you can create a coded UI test that can test that the user interface for an application functions correctly. The coded UI test performs actions on the user interface controls for an application and verifies that the correct controls are displayed with the correct values. For more information about which platforms and configurations are supported by coded UI tests, see Supported Configurations and Platforms for Coded UI Tests and Action Recordings. Creating a coded UI test generates a UIMap object that is specific to your test and represents the windows, controls, parameters, and assertions that are in the UI or that you created during the test recording. You can then perform actions on these UI objects to automate your user interface. For example, you can have your test method click a hyperlink in a Web application, type a value in a text box, or branch off and take different testing actions based on a value in a field. Note You can add multiple coded UI tests and multiple UI map objects and files to facilitate testing a large application. For more information, see Testing a Large Application with Multiple UI Maps. A coded UI test class is identified by a CodedUITestAttribute applied to the class. Each coded UI test is a test method in a coded UI test class. You can add multiple test methods to each coded UI test class and identify each coded UI test method by using the TestMethodAttribute. Your test method can also add validation code for a UI test control to obtain the value of a property of a UI test control. The test method can use an Assert statement to compare the actual value of the property to an expected value. The result of this comparison determines the outcome of the test. Every time that you run a coded UI test, you can analyze the test result and if the test fails, you can see or store the details of which assertion failed. When you create a coded UI test, these files are added to your test project: File CodedUITest1.cs UIMap.uitest Description Contains the coded UI test class, test methods and assertions. Contains the XML model for the UIMap class, including all windows, controls, properties, methods, parameters, actions, and assertions. UIMap.Designer.cs Contains the code representation of the XML contained in the UIMap.uitest file. Do not edit this file. UIMap.cs Contains more of the code for the UIMap class. You can put any customizations for the UI map in this file.
95
96
97
Coding UI Test
The subsequent code example illustrates a coded UI test class and test method that is assigned a test category value of Priority1. The test method is for a simple calculator application that adds two numbers and verifies that they are added together correctly for this test to pass. The code for the two UI map methods, AddTwoNumbers() and AssertForAdd(), are in the noneditable portion of the UI map and cannot be edited directly. However, you can copy the generated code from the UI map to modify it and create alternate or additional methods in the MyCodedUITest class. See module on Best Activities for Coded UI Tests covered later in this book. After you create your coded UI test with specific data, you might want to run your coded UI test several times with different sets of data to test different conditions. To do this you can add parameters from a data source to your coded UI test to create a data-driven coded UI test. This topic will be covered in a later module Creating a Data-Driven Coded UI Test [CodedUITest] public class MyCodedUITest { public CodedUITest() { } [TestCategory("Priority1"), TestMethod] public void CodedUITestMethod1() { // To generate code for this test, select "Generate Code" from // the shortcut menu and select one of the menu items. this.UIMap.AddTwoNumbers(); this.UIMap.AssertForAdd(); } }
98
99
Adds the controls and to your UI map (UIMap.uitest) Adds a method called AddTwoNumbers to your UI map. You can view the method in the UIMap.Designer.cs file. This method performs the actions that you recorded when you run the test. public void AddTwoNumbers() { #region Variable Declarations WinEdit textInput1Edit = this.DemoCalculatorWindowWindow.InputNumber2Window.TextInput1Edit; WinEdit textInput2Edit = this.DemoCalculatorWindowWindow.TextInput2Window.TextInput2Edit; WinButton addButton = this.DemoCalculatorWindowWindow.AddWindow.AddButton; #endregion // Launch '%USERPROFILE%\Desktop\SimpleWinformsCalculator.exe' ApplicationUnderTest demoCalculatorWindowWindow = ApplicationUnderTest.Launch( this.AddTwoNumbersParams.DemoCalculatorWindowWindowExePath, this.AddTwoNumbersParams.DemoCalculatorWindowWindowAlternateExePath); // Type '3' in 'textInput1' text box textInput1Edit.Text = this.AddTwoNumbersParams.TextInput1EditText; // Type '4' in 'textInput2' text box textInput2Edit.Text = this.AddTwoNumbersParams.TextInput2EditText; // Click 'Add' button Mouse.Click(addButton, new Point(83, 18));
} Adds a test method to your coded UI test file that calls your AddTwoNumbers method [TestMethod] public void CodedUITestMethod1() { // To generate code for this test, select "Generate Code" // from the shortcut menu and select one of the menu items. this.UIMap.AddTwoNumbers(); } When you choose a name for the recorded method, choose a name that is descriptive for the actions that you recorded. 7. If the actions that you recorded were not what you want to keep, you can click the Show Recorded Steps icon. Select the actions that you do not want to keep and click the Delete icon.
100
8. To generate assertions for your UI controls, click the crosshairs icon and drag it to the control that you want to verify is correct. The Coded UI Test Builder - Add Assertions dialog box is displayed. 9. Click the vertical bar to view the UI map. The UI control that you want to verify should be highlighted. 10. Right-click the property for the UI control that you want to verify, and then point to Add Assertion. The Add assertion dialog box is displayed. 11. Select the Comparator for your assertion. 12. Type the value for your assertion in Comparison Value. 13. To add the assertion, click OK. 14. When you have added all your assertions for your test, close the Coded UI Test Builder Add Assertions dialog box. 15. To generate the code for your assertions, click the Generate Code icon. The Coded UI Test Builder - Generate Code dialog box is displayed. 16. Type a name for your coded UI test method in Method Name, and then click Add and Generate. This generates code as follows if the name you entered is, for example, AssertForAddTwoNumbers:
Adds a method called AssertForAddTwoNumbers to your UI map (UIMap.uitest). You can view the method in the UIMap.Designer.cs file. This method performs the assert statements that you added. public void AssertForAddTwoNumbers() { #region Variable Declarations WinEdit textAnswerEdit = this.DemoCalculatorWindowWindow.AnswerWindow.TextAnswerEdit; #endregion // Verify that the 'textAnswer' text box's Text property // is '40' Assert.AreEqual( this.AssertForAddTwoNumbersExpectedValues.TextAnswerEditText, textAnswerEdit.Text);
101
} Adds a call to the assert method AssertForAddTwoNumbers to the test method in your coded UI test file [TestMethod] public void CodedUITestMethod1() { // To generate code for this test, select "Generate Code" // from the shortcut menu and select one of the menu items. this.UIMap.AddTwoNumbers(); this.UIMap.AssertForAddTwoNumbers(); }
When you choose a name for the method that has your assert statements, choose a name that is descriptive for these assertions that you created. 17. Click the Close icon to close the Coded UI Test Builder. 18. (Optional) To add code to start your application when the coded UI test runs, record the startup sequence for your application and save that to a method. You can call that method at the start of your test. Note You can add a test initialize method, identified by a [TestInitialize] attribute, which runs code at the start of each test method. For example, the method to start the application could be called from the TestInitialize method. 19. (Optional) To add code to close your application when the coded UI test runs, record the closing sequence for your application and save that to a method. If you do not close the browser or application, it will remain open after your test is finished. Note You can add a test cleanup method, identified by a [TestCleanup] attribute, which runs code at the end of each test method. For example, the method to close the application could be called from the TestCleanup method. 20. To run the test, right-click in the test method, and then click Run Tests. For more information about how to run coded UI tests, see Running Automated Tests. Note After you create your coded UI test with specific data, you might want to run your coded UI test several times with different sets of data to test different conditions. To do this you can add parameters from a data source to your coded UI test to create a datadriven coded UI test. For more information, see How to: Create a Data-Driven Coded UI Test.
102
21. You can now optionally add additional controls and validate them using the UI Test Builder. For more information, see How to: Add UI Controls and Validation Code Using the Coded UI Test Builder.
103
Chapter 7
UI Controls and Validation Code Using the Coded UI Test Builder
104
You can use the Coded UI Test Builder to add a user interface (UI) control to the UIMap for your test, or to generate code for a validation method that uses an assertion for a UI control. Add UI Controls to an Existing Coded UI Test You can add a UI control for the application under test to an existing UI map and generate code to add the control to your coded UI test. To add UI a control to a coded UI Test 1. In Solution Explorer, open a coded UI test source file from a test project in your solution. The file name is CodedUITest1.cs). This is the same as the test name. 2. To start the Coded UI Test Builder, right-click in the CodedUITestMethod1() method, select the Generate Code for Coded UI Test command. In the shortcut menu, select the Use Coded UI Test Builder command. The Coded UI Test Builder dialog box appears. 3. Start your application under test and navigate to the UI control that you want to add. 4. Drag the crosshairs button in the Coded UI Test Builder to the UI control in your application that you want to test. When the box outlines your control, release the mouse. The control class code is immediately created in the UIMap.Designer.cs file.
The properties for this control are now listed in the Coded UI Test Builder - Add Assertions dialog box. To see the UI controls in the map, click the arrow (<<) to expand the view for the UI control map. To find a parent, sibling, or child control, you can click on the map and use the arrow keys to move around the list of controls.
105
Note You can also locate a UI control by pressing the Windows logo key + I. This is useful for testing a shortcut menu where the control is dismissed when you change the focus to the Coded UI Test Builder. For more information, see How to: Select a Control Using the Keyboard in Coded UI Tests. 5. Close the Coded UI Test Builder or go to the next procedure to add validation assertions. 6. To run the test, right-click in the test method, and then click Run Tests. For more information about how to use UI control actions and set properties, see How to: Use UI Control Actions and Properties in Your Coded UI Tests. Verify UI Controls in a Coded UI Test After you add a UI control for the application under test to the UI map in your coded UI test, then you can use the Coded UI Test Builder to create a validation method to validate properties of the added UI control. To add and verify controls in coded UI Test 1. In Solution Explorer, open a coded UI test source file from a test project in your solution. 2. To start the Coded UI Test Builder, right-click in the CodedUITestMethod1() method, select the Generate Code for Coded UI Test command. In the shortcut menu, select the Use Coded UI Test Builder command. The Coded UI Test Builder dialog box appears. 3. Start your application under test and navigate to the UI control that you want to add.
106
4. To add a UI control to the UI map, drag the crosshairs to the UI control in your application that you want to test. When the box outlines your control, release the mouse. The properties for this control are now listed in Coded UI Test Builder - Add Assertions dialog box. To locate the UI control in the map, click the arrow to view the UI control map. Note You can also locate a UI control by pressing the Windows key + I. This is useful for testing a shortcut menu where the control is dismissed when you change the focus to the Coded UI Test Builder. To find a parent, sibling, or child control, you can use the arrows in the dialog box. 5. Click the Add control to UI Control Map button in the toolbar to add this control to the UI map. 6. Right-click the property that you want to verify, and then right-click and select the Add Assertion command. The Add Assertion dialog box appears.
7. Select the Comparator for your assertion: AreEqual. 8. Type the value for your assertion in Comparison Value and click OK to add it . 9. When you have added all your assertions for your test, close the Coded UI Test Builder Add Assertions dialog box. 10. To generate the code for your assertions and add the control to the UI map, click the Generate Code icon. The Coded UI Test Builder - Generate Code dialog box appears. 11. Type a name for your coded UI test method in Method Name, then click Generate. For more information about the anatomy of the generated code, see Anatomy of a Coded UI Test. 12. Click the Close icon to close the Coded UI Test Builder.
107
13. To run the test, right-click in the test method, and then click Run Tests. .Note After you create your coded UI test with specific data, you might want to run your coded UI test several times with different sets of data to test different conditions. To do this you can add parameters from a data source to your coded UI test to create a datadriven coded UI test. For more information, see How to: Create a Data-Driven Coded UI Test.
108
Chapter 8
Data-Driven Coded UI Test
109
Using Visual Studio Premium or Visual Studio Ultimate, you can create a coded UI test that tests whether the user interface (UI) for an application functions correctly. The coded UI test performs actions on the user interface controls for an application and verifies that specified controls display the correct values. For more information about how to create a coded UI test, see How to: Create a Coded UI Test. After you create your coded UI test with specified data, you might want to run your coded UI test multiple times with different sets of data to test different conditions. To do this you can add parameter values from a data source to create a data-driven coded UI test. Each row in the data source is an iteration of the coded UI test. The overall result of the test will be based on the outcome for all the iterations. For example, if one test iteration fails, the overall test result is failure. When you generate methods or assertions for your coded UI test by recording them, all constants referred to in the recorded methods are parameterized into parameter classes. For example, if you add a method named AddTwoNumbers, the coded UI testing framework generates another class named AddTwoNumbersParams and adds a field to this class for each constant value that you used when recording the method. If you generate an assertion method named AssertForAdd, the framework generates a class named AssertforAddExpectedValues for the expected values. These classes are generated automatically and are used in the following steps. All changes to your test to enable it to use a data source for the parameters will be made to your test's source code file (i.e. CodedUITest1.cs). You cannot modify the code in the UIMap.Designer.cs file. After you have created a coded UI test, you can use the steps in the following procedure to add your data source and parameters to your test. This example assumes that you have these code elements in your solution:
The test method is for a simple calculator application that adds two numbers and verifies that they are added together correctly for this test to pass. The data source is a .csv file that contains the following data: Input1 3 5 1200 Input2 4 6 24000 ExpectedResult 7 11 25200
110
To test the functionality of any application, we test it with different identified sets of data to make sure the functionality does not break while iterating through all these data sets. Coded UI supports 3 different datasources that can be used for providing the test data. 1. 2. 3. CSV files Xml files Databases
2. Choose the XML file from the next screen. 3. You will see the data present in XML file as below:
111
4. Clicking Next button will show up a screen with different data tables.
If you have a look at the xml file which I used as the source of test data, it contains two different data tables named Search and Results as shown below:
5. Click on the Finish button to complete the configuration. Clicking on Finish will show a pop up to copy the database file into the current project.
112
6. Click on Yes button and it will copy the XML file in the current project.
Next all steps will be same as discussed in my pervious article. I am reiterating the steps again. 6. Open the CodedUITestMethod1 and notice the attribute on top of the method: [DeploymentItem("SampleCodedUIProject\\XMLFile1.xml"), DataSource("Microsoft.VisualStudio.TestTools.DataSource.XML", "|DataDirectory|\\XMLFile1.xml", "Search", DataAccessMethod.Sequential), TestMethod] public void CodedUITestMethod1() { string testData = TestContext.DataRow["SearchData"].ToString(); this.UIMap.RecordedMethod1(testData);
113
} The information related to the DataSource get added on top of the method. 7. Now, modify the CodedUITestMethod1 a bit to parameterize the test using TestContext. Add the following line in the test method: string testData = TestContext.DataRow ["SearchData"].ToString(); this.UIMap.RecordedMethod1 (testData);
8. As we have changed RecordedMethod1 to take one input parameter testData, we will change its implementation a bit. Go to the method definition and supply the testData where we were taking the Visual Studio 2010 as a hardcoded input. See the below highlighted line, we will change this line to take testData as the value. public void RecordedMethod1(string testData) { #region Variable Declarations HtmlEdit uIEnteryoursearchtermEdit = this.UIExercise3DataDrivenDWindow.UIBingDocument.UIEnteryoursearchtermEdit; HtmlDiv uIItemPane = this.UIExercise3DataDrivenDWindow.UIBingDocument.UISb_formCustom.UIItemPa n; HtmlInputButton uISearchButton = this.UIExercise3DataDrivenDWindow.UIBingDocument.UISearchButton; WinButton uICloseButton = this.UIExercise3DataDrivenDWindow.UIVisualstudio2010BingTitleBar.UICloseButton; #endregion this.UIExercise3DataDrivenDWindow.LaunchUrl(new System.Uri(this.RecordedMethod3Params.UIExercise3DataDrivenDWindowUrl)); Playback.PlaybackSettings.ContinueOnError = true; // Mouse hover 'Enter your search term' text box at (213, 5) Mouse.Hover(uIEnteryoursearchtermEdit, new Point(213, 5)); Playback.PlaybackSettings.ContinueOnError = false; // Click 'Unknown Name' pane Mouse.Click(uIItemPane, new Point(151, 34));
114
// Type 'visual studio 2010' in 'Unknown Name' pane Keyboard.SendKeys(uIItemPane, this.RecordedMethod3Params.UIItemPaneSendKeys, ModifierKeys.None); uIEnteryoursearchtermEdit.Text = testData; Mouse.Click(uISearchButton, new Point(21, 20)); // Click 'Search' button Mouse.Click(uICloseButton, new Point(18, 8)); // Click 'Close' button } Change the highlighted line as follows: uIEnteryoursearchtermEdit.Text = testData; 9. Now we are ready to run our test with different testData values as supplied from the XML file for the Search table. Run your test case, the test will run as many times the data present in XML file. See the result of the sample run below. The test run for two iterations as we have provided two values in the XML file for the Search table.
115
116
DataAccessMethod.Sequential), TestMethod] public void CodedUITestMethod1() { this.UIMap.AddTwoNumbersParams.TextInput1EditText = TestContext.DataRow["Input1"].ToString(); this.UIMap.AddTwoNumbersParams.TextInput2EditText = TestContext.DataRow["Input2"].ToString(); this.UIMap.AddTwoNumbers(); this.UIMap.AssertforAddExpectedValues.TextAnswerEditText = TestContext.DataRow["ExpectedResult"].ToString(); this.UIMap.AssertforAdd(); } 7. Save the changes to the CodedUITest1.cs source code file. 8. To run your coded UI test, right-click the coded UI test in the Test View window and click Run Selection. After the tests have run, the overall test result for all iterations of the test displays in the Test Results window. To see the details of each iteration, double-click the test in the Test Results window.
Action Recording
Using an existing action recording is an efficient way to create coded UI tests. For more information about how to create action recordings, see Recording and Playing Back Manual Tests.
117
118
The Coded UI Test Builder - Generate Code dialog box is displayed. 15. Type a name for your coded UI test method in Method Name, and then click Add and Generate. This generates code as follows if the name that you entered is, for example, AssertForAddTwoNumbers: Adds a method called AssertForAddTwoNumbers to the UIMap.Designer.cs file. This method performs the assert statements that you added. Copy public void AssertForAddTwoNumbers() { #region Variable Declarations WinEdit textAnswerEdit = this.DemoCalculatorWindowWindow.AnswerWindow.TextAnswerEdit; #endregion // Verify that 'textAnswer' text box's Text property = '40' Assert.AreEqual( this.AssertForAddTwoNumbersExpectedValues.TextAnswerEditText, textAnswerEdit.Text); }
Adds a call to the assert method AssertForAddTwoNumbers to the test method in your coded UI test file Copy [TestMethod] public void CodedUITestMethod1() { // To generate code for this test, select "Generate Code" // from the shortcut menu and select one of the menu items. this.UIMap.AddTwoNumbers(); this.UIMap.AssertForAddTwoNumbers(); }
When you choose a name for the method that has your assert statements, choose a name that is descriptive for these assertions that you created. 1. Click the Close icon to close the Coded UI Test Builder. 2. (Optional) To add code to start your application when the coded UI test runs, record the startup sequence for your application and save that to a method. You can call that method at the start of your test. Note You can add a test initialize method, identified by a [TestInitialize] attribute, which runs code at the start of each test method. For example, the method to start the application could be called from the TestInitialize method. 3. (Optional) To add code to close your application when the coded UI test runs, record the closing sequence for your application and save that to a method.
119
If you do not close the browser or application, it will remain open after your test is finished. Note You can add a test cleanup method, identified by a [TestCleanup] attribute, which runs code at the end of each test method. For example, the method to close the application could be called from the TestCleanup method. 4. To run the test, right-click in the test method, and then click Run Tests. For more information about how to run coded UI tests, see Running Automated Tests. Note After you create your coded UI test with specific data, you might want to run your coded UI test multiple times with different sets of data to test different conditions. To do this you can add parameters from a data source to your coded UI test to create a data-driven coded UI test. For more information, see How to: Create a Data-Driven Coded UI Test.
120
Chapter 9
Actions and Properties in a Coded UI
121
When you work with UI test controls in coded UI tests they are separated into two parts: actions and properties.
The first part consists of actions that you can perform on UI test controls. For example, coded UI tests can simulate mouse button clicks on a UI test control, or simulate keys typed on the keyboard to affect a UI test control. The second part consists of enabling you to get and set properties on a UI test control. For example, coded UI tests can get the count of items in a ListBox, or set a CheckBox to the checked state. Note The Coded UI Test Editor lets you easily modify your coded UI tests. Using the Coded UI Test Editor, you can locate, view, and edit your test methods. You can also edit UI actions and their associated controls in the UI control map. The Coded UI Test Editor is included in the Microsoft Visual Studio 2010 Feature Pack 2. To download the feature pack, you must have either Visual Studio 2010 Ultimate, Visual Studio 2010 Premium or Test Professional 2010 with an MSDN subscription, Microsoft BizSpark, or MSDN Academic Alliance. For more information, see Editing Coded UI Tests Using the Coded UI Test Editor and Microsoft Visual Studio 2010 Feature Pack 2.
To perform a mouse-oriented action, such as a mouse click, on a UI test control, use Click. Mouse.Click(buttonCancel);
To perform a keyboard-oriented action, such as typing into an edit control, use SendKeys. Keyboard.SendKeys(textBoxDestination, @"C:\Temp\Output.txt");
Accessing Properties of UI Test Control To get and set UI control specific property values, you can directly get or set the values the properties of a control, or you can use the UITestControl.GetProperty and UITestControl.SetProperty methods with the name of the specific property that you want you get or set.
122
GetProperty returns an object which can be cast to the appropriate Type. SetProperty accepts an object for the value of the property. To get or set properties from UI test controls directly
With controls that derive from T:Microsoft.VisualStudio.TestTools.UITesting.UITestControl, such as T:Microsoft.VisualStudio.TestTools.UITesting.HtmlControls.HtmlList or T:Microsoft.VisualStudio.TestTools.UITesting.WinControls.WinComboBox, you can get or set their property values directly as follows: Copy int i = myHtmlList.ItemCount; myWinCheckBox.Checked = true;
To get a property value from a control, use GetProperty. To specify the property of the control to get, use the appropriate string from the PropertyNames class in each control as the parameter to GetProperty. GetProperty returns the appropriate data type, but this return value is cast as an Object. The return Object must then be cast to the appropriate type. Example: int i = (int)GetProperty(myHtmlList.PropertyNames.ItemCount);
To set a property in a control, use SetProperty. To specify the property of the control to set, use the appropriate string from the PropertyNames class as the first parameter to SetProperty, with the property value as the second parameter. Example: SetProperty(myWinCheckBox.PropertyNames.Checked, true);
123
Best Activities
Use the following guidelines to create a flexible coded UI test.
Use the Coded UI Test Builder whenever possible. Do not modify the UIMap.designer.cs file directly. If you do this, the changes to the file will be overwritten. Create your test as a sequence of recorded methods. For more information about how to record a method, see How to: Create a Coded UI Test. Each recorded method should act on a single page, form, or dialog box. Create a new test method for each new page, form, or dialog box. When you create a method, use a meaningful method name instead of the default name. A meaningful name helps identify the purpose of the method. When possible, limit each recorded method to fewer than 10 actions. This modular approach makes it easier to replace a method if the UI changes. Create each assertion using the Coded UI Test Builder, which automatically adds an assertion method to the UIMap.Designer.cs file. If the user interface (UI) changes, re-record the test methods, or the assertion methods, or re-record the affected sections of an existing test method. Create a separate UIMap file for each MODULE in your application under test. For more information, see Testing a Large Application with Multiple UI Maps. In the application under test, use meaningful names when you create the UI controls. This gives more meaning and usability to the automatically generated control names. If you are creating assertions by coding with the API, create a method for each assertion in the part of the UIMap class that is in the UIMap.cs file. Call this method from your test method to execute the assertion. If you are directly coding with the API, use the properties and methods in the classes generated in the UIMap.Designer.cs file in your code as much as you can. These classes will make your work easier, more reliable, and will help you be more productive.
Coded UI tests automatically adapt to many changes in the user interface. If, for example, a UI element has changed position or color, most of the time the coded UI test will still find the correct element. During a test run, the UI controls are located by the testing framework by using a set of search properties which are applied to each control class in the definitions created by the Coded UI Test Builder in the UIMap.Designer.cs file. The search properties contain name-value pairs of property names and property values that can be used to identify the control, such as the FriendlyName, Name, and ControlType properties of the control. If the search properties are unchanged, the coded UI test will successfully find the control in the UI. If the search properties are changed, coded UI tests have a smart match algorithm which applies heuristics to find controls and windows in the UI. When the UI has changed, you might be able to modify the search properties of previously identified elements to make sure that they are found.
124
Chapter 10
What to do if your UI Changes
125
User interfaces frequently change during development. Here are some ways to reduce the effect of these changes:
Find the recorded method which references this control and use the Coded UI Test Builder to re-record the actions for this method. You can use the same name for the method to overwrite the existing actions. If a control has an assertion that is no longer valid:
Delete the method that contains the assertion. Remove the call to this method from the test method. Add a new assertion by dragging the cross-hair button onto the UI control, open the UI map, and add the new assertion.
For more information about how to record coded UI tests, see How to: Generate a Coded UI Test by Recording the Application Under Test or How to: Create a Coded UI Test. What to do if a background process needs to complete before the test can continue? You might have to wait until a process finishes before you can continue with the next UI action. To do this you can use WaitForReadyLevel to wait before the test continues as in the following sample.
// Set the playback to wait for all threads to finish Playback.PlaybackSettings.WaitForReadyLevel = WaitForReadyLevel.AllThreads; // Press the submit button this.UIMap.ClickSubmit(); // Reset the playback to wait only for the UI thread to finish Playback.PlaybackSettings.WaitForReadyLevel = WaitForReadyLevel.UIThreadOnly;
126
Create a Simple WPF Application 1. On the File menu, point to New, and then select Project. The New Project dialog box appears. 2. In the Installed Templates pane, expand Visual C#, and then select Windows. 3. Above the middle pane, verify that the target framework drop-down list is set to .NET Framework 4. 4. In the middle pane, select the WPF Application template. 5. In the Name text box, type SimpleWPFApp. 6. Choose a folder where you will save the project. In the Location text box, type the name of the folder. 7. Click OK. The WPF Designer for Visual Studio opens and displays MainWindow of the project. 8. If the toolbox is not currently open, open it. Click the View menu, and then click Toolbox. 9. Under the All WPF Controls section, drag a Button, CheckBox and ProgressBar control onto the MainWindow in the design surface. 10. Select the Button control. In the Properties window, change the value for the Content property from Button to Start. 11. Select the ProgressBar control. In the Properties window, change the value for Maximum property from 100 to 1000. 12. Select the Checkbox control. In the Properties window, clear the IsEnabled property.
127
The MainWindow.xmal.cs is displayed in the Code Editor with the cursor in the new button1_Click event. 14. At the top of the MainWindow class, add a delegate. The delegate will be used for the progress bar. To add the delegate, add the following code: Copy private delegate void ProgressBarDelegate( System.Windows.DependencyProperty dp, Object value); 15. In the button1_Click method, add the following code: Copy double progress = 0; ProgressBarDelegate updatePbDelegate = new ProgressBarDelegate(progressBar1.SetValue); do { progress ++; Dispatcher.Invoke(updatePbDelegate, System.Windows.Threading.DispatcherPriority.Background, new object[] { ProgressBar.ValueProperty, progress }); progressBar1.Value = progress; } while (progressBar1.Value != progressBar1.Maximum); checkBox1.IsEnabled = true; Verify the WPF Application Runs Correctly 1. On the Debug menu, select Start Debugging or press F5. 2. Click Start. In a few seconds, the progress bar should be 100% complete. The check box control is now enabled. 3. Close SimpleWPFApp. Create and Run a Coded UI Test for SimpleWPFApp 1. In Solution Explorer, right-click the solution, click Add and then select New Project. The Add New Project dialog box appears. 2. In the Installed Templates pane, expand Visual C#, and then select Test. 3. In the middle pane, select the Test Project template.
128
4. Click OK. In Solution Explorer, the new test project named TestProject1 is added to your solution and the UnitTest1.cs file appears in the Code Editor. You can close the UnitTest1.cs file because it is not used in this walkthrough. 5. In Solution Explorer, right-click TestProject1, click Add and then select Coded UI test. The Generate Code for Coded UI Test dialog box appears. 6. Select the Record actions, edit UI map or add assertions option and click OK. The UIMap Coded UI Test Builder appears. For more information about the options in the dialog box, see How to: Create a Coded UI Test. 7. Locate and run the SimpleWPFApp application that you created earlier. By default, the application will be located at C:\Users\<username>\Documents\Visual Studio 2010\Projects\SimpleWPFApp\SimpleWPFApp\bin\Debug\SimpleWPFApp.exe 8. Create a desktop shortcut to the SimpleWPFApp application. Right-click SimpleWPFApp.exe and choose Copy. On your desktop, right-click and choose Paste shortcut. Tip A shortcut to the application makes it easier to add or modify Coded UI tests for your application because it lets you start the application quickly. You do not have to navigate to it. You will have to run the application again in this walkthrough. 9. Click Start Recording on the UIMap Coded UI Test Builder. In a few seconds, the Coded UI Test Builder will be ready. 10. Run SimpleWPFApp.exe using the desktop shortcut. 11. On the SimpleWPFApp, click Start. In a few seconds, the progress bar should be 100 complete. The check box control is now enabled. 12. Select the box for the CheckBox control. 13. Close the SimpleWPFApp application. 14. On the UIMap - Coded UI Test Builder, click Generate Code. 15. In the Method Name type SimpleAppTest and click Add and Generate. In a few seconds, the Coded UI test appears and is added to the Solution. 16. Close the UIMap Coded UI Test Builder. The CodedUITest1.cs file appears in the Code Editor. Run the Coded UI Test
In the CodedUITest1.cs file, locate the CodedUITestMethod method, right-click and select Run Tests. While the coded UI test runs, the SimpleWPFApp is visible. It conducts the steps that you did in the previous procedure. However, when the test tries to select the check box for the CheckBox control, the Test Results window shows that the test failed. This is because the test tries to select the check box but is not aware that the CheckBox control is disabled until the progress bar is 100% complete. You can correct this and similar issues by using the various UITestControl.WaitForControlXXX() methods that are available for coded UI testing. The next procedure will demonstrate using the
129
WaitForControlEnabled() method to correct the issue that caused this test to fail. For more information, see Making Coded UI Tests Wait For Specific Events During Playback. Edit and Rerun the Coded UI Test 1. In the Test Results window, right-click the failed test and select View Test Results Details. The CodedUITestMethod1[Results] is displayed. 2. In the Error Stack Trace section, click the first link next to TestProject1.UIMap.SimpleAppTest(). The UIMap.Designer.cs file opens with the point of error highlighted in the code: Copy // Select 'CheckBox' check box uICheckBoxCheckBox.Checked = this.SimpleAppTestParams.UICheckBoxCheckBoxChecked; 3. To correct this problem, you can make the coded UI test wait for the CheckBox control to be enabled before continuing on to this line using the WaitForControlEnabled() method. Caution Do not modify the UIMap.Designer.cs file. Any code changes you make in the UIMapDesigner.cs file will be overwritten every time you generate code using the UIMap - Coded UI Test Builder. If you have to modify a recorded method, you must copy it to UIMap.cs file and rename it. The UIMap.cs file can be used to override methods and properties in the UIMapDesigner.cs file. You must remove the reference to the original method in the Coded UITest.cs file and replace it with the renamed method name. 4. In the UIMapDesinger.cs file, select all the code for the entire SimpleAppTest method and copy it. 5. In Solution Explorer, open the UIMap.cs file. 6. Paste the SimpleAppTest method code into the UIMap partial class. 7. Rename the method from SimpleAppTest() to ModifiedSimpleAppTest() 8. Add the following using statement to the file: Copy using Microsoft.VisualStudio.TestTools.UITesting.WpfControls; 9. Add the following WaitForControlEnabled() method before the offending line of code identified previously: Copy uICheckBoxCheckBox.WaitForControlEnabled();
130
// Select 'CheckBox' check box uICheckBoxCheckBox.Checked = this.SimpleAppTestParams.UICheckBoxCheckBoxChecked; 10. In the CodedUITest1.cs file, locate the CodedUITestMethod method and either comment out or rename the reference to the original SimpleAppTest() method and then replace it with the new ModifiedSimpleAppTest(): [TestMethod] public void CodedUITestMethod1() { // To generate code for this test, select "Generate Code for Coded UI Test" from the shortcut menu and select one of the menu items. // For more information on generated code, see http://go.microsoft.com/fwlink/?LinkId=179463 //this.UIMap.SimpleAppTest(); this.UIMap.ModifiedSimpleAppTest(); } 11. On the Build menu, click Build Solution. 12. Right-click the CodedUITestMethod method and select Run Tests. 13. This time the coded UI test successfully completes all the steps in the test and Passed is displayed in the Test Results window. Refactor a Control in the SimpleWPFApp 1. In the MainWindow.xaml file, in the Designer, select the button control. 2. At the top of the Properties window, select the value button1 next to Button and change the value to buttonA. 3. On the Build menu, click Build Solution. 4. On the Test menu, select Windows and then click Test View. 5. In Test View, select CodedUITestMethod1 under the Test Name column and then click Run Selection in the toolbar. The test fails because the coded UI test cannot locate the button control that was originally mapped in the UIMap as button1. Refactoring can impact coded UI tests in this manner. 6. In the Test Results window, right-click the failed test and select View Test Results Details. The CodedUITestMethod1[Results] appears. 7. In the Error Stack Trace section, click the first link next to TestProject1.UIMpa.SimpleAppTest(). The UIMap.Designer.cs file opens. The point of error is highlighted in the code:
131
Mouse.Click(uIStartButton, new Point(27, 10)); Notice that the line of code earlier in this procedure is using UiStartButton, which is the UIMap name before it was refactored. To correct the issue, you can add the refactored control to the UIMap by using the Coded UI Test Builder. You can update the tests code to use the code, as demonstrated in the next procedure. Map Refactored Control and Edit and Rerun the Coded UI Test 1. In the CodedUITest1.cs file, in the CodedUITestMethod1() method, right-click, select Generate Code for Coded UI Test and then click Use Coded UI Test Builder. The UIMap Coded UI Test Builder appears. 2. Using the desktop shortcut you created earlier, Run the SimpleWPFApp application that you created earlier. 3. On the UIMap Coded UI Test Builder, drag the crosshair tool to the Start button on the SimpleWPFApp. The Start button is enclosed in a blue box and the Coded UI Test Builder takes a few seconds to process the data for the selected control and displays the controls properties. Notice that the AutomationUId is named buttonA. 4. In the properties for the control, click the arrow at the upper-left corner to expand the UI Control Map. Notice that UIStartButton1 is selected. 5. In the toolbar, click the Add control to UI Control Map. The status at the bottom of the window verifies the action by displaying Selected control has been added to the UI control map. 6. On the UIMap Coded UI Test Builder, click Generate Code. The Coded UI Test Builder Generate Code appears with a note indicating that no new method is required and that code will only be generated for the changes to the UI control map. 7. Click Generate. 8. Close SimpleWPFApp.exe. 9. Close UIMap Coded UI Test Builder. The UIMap Coded UI Test Builder takes a few seconds to process the UI control map changes. 10. In Solution Explorer, open the UIMap.Designer.cs file. 11. In the UIMap.Designer.cs file, below the constructor in the generated code class UIMainWindow, expand the Properties region. Notice that the public WpfButton UIStartButton1 property has been added. 12. In the UIStartButton1 property, expand the Search Criteria region. Notice the SearchProperties is set to "buttonA":
132
get { if ((this.mUIStartButton1 == null)) { this.mUIStartButton1 = new WpfButton(this); #region Search Criteria this.mUIStartButton1.SearchProperties[WpfButton.PropertyNames.AutomationId] = "buttonA"; this.mUIStartButton1.WindowTitles.Add("MainWindow"); #endregion } return this.mUIStartButton1; } } Now you can modify the coded UI test to use the newly mapped control. As pointed out in the previous procedure if you want to override any methods or properties in the coded UI test, you must do so in the UIMap.cs file. 13. In the UIMap.cs file, add a constructor and specify the SearchProperties property of the UIStartButton property to use the AutomationID property with a value of "buttonA": Copy public UIMap() { this.UIMainWindowWindow.UIStartButton.SearchProperties[WpfButton.PropertyName s.AutomationId] = "buttonA"; } 14. On the Build menu, click Build Solution. 15. In Test View, select CodedUITestMethod1 under the Test Name column, In the toolbar, click Run Selection. This time, the coded UI test successfully completes all the steps in the test. In the Test Results Window, you will see a status of Passed.
133
134
Chapter 11
Extending Coded UI Tests and Action Recordings to Support Microsoft Excel
135
The testing framework for coded UI tests and action recordings does not support every possible user interface. It might not support the specific UI that you want to test. For example, you cannot immediately create a coded UI test or an action recording for a Microsoft Excel spreadsheet. However, you can create your own extension to the coded UI test framework that will support your specific UI by taking advantage of the extensibility of the coded UI test framework. The following topic gives an example of how to extend the framework to support the creation of coded UI tests and action recordings for Microsoft Excel. For more information about the platforms that are supported, see Supported Configurations and Platforms for Coded UI Tests and Action Recordings. This section presents a coded UI test extension that can record and play back tests of Excel Worksheets. Each part of the extension is explained in this section and in the code comments for developers who want to create just such an extension.
Architecture Overview
136
SCENARIO:
A customer Im working with has placed a heavy investment in Waiting testing over the years and with a recent move to TFS2010 they also wanted to take advantage of the new Microsoft Test Manager (MTM) feature and the ability to associate automated tests to test cases in MTM. Heres a quick how-to for those of you wanting to do the same thing.
Pretty simple open browser and check the url in the address bar. What I have also done with this test is use parameters to supply the data allowing testers to decide what values they want to test with. This is better than the devs doing this, plus it makes for a much nicer UI for data driven tests than excel or CSV files do.
137
using Microsoft.VisualStudio.TestTools.UnitTesting; using WatiN.Core; namespace CodedUITesting { [CodedUITest] public class CodedUITest1 { ... [TestMethod] [DataSource("Microsoft.VisualStudio.TestTools.DataSource.TestCase", "http://tfs2008-vm:8080/tfs/TemplateTrials;Scrum v5", "101", DataAccessMethod.Sequential)] public void RunWatinTestParameterized() { var url = TestContext.DataRow["url"].ToString(); var result = TestContext.DataRow["result"].ToString(); using (IE ie = new IE(url)) { Assert.AreEqual(result, ie.Url); } } } The things to note in this test are that the data source attribute is used to pull data from the TFS Test Case work item specifically the URL of the team project is included, as it the test case id. In the test itself we pull data from the parameters in the test case using the TestContext.DataRow[parameter'NameHere].ToString() calls, and the rest is just normal Watin code. If you have any issues with missing references make sure you reference Watin and Interop.ShDocVw and that both get copied to the output folder as part of the build. Attach the Automation The only thing we need to do once we have this test is to link it to the test case as its automation method. Open the test case in Visual Studio, navigate to the Associated Automation tab and click the [] button to select the method. You should see something like this once its done:
138
Run The Test Assuming you have a lab environment with test agents installed you should then be able to trigger a new build and when its complete start an automated test run for that build and see everything working as you would expect. Heres a result from a run on my local TFS server:
Using Visual Studio Lab Management you can run manual or automated tests from a test plan using your virtual environments. This lets you run your tests on an environment that uses a known state for all the virtual machines in the environment. Now, testers no longer need to create clean machines in a known state manually, which can make it much quicker to set up an environment to use for testing. You can also deploy your application to your virtual environment that uses Lab Management. For more information about how to deploy your application, see How to: Deploy an Application on a Virtual Environment. Note If you will be performing manual or automated testing in purely physical environments, or in physical environments composed of third-party virtual machines, you do not need to do any of the following tasks: use Hyper-V, configure SCVMM, or configure lab management. Instead, you can simply install a test agent on the physical machine, register a test controller with your team project collection, and configure the test agent to use that test controller. After you have done this, use the Lab Center in Microsoft Test Manager to create a new physical environment. For the steps to create a physical environment and run tests, see Creating a Physical Environment to Use for Testing. When you run one or more of these tests from your test plan, you can save your test results into the team project for your Team Foundation Server. You can now view the progress of both your automated and manual tests together from your test plan. To run manual tests using a virtual environment, you must use Microsoft Test Manager to add the virtual environment and test settings for manual tests to your test plan and then run your tests from a test plan as shown in the following illustration. Then you can view your test results from your test plan.
140
Note You can also select Run with options from the Run Tests view to select the virtual environment and test settings to use when you run your tests. If you find a bug in your application when you run your manual tests, you can take a snapshot of your virtual environment that a developer can use to connect to the actual state of your machines in your environment when you found the bug. This helps create easily reproducible bugs. If you want to run automated tests using your virtual environments, you must associate your automated tests with test cases using Microsoft Visual Studio 2010, create test settings and add your virtual environment for your automated tests to your test plan, and then run them using Microsoft Test Manager. If you want to schedule your tests to run as part of your build process using Visual Studio Lab Management you can create a workflow using a build definition for Team Foundation Build that enables you to build your application, deploy it to your virtual environment and then run automated tests.
Running Tests
If you have Visual Studio Test Professional 2010 or Visual Studio 2010 Ultimate, you can run manual tests and automated tests from a test plan by using Microsoft Test Manager. When you run any of these tests from your test plan, you can save your test results into the team project for your Team Foundation Server. You can now view the progress of both your automated and manual tests from your test plan. If you have Visual Studio 2010 Ultimate ,Visual Studio 2010 Premium, or Visual Studio Professional you can run automated tests from Microsoft Visual Studio 2010 or from the command line. You can also run tests by using Team Foundation Build. Test results are created every time that you run a group of tests. To run manual tests, you must use Microsoft Test Manager to run your tests from a test plan, as shown in the following illustration. Then you can view your test results from your test plan.
141
After you have created any of the following types of automated tests by using Microsoft Visual Studio 2010, you can use Microsoft Test Manager or Microsoft Visual Studio 2010 to run the tests: Unit tests. Coded UI tests. Database unit tests. Load tests. Generic tests.
For example, if you want to run your automated tests and see the results immediately, you can just run your tests from Microsoft Visual Studio 2010 and view the test results, as shown in the following illustration.
142
If you want to run your automated tests and have the results added to a test plan, you can associate your automated tests with test cases using Microsoft Visual Studio 2010, and then run them using Microsoft Test Manager, as shown in the following illustration.
143
If you run database unit tests locally by using Visual Studio, you must have one of these SKUs installed locally. If you run database unit tests by using a test controller and test agents from Visual Studio, you must install one of these SKUs on any test agent computer that can be used to run these tests. If you associate your database unit tests with a test case and run these tests from Microsoft Test Manager, you must install one of these SKUs on any test agent in your environment that is included in the role to run tests. You must also install one of these SKUs on the test controller for this environment.
144
Chapter 12
Running Automated Tests
145
After you have created any one of the following types of automated tests by using Microsoft Visual Studio 2010, you can then run the tests.
Unit tests. Coded UI tests. Database unit tests. Load tests. Generic tests.
Many methods of running your automated tests are available, depending on how you want to run tests and view the results. If you run your automated tests by using a test plan, you can view your testing progress and easily rerun your tests as required. To run your automated tests by using a test plan, you associate your automated tests with test cases and run these test cases by using Microsoft Test Manager. To run your automated tests in this manner, you must create a physical or virtual environment to use when you run your tests. For more information about how to create virtual environments to use to run your tests, see Using a Virtual Lab for Your Application Lifecycle. Note: It is not a good idea to run load tests by using Microsoft Test Manager because you cannot view the test as it runs. It is useful to be able to do this for load tests. The environment enables you to run tests, gather data, or perform system actions on machines for each specific role that you add to the environment. A role specifies the purpose of a machine in the environment. For example, a specific role could be called "Web Site for Customer Data Store." A machine can be a physical computer or a virtual machine. You select which machines to use in an environment for each role. Scenario 1: A tester could run your tests on one machine and gather system information about a machine that runs the Web server for your application. Alternatively, you could run your tests on an environment that uses multiple machines and collect test impact data on those machines. In addition, you can also perform network emulation on the machine that runs the Web server for your application. The following illustration shows three examples of scenarios for how you can set your test settings to run your tests by using environments from Microsoft Test Manager.
146
This method of running automated tests lets you view the overall status of any tests in your test plan. You can view the results of both manual and automated tests together, if you want. You can also run these test cases by using associated automation from the command line. You can also run your tests directly from Microsoft Visual Studio 2010 or the command line without being part of a test plan or without using an environment to run them remotely. In addition, if you add your automated tests to a test category or a test list, your automated tests can be run automatically as part of the build process. Note To take advantage of the multiple processors in a multiprocessor machine, you must run your tests by using test agent controllers and test agents. Otherwise, if you run your tests locally, you will not use this additional processing power. For more information, see Setting Up Test Machines to Run Tests or Collect Data Based on the following ways that you can run database unit tests, you may have to install one of these SKUs on additional machines:
If you run database unit tests locally by using Visual Studio, you must install one of these SKUs locally. If you run database unit tests by using a test controller and test agents from Visual Studio, you must install one of these SKUs on any test agent computer that can be used to run these tests.
147
If you associate your database unit tests with a test case and run these tests from Microsoft Test Manager, you must install one of these SKUs on any test agent in your environment that is included in the role to run tests. You must also install one of these SKUs on the test controller for this environment.
148
149
Chapter 13
Introduction to Debugging
150
Introduction
In the software development life cycle, testing and defect fixing take more time than actually code writing. In general, debugging is a process of finding out defects in the program and fixing them. Defect fixing comes after the debugging, or you can say they are co-related. When you have some defects in your code, first of all you need to identify the root cause of the defect, which is called the debugging. When you have the root cause, you can fix the defect to make the program behavior as expected. Now how to debug the code? Visual Studio IDE gives us a lot of tools to debug our application. Sometimes debugging activity takes a very long time to identify the root cause. But VS IDE provides a lot of handy tools which help to debug code in a better way. Debugger features include error listing, adding breakpoints, visualize the program flow, control the flow of execution, data tips, watch variables and many more. Many of them are very common for many developers and many are not. In this article, I have discussed all the important features of VS IDE for debugging like Breakpoint, labeling and saving breakpoints, putting conditions and filter on breakpoints, DataTips, Watch windows, Multithreaded debugging, Thread window, overview of parallel debugging and overview of IntelliTrace Debugging. I hope this will be very helpful for beginners to start up with and for becoming an expert on debugging. Please note, targeted Visual Studio version is Visual Studio 2010. Many things are common in older versions, but many features such as Labeling breakpoint, Pinned DataTip, Multithreaded Debugging, Parallel debugging and IntelliTrace are added in VS 2010. Please provide your valuable suggestions and feedback to improve my article.
151
There is another way to start the debugging by "Attach Process". Attach process will start a debug session for the application. Mainly we are very much familiar with the attach process debugging for ASP.NET Web Application. I have published two different articles on the same on CodeProject. You may have a look into this. Debug Your ASP.NET Application that Hosted on IIS Remote IIS Debugging: Debug your ASP.NET Application which is hosted on "Remote IIS Server" We generally start debugging any application just by putting breakpoint on code where we think the problem may occur. So, lets start with breakpoints.
Breakpoints
Breakpoint is used to notify debugger where and when to pause the execution of program. You can put a breakpoint in code by clicking on the side bar of code or by just pressing F9 at the front of the line. So before keeping a breakpoint, you should know what is going wrong in your code and where it has to be stopped. When the debugger reaches the breakpoint, you can check out what's going wrong within the code by using a different debugging tool.
152
Step Over
After debugger hits the breakpoint, you may need to execute the code line by line. "Step Over" [ F10 ] command is used to execute the code line by line. This will execute the currently highlighted line and then pause. If you select F10 while a method call statement is highlighted, the execution will stop after the next line of the calling statement. Step Over will execute the entire method at a time.
Step Into
This is similar to Step Over. The only difference is, if the current highlighted section is any methods call, the debugger will go inside the method. Shortcut key for Step Into is "F11".
Step Out
This is related when you are debugging inside a method. If you press the Shift - F11 within the current method, then the execution will complete the execution of the method and will pause at the next statement from where it called.
Continue
It's like run your application again. It will continue the program flow unless it reaches the next breakpoint. The shortcut key for continue is "F5".
153
154
public static void Method1() { Console.WriteLine("Break Point in Method1"); // BreakPoint } public static void Method2() { Console.WriteLine("Break Point in Method2"); // BreakPoint Console.WriteLine("Break Point in Method2"); // BreakPoint } public static void Method3() { Console.WriteLine("Break Point in Method3"); // Breakpoint } } If you run the program, execution will pause on the first breakpoint. Now see the below picture, where you have the list of breakpoints.
In the given picture label column in blank. Now, see how you can set the label on break point and what the use of it is. To set label for any breakpoint, you just need to right click on the breakpoint symbol on the particular line or you can set it directly from breakpoint window.
Conditional Breakpoint
Suppose you are iterating through a large amount of data and you want to debug a few of them. It means you want to pause your program on some specific condition. Visual Studio Breakpoints allow you to put conditional breakpoint. So if and only if that condition is satisfied, the debugger will pause the execution. To do this, first of all you need to put the breakpoint on a particular line where you want to pause execution. Then just "Right Click" on the "Red" breakpoint icon. From there you just click on "Condition"
155
Let's assume that you have the following code block: class Program { static void Main(string[] args) { string [] strNames = { "Name1","Name2", "Name3", "Name4", "Name5", "Name6"}; foreach(string name in strNames) { Console.WriteLine(name); // Breakpoint is here } } } You have a breakpoint on Console.WriteLine() statement. On running of the program, execution will stop every time inside that for-each statement. Now if you want your code to break only when name="Name3". What needs to be done? This is very simple, you need to give the condition like name.Equals("Name3").
156
Immediate Window
Immediate window is very much common and a favorite with all developers. It's very much helpful in debug mode of the application if you want to change the variable values or execute some statement without impacting your current debugging steps. You can open the Immediate window from menu Debug > Window > Immediate Window { Ctrl + D, I / Alt + Ctrl - I }. Immediate window has a set of commands which can be executed any time during debugging. It also supports Intellisense. During Debug mode, you can execute any command or execute any code statement from here.
These are very much common features for all the developers, so I am not going into details of each and every command of Immediate window.
Call Stack
These features also improve the productivity during debugging. If you have multiple method calling or nested calling all over your application and during debugging, you want to check from where this method has invoked, "Call Stack" comes into the picture. The Call Stack Window shows that current method call nesting. In Call Stack window if you clicked on any of the rows, it will point you to the actual code of line of Visual Studio Code Editor. You can also customize the call stack row view by selecting different types of columns. To customize, Right Click on the "Call Stack" window, and from the context menu, you can select or deselect the option.
157
Call stack is very much important when you have multiple methods call all across the application and one particular method throwing an exception on some particular case. At that time, you can use call stack to see from where this method is getting invoked, based on that you can fix the defect.
158
In the sample code, you have three different threads - Main Thread, Thread 1, Thread 2. I have given a thread name to make you understand better. Now set a breakpoint inside "Go()" and run the application. When debugger hits the breakpoint, Press Ctrl+D,T or Navigate through Debug > Window > Threads. Threads window will appeared on the screen.
By default thread window having ID, Managed ID, Category, Name, Location and Priority column. At the start, execution pauses at "Main Thread". "Yellow Arrow" indicates the current executable thread. Category column indicates the category of threads, like main thread or worker thread. If you check the thread location, it is nothing but Namespace > Class > Method name. In the diagram, it is showing that the Main Thread will be executed next. Now to explore the next step by just pressing "F5" and see what are the changes in thread window.
So after pressing F5, it jumped to the next step to thread 1. you can also check the current location for Main Thread. It says "Sleep/ Wait / Join" , means waiting for something to complete. Similarly the next step will move you to thread 2. From the Thread window, you can understand how easy it is to monitor your threads using this debugger tool. There is another great feature available within the thread window. You can expand/collapse the Thread Location and can see what is next. For example, if you expand the location for "Main Thread", it will look like the diagram given below:
159
160
Here ThreadName was one of the criteria by which you can filter, but you can filter on multiple clauses like ThreadID, ProcessName, ProcessID, etc. After setting the breakpoint filter, run the application and open the "Threads" window.
You will find your program execution has only paused during the execution of "Thread 2" . This is all about the debugging with multithreaded application. Hope you have learned something from it. Let's start with another most important topic "Parallel Debugging".
161
Now Debugging the parallel program is also a big topic. Here I will give you a basic overview to know about the debugging of parallel program. To discuss about it, let's consider you have the following piece of code: using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading; using System.Threading.Tasks; namespace ParalleTaskDebugging { class Program { static void Main(string[] args) { var task_a = Task.Factory.StartNew(() => DoSomeWork(10000)); var task_b = Task.Factory.StartNew(() => DoSomeWork(5000)); var task_c = Task.Factory.StartNew(() => DoSomeWork(1000)); Task.WaitAll(task_a, task_b, task_c); } static void DoSomeWork(int time) { Thread.Sleep(time); } } } To understand the parallel program debugging, we need to be aware about two window options: 1. Parallel Tasks 2. Parallel Stacks
162
to check the status of each task. To test, put a breakpoint on DoSomeWork() method and run the application. You will see your program execution paused on the breakpoint. After the program break, you can go to Debug > Window > Open Parallel Tasks and Parallel Stacks window. I asked to open both at the same time only because you can visualize what is going on.
Parallel Task window will show you what are the different tasks that have been created for the program and what is their current status. On the other hand, Parallel Stacks will show you the graphical view of all thread creation, containing tasks, how they are related. If you click on the thread from the Parallel Stacks, it will show you the code line related with the thread (as shown in the picture with a Green Arrow). To move ahead, press F5. Let's see what comes next.
163
In the above diagram, you can find one of the tasks has been executed and the other two are remaining. Current execution point is set to AsyncMethod_1, so if you continue, this method will execute first and next time the others. When you are working with parallel programming, there are many scenarios which will come like Deadlock, Dependency problem, etc. These topics is very interesting and long to discuss. Please check further study section of the article to know more details.
164
Chapter 14
Create Test Cases from Automated Tests Using TCM.exe
165
Introduction
If you have created automated tests using Visual Studio, you might want to run these tests as part of a test plan. By adding them to a test plan, you can measure your testing progress. You can view the results from these automated tests and any other tests in your test plan together and track the progress that you are making. You can also link these test cases to requirements, so that you can view the status of automated tests for the requirement. For more information about how to do this, see Link Existing Test Cases to a Requirement. To run these tests as part of a test plan, you can use a command line tool tcm.exe to create test cases for all your test methods in an assembly or some of your test methods based on test method properties or the test category. If you want to add a test to an existing test case, you can associate a test method directly with that test case. For more information, see How to: Associate an Automated Test with a Test Case. If you have existing automated tests from earlier versions of Microsoft Visual Studio 2010, you can associate these tests with a test case after you upgrade the test project that contains the automated tests to Visual Studio 2010. After you have imported your test methods and created test cases, you have to add these test cases into your test plan. To run these test cases from your test plans, you must use a physical or a virtual environment. You cannot run automated tests using Microsoft Test Manager without an environment. You must check in the test project that contains the automated test methods and make sure that the test project is part of your build definition. You must make sure that a build created by using this build definition is selected in your test plan. The information from the build is used by Microsoft Test Manager to locate the correct test method assembly and use this to run the automated tests. For more information about how to check in your test project and team build, see Add Files to Version Control and Building the Application. You can also run automated tests from the command line using test settings and a test environment. For more information, see How to: Run Test Cases with Automation from the Command Line Using Tcm. Use the following procedures to create test cases from an assembly of automated tests that you can then run as part of your test plan:
Create the test cases from your assembly of automated tests and add them to your plan Set up your test plan to use your team build Create your test settings and environment to run your tests
166
Run the automated test using Microsoft Test Manager View and update the test results
/teamproject:
TeamProjectName
/login: UserName,Password
/? or help
167
Note: The test cases that are created will have the same names as the test methods that were imported. 3. (Optional) You can limit the test methods that are imported from the assembly based on test categories. For more information about test categories, see Defining Test Categories to Group Your Tests. For example, the following command shows you how to import tests that are in both the ShoppingCart and SmokeTest categories. tcm testcase /collection:<CollectionURL> /teamproject: <Team Project name> /import /storage: <name of your assembly> .dll /category:"ShoppingCart&SmokeTest" 4. (Optional) You can limit the test methods that are imported from the assembly based on a test method property. For example, the following command shows how to import all test methods that have priority set to 0 or 1.
168
tcm testcase /collection:<CollectionURL> /teamproject: <Team Project name> /import /storage: <name of your assembly> .dll/maxpriority:1 5. Open Microsoft Test Manager. Note: To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010. 6. To select a test plan, click the down-arrow on the center group switcher and then click Testing Center. 7. On the center group menu bar, click Plan. 8. (Optional) To select a different test plan, click the test plan name hyperlink in the upperright corner, or click the home icon. 9. You can add the test cases to a new test suite, or an existing test suite. To add the test cases into your plan, click Contents and then select the test suite in the test suite hierarchy or add a new test suite. Then click Add in the test suite details pane. The Add Test Cases to Suite dialog box is displayed. Note For more information about test suites in test plans, see Organizing Test Cases Using Test Suites. 10. Select the test cases that were created. These test cases will have the same names as the test methods in the test assembly. Then click Add Test Cases. The test cases are shown in the list of tests for the test suite that you selected. Note: You can also import your test cases directly into a static test suite. For more information about how to do this, see tcm: Importing Automated Tests into Test Cases.
169
To set up your test plan to run the automated test that you have created, you must choose the correct build definition used to build your automated test methods. You must do this so that the automated test assembly can be found in the share location for your build definition and then it can be run from Microsoft Test Manager.
EXERCISE:
1. Open Microsoft Test Manager. Note To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010. 2. To select a test plan, click the down-arrow on the center group switcher and then click Testing Center. 3. On the center group menu bar, click Plan. 4. To set up your test plan to run the automated tests, click Properties and then click Filter for builds. The Filter completed builds dialog box is displayed. 5. To select the build definition that is used to build your automated tests, click Build definition. 6. Each build can be given a specific value to reflect the quality of the build. To select the quality of the builds you want to be able to view, click Build quality. Note For more information about build definitions and build quality, see Define Your Build Process 7. To save your changes, click Set build filter. 8. To select the most recent build to use with this test plan that includes the latest changes to the automated test, you must first click Save to save the plan and then click Modify. The Assign Build activity is displayed. You can compare your current build with a build you plan to take. The associated items list shows the changes to work items between the builds. You can then assign the latest build to use for testing with this plan. For more information about how to select a build, see Determining Which Builds Have Bug Fixes, New Features, or Requirements. 9. To close the Assign Build activity and return to the test plan properties, click the Close icon.
170
To save these changes for this test plan, click Save in the toolbar.
EXERCISE:
To run your automated tests, you must use a physical or virtual environment. You cannot run automated tests using Microsoft Test Manager without an environment. You must create an environment that contains the roles in your test settings and then use this environment in your test plan. For more information about how to create your environment and roles and test settings, see Setting Up Test Machines to Run Tests or Collect Data. Note If you want to run automated tests that interact with the desktop, you must set up your agent to run as a process instead of a service. For more information, see How to: Set Up Your Test Agent to Run Tests that Interact with the Desktop. To create your test settings and environment to run your tests 1. To create a physical environment, follow the steps in this topic: Creating a Physical Environment to Use for Testing. 2. If you are using Visual Studio Lab Management, you can create a virtual environment. To create a virtual environment, follow the steps in this topic: How to: Create an Environment from Virtual Machines or Templates. If you want more information about how to create virtual environments, see Virtual Environments Concepts and Guidelines. 3. To create your test settings, follow the steps in this topic: Create Test Settings for Automated Tests as Part of a Test Plan.
EXERCISE:
EXERCISE: To run the automated test using Microsoft Test Manager 1. Open Microsoft Test Manager. Note To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010. 2. To run the automated test, click the down-arrow on the center group switcher and then click Testing Center. 3. In the center group menu bar, click Test.
171
4. (Optional) To override the build, the test settings or the environment to use for running the automated tests that you select in this test plan, right-click the test and then click Run with options. For example, if you want to run on a staging environment instead of your standard testing environment then you might choose a different environment. From the Run options dialog box, you can change these settings, and then click Run to run the selected test. Note If you select a different environment, it must have the same roles as the environment that was created in the test settings that you use. 5. To run the automated test without changing any options, right-click the test and then click Run. The Analyze Test Runs activity is displayed that shows the progress of the test run that contains this test. Note You can run multiple automated tests by selecting multiple tests, or you can select to run a whole suite of tests. To run a suite, right-click the test suite and then click Run.
172
EXERCISE:
To view and update the test results 1. Open Microsoft Test Manager. Note To display the Microsoft Test Manager window, click Start, and then click All Programs. Point to Microsoft Visual Studio 2010 and then click Microsoft Test Manager 2010. 2. To view the test results, click the down-arrow on the center group switcher and then click Testing Center. 3. On the center group menu bar, click Test and then click Analyze Test Runs. The Analyze Test Runs activity is displayed. It shows any test runs for this test plan 4. Double-click a test run to open it and view the details. The test run details are displayed. 5. (Optional) To update the title of your test run to be more meaningful, type the new name in Title. 6. (Optional) If your test failed, you can update the reason for the failure. Click Resolution and select the reason for the failure from the list. 7. (Optional) To add comments to the test result, click the Comments icon. Type your comments and then click Save comments. 8. (Optional) To view the details of an individual test, double-click the test. The test result is displayed. It shows the details from the test run, the attachments for data collected for this test result, and the test results history for that test. You can close this view to return to the test run. Note If you determine that there is a bug from your analysis, you can create a bug from this view. 9. To save these changes for this test run, click Save in the toolbar.
173
174
INDEX
Application Lifecycle Management, 4, 18, 19, 23, 25 Application Tier, 4, 21 Automated software testing, 4, 10, 11 Automatic Compiling, 4, 22 Client Tier, 4, 21 collect data remotely, 7, 63 Create a test plan, 6, 57 Creating and managing tests, 7, 63 Data Tier, 4, 21 Host, 5, 31, 32 Lab Center, 5, 33, 129 Library server, 5, 32 Library share, 5, 32 Manual software testing, 4, 10 Microsoft Test Manager, 4, 23, 24, 26, 27, 28, 33, 35, 51, 54, 56, 59, 61, 62, 63, 66, 67, 68, 69, 76, 129, 130, 131, 132, 133, 134, 135, 137, 138, 140, 141, 142, 144 Physical environment, 5, 33 Planning your testing effort, 6, 63 Running Your Tests, 7, 63 Setting up how to run tests, 7, 63 Snapshot, 5, 32, 34 Submitting bugs, 7, 63 tcm.exe, 8, 59, 137 Team project library, 5, 33 Template, 5, 32, 83, 84 Tracking software quality, 7, 63 UI Automation, 4, 11, 12, 13, 14, 15, 16 Virtual environment, 5, 33 Virtual machine, 5, 31 Visual Studio Ultimate, 4, 19, 23, 24, 49, 56, 62, 63, 89, 102
175
176
177
178