Sikuli

ome

Satish Gorripotu

SIKULI
SATISH GORRIPOTU
Gsatish_leela@yahoo.com 9885121381

Sikuli
TOPICS AND STATUS

Satish Gorripotu

Topic
Sikuli Sikuli and Other Supporting Softwares Required Installing/Configuring JDK or JRE / Setting Class Path Installing Sikuli Starting Sikuli IDE Stop the script during run time: Sikuli Working Architecture: Check Box: dragDrop: Sliders Testing on Skype Application: Meeting Request Arrival Notifier: Images Dictionary/Repository: Open any application in run time. Empty Recycle Bin: Run Sikuli from Command Line: Create Unit Testing Scripts for GUI: Sikuli Script in JAVA programs: Sikuli together with other IDEs - Eclipse: Import Packages/ Files Controlling Sikuli Scripts and their Behavior: Setting Controlling Applications and their Windows: App Interacting with the User Listening to Global Hotkeys General Settings and Access to Environment Information Region Extending a Region Finding inside a Region and Waiting for a Visual Event Observing Visual Events in a Region SikuliEvent Low-level Mouse and Keyboard Actions Exception FindFailed Screen Match Pattern Finder Key Constants related to Mouse and Keyboard Actions

Comments Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented Documented

Page Number

Sikuli
Extensions Summary Java Docs Jython Basics References Excel Handling in Sikuli Object Repository Handling Synchronization Problems Methods that are mostly used Maintaining the Test Results Frameworks Database Testing using Sikuli Sending the test results to email by using Sikuli Challenges faced

Satish Gorripotu
Documented Documented Documented Documented Documented Yet to Complete Yet to Complete Yet to Complete Yet to Complete Yet to Complete Yet to Complete Yet to Complete Yet to Complete Yet to Complete

Sikuli
Sikuli

Satish Gorripotu

Sikuli is an open-source research project. This is a visual technology to automate and test graphical user interfaces (GUI) using images (screenshots). It includes 1. Sikuli Script, 2. Visual scripting API for Jython 3. Sikuli IDE. Sikuli Script automates anything you see on the screen without internal API's support. We can programmatically control a web page, a Windows/Linux/Mac OS X desktop application, or even an iphone or android application running in a simulator or via VNC. Download and unzip the Sikuli Using following link http://sikuli.org/download.shtml Sikuli and Other Supporting Softwares Required 1. Sikuli-X-1.0rc2-win32.exe

2. Java Runtime Environment (JRE) 6+

Make sure to have only one instance of each software is available on machine. You are provided with Admin access to configure the paths in the Environmental variables.

Installing/Configuring JDK or JRE / Setting Class Path Install the JDK package and update the JAVA_HOME environment variable (User) in the Advanced tab. Of System Properties. In system variable edit 'PATH' by adding C:\Program Files\Java\jdk1.6.0_14\bin; I . Right click on my computer->Properties->Advanced->Environment variables II . 1. Click on "New" in system variables 2. Create a new variable as shown below Variable name = JAVA_HOME Variable value = C:\Program Files\Java\jdk1.6.0_14 // [path of jdk is installed] III. Select the existing path variable and edit it as shown below PATH = % JAVA_HOME%\bin; % JAVA_HOME%\lib; IV. Click on OK

Sikuli

Satish Gorripotu

V . START-> RUN ->type CMD VI. Now enter "javac" to check whether the java correctly installed or not.

Sikuli
Installing Sikuli

Satish Gorripotu

Sikuli-X-1.0rc2-win32.exe is self extracting file. Just double click and click next next until finish whilst follow the instructions. 1. It will automatically install the software. 2. It will configure the environment variables. 3. In desk top or program files we can see the short cut for Sikuli IDE as

Starting Sikuli IDE Double click on the Sikuli IDE will give the below IDE

There are two ways to capture the screen images 1. Click on in the IDE

2. Press a hot-key (Ctrl + Shift + 2)

Sikuli

Satish Gorripotu

Screen will look darker and freeze momentarily. The entire desktop becomes like a canvas where we can draw a rectangle around the target to capture an image. Now that image will be captured and inserted in IDE.

We can use Auto List command capture instead of typing the commands manually. 1. Find a. find b. find all c. wait d. waitvanish e. exists 2. Mouse Actions a. Click b. doubleClick c. rightClick d. hover e. dragDrop 3. Keyboard Actions a. type b. paste 4. Event Observation a. onAppear b. OnVanish c. onChange d. observe

to enable the basic commands in the IDE

Below script will open the Keane web site www.keane.com and search with the keyword Testing.

Sikuli

Satish Gorripotu

Type: type ("http://sikuli.org" + Key.ENTER) miscellanous keys ENTER - TAB - ESC - BACKSPACE - DELETE - INSERT function keys F1 - F2 - F3 - F4 - F5 - F6 - F7 - F8 - F9 - F10 - F11 - F12 - F13 - F14 - F15 navigation keys HOME - END - LEFT - RIGHT - DOWN - UP - PAGE_DOWN - PAGE_UP key modifiers ALT - CMD - CTRL - META - SHIFT - WIN

Sikuli

Satish Gorripotu

Stop the script during run time: Press alt-shift-c

Sikuli
Sikuli Working Architecture:

Satish Gorripotu

Sikuli Script Sikuli Script is a Jython and Java library that automates GUI interaction using image patterns to direct keyboard/mouse events. The core of Sikuli Script is a Java library that consists of two parts: java.awt.Robot, which delivers keyboard and mouse events to appropriate locations, and a C++ engine based on OpenCV, which searches given image patterns on the screen. The C++ engine is connected to Java via JNI and needs to be compiled for each platform. On top of the Java library, a thin Jython layer is provided for end-users as a set of simple and clear commands. Therefore, it should be easy to add more thin layers for other languages running on JVM, e.g. JRuby, Scala, Javascript, etc. The Structure of a Sikuli source/executable script (.sikuli, .skl) A Sikuli script (.sikuli) is a directory that consists of a Python source file (.py), and all the image files (.png) used by the source file. All images used in a Sikuli script are simply a path to the .png file in the .sikuli bundle. Therefore, the Python source file can also be edited by any text editor. While saving a script using Sikuli IDE, an extra HTML file is also created in the .sikuli directory so that users can share the scripts on the web easily. A Sikuli executable script (.skl) is simply a zipped file of all files in the .sikuli directory. When a script is passed to Sikuli IDE as a command line argument, Sikuli IDE recognizes its type by check its filename extension. If a .skl is seen, Sikuli IDE runs it without showing the IDE window. If a .Sikuli is seen, Sikuli IDE opens it in a source code editor.

Sikuli
Sikuli IDE

Satish Gorripotu

Sikuli IDE edits and runs Sikuli source scripts. Sikuli IDE integrates screen capturing and a custom text editor (SikuliPane) to optimize the usability of writing a Sikuli script. To show embedded images in the SikuliPane, all string literals that ends with ".png" are replaced by a custom JButton object, ImageButton. If a user adjusts the image pattern's similarity, a Pattern() is automatically constructed on top of the image. To execute a Sikuli script, Sikuli IDE creates a org.python.util.PythonInterpreter and automatically passes a few lines of headers (e.g. to import Sikuli's Jython modules, and to set the path to .sikuli directory) to the interpreter. Once these headers are set, the .py script is simply executed by PythonInterpreter.execfile(). Working with Check Box: 1. Check the all check boxes: findAll() : searches the screen for all the matching visual patterns and returns a list of locations. Then we can use click() function to check the all check box.

2. Check the particular check box:

Sikuli

Satish Gorripotu

Step 1: we have to select the region

Step 2: Then identify check box

Below is the code for that.

Sikuli
dragDrop:

Satish Gorripotu

The above script will drag and drop all the word files into specified folder.

Sikuli
Working with Sliders

Satish Gorripotu

Same procedure as check boxes. But we can use dragDrop() function to move slide. Way 1:

Way 2: Need to work with Coordinates. Moving to Null. Need to resolve by Ram Raju. Find Below Screen shot. There x, Y are the coordinates. Main Scenario is: Zoom in and Zoom Out of the the word document.

Sikuli

Satish Gorripotu

hover(find(

).nearby(200).right().find(

).below().find(

).getCenter().offset(3,18))

Sikuli
Testing on Skype Application:

Satish Gorripotu

Now status of the Family Bujji is

. We have to wait up to status as

We can use any of the below way to Way 1: while not exists( ):

Sikuli
sleep(5) Way 2: wait( Way 3: ,FOREVER)

Satish Gorripotu

wait( Way 4: while exists( sleep(5) Way 5: waitVanish(

,3600)#wait for an hour

):

, FOREVER)

popup(Oyee Satti !!!!!! Your Wife Online)

Sikuli
Meeting Request Arrival Notifier:

Satish Gorripotu

Define a region and focus only on that region to search for a particular visual pattern. So tool dont have to scan the whole screen and we can resize the browser to show just that portion of the map, while leaving a lot of screen space to do something else. To do so, click on the Create region button.

The entire screen will freeze and turn darker, similar to what happen in the screen capture mode. Simply draw a rectangle to cover the entire neighborhood map. The region covered by this rectangle is visually represented as a thumbnail image of the entire desktop where the region is shaded in red.

Sikuli

Satish Gorripotu

Next, we capture the screenshot image of the meeting symbol so that we can tell Sikuli Script to watch for

its appearance. We can write the following script to check the mails and watch for the appearance of a meeting Request symbol: Way 1:

Way 2:

Images Dictionary/Repository: 1. Store Images as Predefined.

Sikuli
Filename = VDict() Filename [ ]= 'Recycle'

Satish Gorripotu

Filename [

] = 'Skype'

2. Store Images in runtime img = capture() Filename [img] = input("Please enter Name to Captured image") 3. Retrieve name of the Image on popup. Way 1: query = capture() gotit = Filename[query][0] popup(gotit) Way 2:

popup("select a icon in the ") query = capture() Result = Filename [query] if result: popup(Result[0]) else: popup("unknown Image")

Open any application in run time. setThrowException(True) setAutoWaitTimeout(10000) switchApp("C:\WINDOWS\system32\calc.exe")

Sikuli
Empty Recycle Bin:

Satish Gorripotu

Run Sikuli from Command Line: PATH-TO-SIKULI/sikuli-ide.exe [options] or PATH-TO-SIKULI/sikuli-ide.bat [options] Command Line Options Sikuli-IDE --args <arguments> specify the arguments passed to Jythons sys.argv -h,--help print this help message

Sikuli

Satish Gorripotu
usage: Sikuli-IDE [--args <arguments>] [-h] [-r <sikuli-file>] [-s] [-t <sikuli-file>] --args <arguments> specify the arguments passed to Jython's sys.argv -h,--help print this help message -r,--run <sikuli-file> run .sikuli or .skl file -s,--stderr print runtime errors to stderr instead of popping up a message box -t,--test <sikuli-file> run .sikuli as a unit test case with junit's text UI runner

-r,--run <sikuli-file> run .sikuli or .skl file -s,--stderr print runtime errors to stderr instead of popping up a message box -t,--test <sikuli-file> run a .sikuli (.skl not possible) as a unit test with junits text UI runner (A script that can also be run in IDEs Unit Test View). Example - Run xxxx.sikuli with three arguments: PATH-TO-SIKULIsikuli-ide.exe -r xxxx.sikuli args a1 a2 a3 Running sikuli-script.jar from Command line Run a script without IDE java -jar %SIKULI_HOME%\sikuli-script.jar path-to-your-script\yourScript.sikuli

Run a script from Command line using the Sikuli contained Jython This option might be helpful in some cases, where you want to have access to the Jython layer, before any Sikuli feature is touched. Be aware, that a from sikuli.Sikuli import * is needed to have access to Sikuli. You might have to take care somehow, that images are found, since bundle path is not set. set SCRIPT=path-to-your-script\yourScript.sikuli\yourScript.py java -cp %SIKULI_HOME%\sikuli-script.jar org.python.util.jython %SCRIPT% Interactive Sikuli Jython Session from Command Line (Option -i) sikuli-script.jar -i Start an interactive Jython session with the Sikuli environment already in place. This might be helpful to do some testing on commandline, without having to start the Sikuli IDE. A specified script and other options will be ignored. java -jar %SIKULI_HOME%\sikuli-script.jar -i Create Unit Testing Scripts for GUI:

Sikuli

Satish Gorripotu

Sikuli integrates with jUnit and supports unit testing for Graphical User Interfaces (GUI). The unit testing panel can be opened by clicking the menu View Unit Test or by the hot key -U on Mac or Ctrl-U on Windows/Linux. Sikuli IDE aims to minimize the effort of writing code. With Sikuli IDE, a Python class inherited from junit.framework.TestCase is automatically generated to wrap your unit testing script. A typical unit testing script consists of two constructing and destructing methods, setUp() and tearDown(), and a bunch of methods named with a prefix test. The basic structure of a script is given as following: TestJEdit.sikuli def setUp(self): openApp("/Applications/jEdit.app") wait( close = ( # if find(close): # click(close) , 20000) )

def tearDown(self): closeApp("jEdit.app") untilNotExist( , 20000)

def test_textarea_add_del_by_key(self): type("hello world") assertExist( type("a",KEY_CTRL) type("\n") assertNotExist( ) )

def test_textarea_add_del_by_menu(self): type("hello world") assertExist( click( click( type("\n") assertNotExist( ) ) ) )

def test_new_tab_by_key(self):

Sikuli
type("n",KEY_CTRL) assertExist(Pattern( def test_new_tab_by_menu(self): click( click( ) ) ).similar(0.95)) ).similar(0.95))

Satish Gorripotu

assertExist(Pattern( def test_new_tab_by_toolbar(self): click( )

assertExist(Pattern(

).similar(0.95))

def _verify_find(self): sleep(0.5) type("hello") type("\n")

click(Pattern( assertExist(Pattern( type("\b") assertExist(

).similar(0.90)) ).similar(0.94))

def test_find_by_key(self): type("a long hello world\n") type("f",KEY_CTRL) self._verify_find() def test_find_by_menu(self): type("a long hello world\n") click( click( ) )

self._verify_find()

Sikuli
def test_find_by_toolbar(self): type("a long hello world\n") click( self._verify_find() )

Satish Gorripotu

def test_textfield_on_toolbar(self): type("a long hello world\n") assertExist( type("f",KEY_CTRL) assertExist(Pattern( type("hello") ).similar(0.85)) )

assertExist(

def test_toolbar_print_dialog(self):

click(

assertExist(

click(

def test_menu_submenu(self): click( click( ) )

assertExist( type("test")

click( click( )

Sikuli
click( click( click( click( assertExist( def test_scrollbar(self): for i in range(1,50): type("line %d\n" % i) tail = Pattern( wait(tail, 10000) assertExist(tail) ).similar(0.92) ) ) ) ) )

Satish Gorripotu

thumb_at_top = Pattern( assertNotExist(thumb_at_top)

).similar(0.98)

dragDrop(

, [0, -1000])

assertExist(thumb_at_top) assertNotExist(tail) assertExist(Pattern( ).similar(0.92))

dragDrop(Pattern( assertExist(tail) def test_quit(self): type("test") type("e",KEY_CTRL) type("q",KEY_CTRL)

).similar(0.90), [0, 1000])

click(

Sikuli

Satish Gorripotu

assertExist(Pattern(

).similar(0.90))

To run a unit testing script, you need to click on the Run button in the unit testing panel instead of the ordinary button. IMPORTANT: Before you try to run your script in this test mode the first time, it has to be saved. Everytime you change something, you have to save it again, before the next test run. Sikuli Script in JAVA programs: The core of Sikuli Script is written in Java, which means you can use Sikuli Script as a standard JAVA library in your program. This document lets you know how to do that. After having installed Sikuli on your system, as recommended on the download page, you have to do the following: 1. Get sikuli-script.jar from your Sikuli IDE installation path. Sikuli-IDE/sikuli-script.jar 2. Make the native libraries available 3. Include sikuli-script.jar in the CLASSPATH of your Java project. 4. Import the Sikuli classes you need You can simply import org.sikuli.script.* or import the classes you need. In most cases, you would need at least Region or Screen. the package name was edu.mit.csail.uid. Sikuli together with other IDEs - Eclipse: Install Python plugin (PyDev) and Jython Since the top level language used by Sikuli script is Python, you need the Python plugin PyDev. Different from NetBeans, though PyDev is prepared to use Jython as interpreter, you have to install Jython on your own seperately. So first install Jython, by downloading the version you want to use from the http://www.jython.org/downloads.html Sikuli currently is based on Jython 2.5.1, but you may choose Jython 2.5.2 as well. Install it according to Jythons installation HowTo. After installation, make a test from a commandline by typing jython.bat (Windows) or jython (Mac/Linux) to open an interactive Jython session. You might type the following (<enter> means pressing the enter/return key) import os <enter> import sys <enter> for e in sys.path: print e <enter> <enter> type exit() <enter> (to leave interactive Jython)

Sikuli

Satish Gorripotu

This shows the current Jython configuration of the Python path. Remember the place where Jython is installed. Now we install the PyDev plugin from inside Eclipse: Menu Help -> Eclipse Marketplace. Either search it or find it on the Popular tabs lower part. Simply click the Install button and do what you are asked to do ;-) The next step is to tell Eclipse PyDev, where it can find the Jython interpreter. Navigate to the Eclipse Preferences pane and open the category PyDev and inside the subcategory Interpreter-Jython. First try to Auto Config by clicking the appropriate button. If this does not work, click the button New, name the entry and navigate to the folder, where Jython was installed and select jython.jar. After clicking ok, a window Selection needed might come up: click Select all and ok to finalize this step. Other options with PyDev are available, but not relevant for our actual matter http://pydev.org/manual_101_root.html Configure for using Sikuli script features at runtime Again we have a difference to Netbeans: The PyDev plugin does not allow library references to folders inside of jar-files in the respective configuration dialog (it does not insert them to the Python path), though Jython itself accepts them, when specified on the Python path. So if you want to run and debug your script in Eclipse, you have to extract the folder Lib/sikuli from sikuli-script.jar Menu File -> New -> Project -> PyDev -> (open sublist) -> PyDev Project -> click Next button. On the configuration pane name your project, select Jython as Project type, grammar version 2.5 (higher Python language versions are not supported by Jython 2.5.x) and click the Finish button. Your project is created. Add at least one you-name-it.py file to the source folder and put from sikuli.Sikuli import * as the first line. In the last step, we tell PyDev, where to find the Sikuli libraries. Goto Menu Project -> Properties -> select category PyDev - PYTHONPATH and go to the tab External Libraries. We need a reference to path-to/sikuli-script.jar and another one to the extracted folder Lib containing the folder sikuli. reference to path-to/sikuli-script.jar o click button Add zip/jar/egg and select sikuli-script.jar from the Sikuli installation.

reference to the extracted folder Lib containing the folder sikuli o o click Add source folder and select the folder Lib in the place you had it extracted. this is not needed, if you have moved the extracted folder sikuli to a folder, that is already on the Python path (e.g. jython-intallation/Lib/site-packages). Now you are able to run your first script. Remember, that in every script including the main script, that you are editing now, as the first line you need from sikuli.Sikuli import *, to have access to the Sikuli features at runtime.

Sikuli

Satish Gorripotu

Everytime later on you might come back to the projects preferences with Project -> Properties. Code Completion works from the start without any further configuration and even steps into the Java classes where appropriate. Configuration Eclipse: 1) Open the Eclipse by double clicking on eclipse.exe, Click on OK for workspace launcher. Click on workbench, it will allow you to move inside the eclipse.

2) Click on File->New->Select Java Project.

Sikuli

Satish Gorripotu

3) Enter the Project name and then click on Configue JREs

Sikuli

Satish Gorripotu

Improve the performance of Sikuli from org.sikuli.script.natives import Vision Vision.setParameter("MinTargetSize", 6) Import Packages/ Files All files must be in Sys path. So Sikuli will find the files automatically, if we give it as from sikuli import * Example: myScriptPath = "c:\\SikuliFiles\\myLibrary" if not myScriptPath in sys.path: sys.path.append(myScriptPath) # supposing there is a myLib.sikuli import myLib # supposing myLib.sikuli contains a function "def myFunction():" myLib.myFunction() # makes the call

As Python load the file at once and will continue the library to entire script. If we modify the code in library file, then Python wouldnt reflect the changes. So, # instead of: import module

Sikuli
import module reload(module) # instead of: from module import * import module reload(module) from module import *

Satish Gorripotu

myLib.myFunction() or myLib import * # which integrates all names from myLib into your current namespace. myFunction() # We can use directly

Importing from the directory: import os # get the directory containing your running .sikuli myPath = os.path.dirname(getBundlePath()) if not myPath in sys.path: sys.path.append(myPath) # now you can import every .sikuli in the same directory import myLib Scripts in same directory are found automatically # nothing else needed # now you can import every .sikuli in the same directory import myLib

load(jar-file) : with path or no path if no path is specified, Sikuli first looks into the bundle (the Sikuli folder of the running script) and then into the extensions folder. Controlling Sikuli Scripts and their Behavior: setShowActions(False | True) True, when a script is run, Sikuli shows a visual effect (a blinking double lined red circle) on the spot. exit([value]) Stops the script gracefully at this point. The value is returned to the calling environment.

Setting is a class available in Sikuli Package.

Sikuli
This class has several methods Class Setting { 1. ActionLogs (True/False) will display the log messages in console 2. InfoLogs (True/False) will display the Info log messages in console 3. DebugLogs (True/False) will display the Debug log messages in console

Satish Gorripotu

4. MinSimilarity Sikuli will search for the minimal similarity of the object in the screen. Like smart Identification 5. MoveMouseDelay Control the mouse move animation. mmd = Settings.MoveMouseDelay # save default/actual value click(image1) # implicitly wait 0.5 seconds before click Settings.MoveMouseDelay = 3 click(image2) # give app 3 seconds time before clicking again Settings.MoveMouseDelay = mmd # reset to original value

6. DelayAfterDrag Specifies the waiting time after mouse down at the source location as a decimal value (seconds). 7. DelayBeforeDrop Specifies the waiting time before mouse up at the target location as a decimal value (seconds).

Above DelayAfterDrag, DelayBeforeDrop can be used as alternative of Wait Time. # you may wish to save the actual settings before Settings.DelayAfterDrag = 1 Settings.DelayBeforeDrop = 1 Settings.MoveMouseDelay = 3 dragDrop(source_image, target_image) # time for complete dragDrop: about 5 seconds + search times 8. SlowMotionDelay Control the duration of the visual effect (seconds). 9. WaitScanRate 10.ObserveScanRate Specify the number of times actual search operations are performed per second while waiting for a pattern to appear or vanish. Example: Region.wait(), Region.exists(), Region.waitVanish(), Region.observe()

Sikuli

Satish Gorripotu

def myHandler(e): print "it happened" # you may wish to save the actual settings before Settings.ObserveScanRate = 0.2 onAppear(some_image, myHandler) observe(FOREVER, background = True) # the observer will look every 5 seconds # since your script does not wait here, you # might want to stop the observing later on ;-) 11.ObserveMinChangedPixels The minimum size in pixels of a change to trigger a change event. Ex: Region.onChange() without specifying this value. The default value is 50. }

Controlling Applications and their Windows: Note: backslashes \ and double apostrophes In a Sikuli script in normal strings enclosed in (double apostrophes), these special characters \ and have to be escaped using a backslash, when you have them inside the string. So for one backslash you need \\ and for one you need \. In a string enclosed in (single apostrophes), a has to be \ and a is taken as such. To avoid any problems, it is recommended to use the raw string r'some text with \\ and " ...', since there is no need for escaping. This is especially useful, when you have to specify Windows paths or want to setup command lines for use with App.open(), openApp(), os.popen or Jythons Subprocess module. a fictive command line example: cmd = r'c:\Program Files\myapp.exe -x "c:\Some Place\some.txt" >..\log.txt' openApp(cmd) Open an application Switch to an application Close an application : os.open --> openApp() > App.open() : switchApp() > App.focus() : closeApp() > App.close()

Sikuli
openApp(Application) Deprecated. Now it is App.open()

Satish Gorripotu

This function may switch to an already opened application or may open a new instance of the application.

# opens command prompt (found through PATH) openApp("cmd.exe") # with parameters (no sense, only to show ;-) openApp(r'cmd.exe /c start c:\Program Files\myapp.bat') # opens Firefox (full path specified) openApp("c:\\Program Files\\Mozilla Firefox\\firefox.exe") switchApp(application) Deprecated. Now it is App.focus() This function switches the focus to the specified application and brings its windows to the front. The window is identified by the application string. This string (needs not be an applications name) is used to search the title text of all the opened windows for any part of the title matching the string. Example: # switches to an existing command prompt or starts a new one switchApp("cmd.exe") # opens a new browser window switchApp("c:\\Program Files\\Mozilla Firefox\\firefox.exe") # switches to the frontmost opened browser window (or does nothing if no browser window is currently opened) switchApp("mozilla firefox") closeApp(application) Deprecated. Now it is App.close(). Close the specified application. Parameters: application the name of an application or part of a window title This function closes the application indicated by the windows whose titles contain the string application. Example: # closes an existing command prompt closeApp("cmd.exe") # does nothing, since text cannot be found in the window title closeApp("c:\\Program Files\\Mozilla Firefox\\firefox.exe") # stops firefox including all its windows

Sikuli
closeApp("mozilla firefox") run(command) Run command in the command line

Satish Gorripotu

Parameters: command a command that can be run from the command line. This function executes the command and the script waits for its completion. Note: Windows: Sikulis strategy on these systems in the moment is to rely on implicit or explicit path specifications to find an application that has to be started. Running applications can either be identified using their PID (process ID) or by using the window titles. So using a path specification will only switch to an open application, if the application internally handles the more than one instance situation. We usually use App.open("c:\\Program Files\\Mozilla Firefox\\Firefox.exe") to start Firefox. This might open an additional window. And you can use App.focus("Firefox") to switch to the front most Firefox window Note: When specifying a path in a string, we have to use \ (double backslash) for each (backslash) e.g. myPath = "c:\\Program Files\\Sikuli-IDE\\Lib\\" ) App is a class available in Sikuli Package. This class has several methods. We can use App class methods in two ways 1. 2. class methods instance methods

Class App { 1. Open Open the specified application. It opens the specified application and brings its window the front. Whether this operation switches to an already opened application or opens a new instance of the application depends on the system and application 1. Class Method : open(application) Usage : App.open("application-identifier")

2. Instance method : open() # App instance someApp was created before Usage : myApp = App("application-identifier") myApp.open() 2. Focus Switch the focus to an application. Parameters are the name of an application or part of a window title 1. Class Method Usage : focus (application) : App.focus("application-identifier")

Sikuli

Satish Gorripotu
2. Instance method : open() # App instance someApp was created before Usage : myApp = App("application-identifier") myApp. focus () 3. Close It closes the given application or the matching windows. It does nothing if no opened window or running application can be found. Whether the application itself is closed depends on whether all open windows are closed Or a main window of the application is closed, that in turn closes all other opened windows.

1. Class Method : close (application) Usage : App. close ("application-identifier")

2. Instance method : close () # App instance someApp was created before Usage : myApp = App("application-identifier") myApp. close()

4. focusedWindow Identify the currently focused or the front most window and switch to it. Sikuli does not tell you, to which application this window belongs. Region object representing the window or None if there is no such window. This method always returns a region. When there is no window opened on the desktop, the region may refer to a special window such as the task bar or an icon in the system tray. 1. Class Method : focusedWindow (application) Usage : App. focusedWindow ("application-identifier")

2. Instance method : close () # App instance someApp was created before Usage Example: # highlight the currently fontmost window for 2 seconds App.focusedWindow().highlight(2) # save the windows region before firstWindow = App.focusedWindow() firstWindow.highlight(2) : myApp = App("application-identifier") myApp. focusedWindow ()

5. window([n])

Sikuli
Get the series of windows with the matching title.

Satish Gorripotu

1. Class Method : App(application).window([n])#an App instance is created on the fly. Usage : App(application).window([n])

2. Instance method : close () # App instance someApp was created before Usage Parameters: Returns Example : #opens a Firefox browser window and switches to the address field. # using an existing window if possible myApp = App("Firefox") if not myApp.window(): # no window(0) - Firefox not open App.open("c:\\Program Files\\Mozilla Firefox\\Firefox.exe") wait(2) myApp.focus() wait(1) type("l", KEY_CTRL) # switch to address field Afterwards, it focuses on the Firefox application, uses the window () method to obtain the region of the front most window, Applies some operations within the region, And finally closes the window: # using a new window firefox = App.open("c:\\Program Files\\Mozilla Firefox\\Firefox.exe"); wait(2) firefox.focus() wait(1) # now your just opened new window should be the frontmost with firefox.window(): # see the general notes below # some actions inside the window(0)'s region click("somebutton.png") firefox.close() # close the window - stop the process Below is another example that highlights all the windows of an application by looping through them # not more than 100 windows should be open ;-) myApp = App("Safari") for n in range(100): w = myApp.window(n) if not w: break # no more windows w.highlight(2) # window highlighted for 2 second : myApp = App("application-identifier") myApp.window([n]) n 0 or a positive integer number. If ommitted, 0 is taken as default. : the region on the screen occupied by the window, if such window exists and None if otherwise.

Sikuli

Satish Gorripotu

Interacting with the User popup(text[, title]) Display a dialog box with an OK button and text as the message. The script then waits for the user to click the OK button. Parameters: Example: popup("Hi satish.\n How are you") text text to be displayed as message title optional title for the messagebox

input([text][, default]) Display a dialog box with an input field, a Cancel button, and an OK button. The optional text can be displayed as a caption. The script then waits for the user to click either the Cancel or the OK button. Parameters: text optional text to be displayed as message default optional preset text for the input field Returns: the text, contained in the input field, when the user clicked OK None, if the user pressed the Cancel button Example: name = input("Please enter your name:") name = input("Please enter your name to log in:", "anonymous") # a preset input text When using the parameter default, the text input field will be pre-populated with the given text, So we click OK/Cancel or edit the content of the input field

Sikuli

Satish Gorripotu

Listening to Global Hotkeys Env.addHotkey(key, modifiers, handler) Register the specified key + modifiers as a global hotkey. When the hotkey is pressed, the specified function handler will be called. Parameters: key a character or a constant value defined in Key. modifiers Key modifiers, which can be one or multiple constants defined in KeyModifier. def openAppleMenu(event): click(Any Image) # When we pressed Ctrl+Alt+F1, click the top-left apple icon. Env.addHotkey(Key.F1, KeyModifier.ALT+KeyModifier.CTRL, openAppleMenu) Env.removeHotkey(key, modifiers) Unregister the registered global hotkey key + modifiers. Parameters: key a character or a constant value defined in Key. modifiers Key modifiers, which can be one or multiple constants defined in KeyModifier. General Settings and Access to Environment Information Sikuli Level Sikuli internally uses the class Settings to store globally used settings. Publicly available attributes accessed by using Settings.[name-of-an-attribute] To get its value and Settings.attribute = value to set it. Jython/Python Level We can use all settings defined in standard Python/Jython which are available in our system environment. The modules sys and time are already imported, so you can use their methods without the need for an import statement. sys.path may be one of the most valuable settings, since it is used by Python/Jython to locate modules, that are referenced using import module. myPath = "some-absolute-path" if not myPath in sys.path: sys.path.append(myPath) Java Level import java

Sikuli
# get a value val = java.lang.System.getProperty("key-of-property") # set a property's value java.lang.System.getProperty("key-of-property", value)

Satish Gorripotu

Image Search Path Sikuli maintains a list of locations to search for images when they are not found in the current .sikuli folder (a.k.a. bundle path). This list is maintained internally but can be inspected and modified. getImagePath() Get a list of paths where Sikuli will search for images. # getImagePath() returns a Java array of unicode strings imgPath = list(getImagePath()) # makes it a Python list # to loop through for p in imgPath: print p addImagePath(a-new-path) Add a new path to the list of image search paths removeImagePath(a-path-already-in-the-list) Remove a path from the list of image search paths Note: paths must be specified using the correct path separators as double blackslashes . Default bundle path can accessed and modified. setBundlePath(path-to-a-folder) Set the path for searching images in all Sikuli Script methods. Sikuli IDE sets this automatically to the path of the folder where it saves the script (.sikuli). getBundlePath() Get a string containing a path to a folder containing our images used for finding patterns. Sikuli IDE sets this automatically to the path of the folder where it saves the script (.sikuli). We can use this function to package our private files together with the script or to access the picture files in the .sikuli bundles for other purposes. Sikuli only gives access to the path name, We need other python modules for I/O or other purposes. Other Environment Information Env.getOS() Env.getOSVersion() Env.getSikuliVersion() Example: if not Env.getSikuliVersion() == "X-1.0rc2":

Sikuli
print "This script needs Sikuli X-1.0rc2" exit(1) Env.getClipboard() Get the content of the clipboard if it is text, otherwise an empty string. Env.getClipboard().paste() clipboard will contain what we just pasted. Env.getClipboard().strip() To get rid of surrounding white spaces.

Satish Gorripotu

Env.isLockOn(key-constant) Get the current status ( on / off ) off the respective key. Only one key can be specified. key-constant one of the key constants Key.CAPS_LOCK, Key.NUM_LOCK, Key.SCROLL_LOCK Env.getMouseLocation() Get the current location of the mouse cursor. Advanced Settings for Tuning Vision Algorithm Vision.setParameter(param, value) Vision.getParameter(param) Get/Set the parameter param of the vision algorithm. Parameters: param a string that indicates the parameter to get. Returns: The float value of the specified parameter.

MinTargetSize MinTargetSize is the minimum image size to which Sikuli can resize. Sikuli resizes the screen images to a smaller scale for faster matching. This scaling process speeds up the matching process, but also increases the possibility of false matching. The default value of MinTargetSize is 12, < 12. May increase speed but may returns a match that is not what we expect. Example Jython code. from org.sikuli.script.natives import Vision Vision.setParameter("MinTargetSize", 6) # the default is 12. Setting the size to a smaller value would make the matching algorithm be faster.

Sikuli

Satish Gorripotu

Region Region is a class available in Sikuli Package. This class has several methods Class Region { class Region Region is a rectangular area on a screen. It knows nothing about its visual content (windows, pictures, graphics, text, ...). It only knows the position on the screen and its dimension. Creating a Region, Setting and Getting Attributes Region(x, y, w, h) Region(region) Region(Rectangle)

Sikuli
Parameters:

Satish Gorripotu
x x position of a rectangle. #upper left corner of the screen (0, 0) y y position of a rectangle. #upper left corner of the screen (0, 0) w height of a rectangle. h width of a rectangle. region an existing region object. rectangle an existing object of Java class Rectangle Returns: a new region object.

region can also be created in run-time using Region.selectRegion(). You can create a region based on another region. This just duplicates the region (new object). Another way to create a region is to specify a rectangle object or to extend an existing region. Note: topLeft = Location(reg.x, reg.y) # equivalent to topLeft = reg.getTopLeft() theWidth = reg.w # equivalent to theWidth = reg.getW() reg.h = theWidth # equivalent to reg.setH(theWidth) We can use selectRegion() to interactively create a new region.

setX(number) setY(number) setW(number) setH(number) Set the respective attribute of the region to the new value. This effectively moves the region around and/or changes its dimension. moveTo(location) Set the position of this region regarding its top left corner to the given location (the x and y values are modified). Parameters Returns : location location object becomes the new top left corner : the modified region object

Example: reg.moveTo(anotherLocation) # equivalent to reg.setX(anotherLocation.x); reg.setY(anotherLocation.y) setROI(x, y, w, h) setROI(rectangle) setRect(x, y, w, h)

Sikuli
setRect(rectangle)

Satish Gorripotu

Both methods do the same: setting position and dimension to new values. setROI() is intended to restrict the search to a smaller area to speed up processing setRect() should be used to redefine a region (which could be enlarging it). Parameters: x, y, w, h (all) the attributes of a rectangle rectangle a rectangle object . morphTo(region) Set the position and dimension of this region to the corresponding values of the region given as parameter. Parameters:region a region object Returns: the modified region object Note: reg.morphTo(anotherRegion) # equivalent to r = anotherRegion; reg.setX(r.x); reg.setY(r.y); reg.setW(r.w); reg.setH(r.h)

getX() getY() getW() getH() Get the respective attribute integer value of the region. getCenter() Get the center of the region or Location as an object. getTopLeft() getTopRight() getBottomLeft() getBottomRight() Get the location object of the regions respective corner getScreen() Returns the screen object that contains this region. This method makes sense in Multi Monitor Environments getLastMatch() getLastMatches() Returns the best match as a Match object or one or more match objects as an Iterator object respectively setAutoWaitTimeout(seconds) Set the maximum waiting time for all subsequent find operations.

Sikuli

Satish Gorripotu

This method enables all find operations to wait for the given pattern to appear until the specified amount of time has elapsed. The default is 3.0 seconds. This method is intended for users to override this default setting. Region.find() work like Region.wait(), without being able to set an individual timeout value for a specific find operation. getAutoWaitTimeout() Get the current value of the maximum waiting time for find operations. Extending a Region Note: In all cases the current region remains unchanged. In all cases the new region does not extend beyond any boundary of the screen that contains the given region. offset(location) Returns a new Region object, whose upper left corner is relocated adding the locations x and y value to the respective values of the given region. Width and height are the same. So this clones a region at a different place.

Parameters Returns

: location a Location object :a new Region object

Example: new_reg = reg.offset(Location(xoff, yoff)) # same as new_reg = Region(reg.x + xoff, reg.y + yoff, reg.w, reg.h) inside() Returns the same Region object. Retained for upward compatibility. region.inside().find() is totally equivalent to region.find().

nearby([range]) Returns a new Region object that includes the nearby neighbourhood of the the current region. The new region is defined by extending the current regions dimensions in all directions by range number of pixels. The center of the new region remains the same.

Sikuli

Satish Gorripotu

Parameters: range a positive integer indicating the number of pixels (default = 50). above([range]) Returns a new Region object that is defined above the current regions top border with a height of range number of pixels. It does not include the current region. If range is omitted, it reaches to the top of the screen. The new region has the same width and xposition as the current region. Parameters:range a positive integer defining the new height below([range]) Returns a new Region object that is defined below the current regions bottom border with a height of range number of pixels. It does not include the current region. If range is omitted, it reaches to the bottom of the screen. The new region has the same width and x-position as the current region. Parameters:range a positive integer defining the new height left([range]) Returns a new Region object that is defined left of the current regions left border with a width of range number of pixels. It does not include the current region. If range is omitted, it reaches to the left border of the screen. The new region has the same height and y-position as the current region. Parameters:range a positive integer defining the new width right([range]) Returns a new Region object that is defined right of the current regions right border with a width of range number of pixels. So it does not include the current region. If range is omitted, it reaches to the right border of the screen. The new region has the same height and y-position as the current region. Parameters:range a positive integer defining the new width Finding inside a Region and Waiting for a Visual Event Besides acting on visual objects, finding them is one of the core functions of Sikuli. PS: means, that either a Pattern or a string (path to an image file or just plain text) can be used as parameter. find(PS) Find a particular GUI element, which is seen as the given image or just plain text. The given file name of an image specifies the elements appearance. It searches within the region and returns the best match, which shows a similarity greater than the minimum similarity given by the pattern. Parameters:PS a Pattern object or a string (path to an image file or just plain text) Returns: a Match object that contains the best match or fails if not found findAll(PS)

Sikuli

Satish Gorripotu

Repeatedly find ALL instances of a pattern, until no match can be found anymore, that meets the requirements for a single Region.find() with the specified pattern. Parameters:PS a Pattern object or a string (path to an image file or just plain text) Returns :one or more Match objects as an iterator object or fails if not found By default, the returned matches are sorted by the similiarty. If you need them ordered by their positions, we have to use Pythons sorted function.

Example: def by_y(match): return match.y icons = findAll(Image screen shot) # sort the icons by their y coordinates and put them into a new variable sorted_icons sorted_icons = sorted(icons, key=by_y) # another shorter version is using lambda. sorted_icons = sorted(icons, key=lambda m:m.y) for icon in sorted_icons: pass # do whatever you want to do with the sorted icons wait([PS][, seconds]) a Match object that contains the best match or fails if not found Parameters: PS a Pattern object or a string (path to an image file or just plain text) seconds a number, which can have a fraction, as maximum waiting time in seconds. If not specified, the auto wait timeout value set by Region.setAutoWaitTimeout() is used. Use the constant FOREVER to wait for an infinite time. If PS is not specified, the script just pauses for the specified amount of time. It is still possible to use sleep(seconds) instead, but this is deprecated. If PS is specified, it keeps searching the given pattern in the region until the image appears ( would have been found with Region.find()) or the specified amount of time has elapsed. At least one find operation is performed, even if 0 seconds is specified.) waitVanish(PS[, seconds]) Wait until the give pattern PS in the region vanishes. Returns True if the pattern vanishes within the specified waiting time, or False if the pattern stays visible after the waiting time has elapsed. This method keeps searching the given pattern in the region until the image vanishes (can not be found with Region.find() any longer) or the specified amount of time has elapsed. At least one find operation is performed, even if 0 seconds is specified Parameters: PS a Pattern object or a string (path to an image file or just plain text) seconds a number, which can have a fraction, as maximum waiting time in seconds. The internal granularity is milliseconds. If not specified, the auto wait timeout value set by Region. setAutoWaitTimeout() is used. Use the constant FOREVER to wait for an infinite time.

Sikuli

Satish Gorripotu

. exists(PS[, seconds]) Check whether the give pattern is visible on the screen. And returns a Match object that contains the best match. None is returned, if nothing is found within the specified waiting time Does exactly the same as Region.wait(), but no exception is raised in case of FindFailed. It can be used to symplify scripting in case that we want to know wether something is there or not to decide how to proceed in your workflow. It is typically used with an if statement. At least one find operation is performed, even if 0 seconds is specified. Parameters: PS a Pattern object or a string (path to an image file or just plain text) seconds a number, which can have a fraction, as maximum waiting time in seconds. The internal granularity is milliseconds. If not specified, the auto wait timeout value set by Region.setAutoWaitTimeout() is used. Use the constant FOREVER to wait for an infinite time. Observing Visual Events in a Region handler: as a parameter in the following methods, you have to specify the name of a function, which will be called by the observer, in case the observed event happens. The function itself has to be defined in your script before using the method that references the function. The existance of the function will be checked before starting the script. So to get your script running, you have to have at least the following statements in your script: def myHandler(event): # you can choose any valid function name # event: can be any variable name, it references the SikuliEvent object pass # add your statements here onAppear("path-to-an-image-file", myHandler) # or any other onEvent() observe(10) # observes for 10 seconds onAppear(PS, handler) With the given region you register an observer, that should wait for the pattern to be there or to appear and is activated with the next call of observe(). In the moment the internal find operation on the given pattern is successful during observation, your handler is called and the observation is paused until you return from your handler. Parameters: PS a Pattern object or a string (path to an image file or just plain text. handler the name of a handler function in the script

Sikuli

Satish Gorripotu

onVanish(PS, handler) With the given region you register an observer, that should wait for the pattern to be not there or to vanish and is activated with the next call of observe(). In the moment the internal find operation on the given pattern fails during observation, your handler is called and the observation is paused until you return from your handler. Parameters: PS a Pattern object or a string (path to an image file or just plain text. handler the name of a handler function in the script

onChange([minChangedSize], handler) Parameters: minChangedSize the minimum size in pixels of a change to trigger a change event. If omitted: 50 is used handler the name of a handler function in the script minChangedSize With the given region we register an observer, that should wait for the visual content of the given region to change and is activated with the next call of observe(). In the moment the visual content changes during observation, our handler is called and the observation is paused until you return from your handler. Below example highlights all changes in an observed region. def changed(event): print "something changed in ", event.region for ch in event.changes: ch.highlight() # highlight all changes sleep(1) for ch in event.changes: ch.highlight() # turn off the highlights with selectRegion("select a region to observe") as r: # any change in r larger than 50 pixels would trigger the changed function onChange(50, changed) observe(background=True) wait(30) # another way to observe for 30 seconds r.stopObserver()

observe([seconds][, background = False | True]) Begin observation within the region. For each region object, only one observation can be running at a given time.

Sikuli
Parameters:

Satish Gorripotu

seconds a number, which can have a fraction, as maximum observation time in seconds. Omit it or use the constant FOREVER to tell the observation to run for an infinite time (or until stopped by a call of stopObserve()). background a flag indicating whether observation is run in the background. when set to True, the observation will be run in the background and processing of your script is continued immediately. Otherwise the script is paused until the completion of the observation. stopObserver() Stop observation within the region. This must be called on a valid region object. The source region of an observed visual event is available as one of the attributes of the event parameter that is passed to the handler function when the function is invoked. Example: to stop observation within a handler function, call event.region.stopObserver() inside the handler function. def myHandler(event): event.region.stopObserver() # stops the observation onAppear("path-to-an-image-file", myHandler) observe(FOREVER) # observes until stopped in handler }

SikuliEvent SikuliEvent is a class available in Sikuli Package. This class has several methods Class SikuliEvent {

Sikuli

Satish Gorripotu

When processing an observation in a region, a handler function is called, when one of the registered events Region.onAppear(), Region.onVanish() or Region.onChange() One parameter, we have access to in handler function is an instance of Class SikuliEvent. Following attributes of the event, might help to identify the cause of the event and act on the resulting matches. Usage: event.attribute where event is the parameter from the definition of handler function: def myHandler(event): type SikuliEvent.Type.APPEAR, SikuliEvent.Type.VANISH, or SikuliEvent.Type.CHANGE. pattern The source pattern that triggered this event. This is only valid in APPEAR and VANISH events. region The source region of this event. match In an APPEAR event, match saves the top Match object that appeared in the observed region. In a VANISH event, match saves the last Match object that was in the observed region but vanished. This attribute is not valid in a CHANGE event. changes This attribute is valid only in a CHANGE event. This changes attribute is a list of Match objects that changed and their sizes are at least minChangedSize pixels. }

Acting on a Region PSMRL: which means, that either a Pattern object or a string (path to an image file or just plain text) or a Match or a Region or a Location can be used as parameter. P: pattern: a Pattern object. An implicit find operation is processed first. If successful, the center of the resulting matches rectangle is the click point. If the pattern object has a target offset specified, this is used as click point instead. S: string: a path to an image file or just plain text. An implicit find operation with the default minimum similarity 0.7 is processed first. If successful, the center of the resulting match object is the click point.

Sikuli

Satish Gorripotu

M: match: a match object from a previous find operation. If the match has a target specified it is used as the click point, otherwise the center of the matches rectangle. R: region: a region object whose center is used as click point. L: location: a location object which by definition represents a point on the screen that is used as click point. Region Region is a class available in Sikuli Package. This class has several methods continuation to above methods. Class Region { click(PSMRL[, modifiers]) Perform a mouse click on the click point using the left button. Parameters: PSMRL a pattern, a string, a match, a region or a location that evaluates to a click point. modifiers one or more key modifiers Returns: Number of performed clicks (actually 1). A 0 (integer null) means that because of some reason, no click could be performed (in case of PS may be not Found). doubleClick(PSMRL[, modifiers]) Perform a mouse double-click on the click point using the left button. Parameters: PSMRL a pattern, a string, a match, a region or a location that evaluates to a click point. modifiers one or more key modifiers Returns: Number of performed double-clicks (actually 1). A 0 (integer null) means that because of some reason, no click could be performed (in case of PS may be not Found). rightClick(PSMRL[, modifiers]) Perform a mouse click on the click point using the right button. Parameters: PSMRL a pattern, a string, a match, a region or a location that evaluates to a click point. modifiers one or more key modifiers Returns: Number of performed right cilicks (actually 1). A 0 (integer null) means that because of some reason, no click could be performed (in case of PS may be not Found). highlight([seconds]) Highlight the region for some period of time. Parameters:seconds a decimal number taken as duration in seconds The region is highlighted showing a red colored frame around it. If the parameter seconds is given, the script is suspended for the specified time. If no time is given, the highlighting is started and the script continues.

Sikuli

Satish Gorripotu

When later on the same highlight call without a parameter is made, the highlighting is stopped (behaves like a toggling switch).

Example: m = find(some_image) # the red frame will blink for about 7 - 8 seconds for i in range(5): m.highlight(1) wait(0.5) # a second red frame will blink as an overlay to the first one m.highlight() for i in range(5): m.highlight(1) wait(0.5) m.highlight() # the red frame will grow 5 times for i in range(5): m.highlight(1) m = m.nearby(20) hover(PSMRL) Move the mouse cursor to hover above a click point. Parameters: PSMRL a pattern, a string, a match, a region or a location that evaluates to a click point. modifiers one or more key modifiers Returns: The number 1 if the mousepointer could be moved to the click point. A 0 (integer null) returned if no move could be performed (in case of PS may be not Found). dragDrop(PSMRL, PSMRL[, modifiers]) Perform a drag-and-drop operation from a starting click point to the target click point indicated by the two PSMRLs respectively. Parameters: PSMRL modifiers a pattern, a string, a match, a region or a location that evaluates to a click point. one or more key modifiers

If one of the parameters is PS, the operation might fail due to not Found. drag(PSMRL) Start a drag-and-drop operation by dragging at the given click point. Parameters: PSMRL a pattern, a string, a match, a region or a location that evaluates to a click point.

Sikuli

Satish Gorripotu

Returns: Number 1 if the operation could be performed. A 0 (integer null) returned if no move could be performed (in case of PS may be not Found). dropAt(PSMRL[, delay]) Complete a drag-and-drop operation by dropping a previously dragged item at the given target click point. Parameters: PSMRL a pattern, a string, a match, a region or a location that evaluates to a click point. delay time to wait after in seconds as decimal value Returns: Number 1 if the operation could be performed. A 0 (integer null) returned if no move could be performed (in case of PS may be not Found). type([PSMRL], text[, modifiers]) Type the text at the current focused input field or at a click point specified by PSMRL. Parameters: PSMRL a pattern, a string, a match, a region or a location that evaluates to a click point. modifiers one or more key modifiers Returns: Number 1 if the operation could be performed, 0 (integer null) if it was not possible or the click couldnt be performed (in case of PS may be not Found). paste([PSMRL], text) Paste the text at a click point. Parameters: PSMRL a pattern, a string, a match, a region or a location that evaluates to a click point. Modifiers one or more key modifiers Returns: Number 1 if the operation could be performed, 0 (integer null) if not possible or the click couldnt be performed (in case of PS may be not Found). Extracting Text from a Region text() Extract the text contained in the region using OCR.

Sikuli

Satish Gorripotu

Low-level Mouse and Keyboard Actions mouseDown(button) Press the mouse button down. Parameters: button one or a combination of the button constants Button.LEFT, Button.MIDDLE, Button.RIGHT. Returns: Number 1 if the operation is performed successfully, and zero if otherwise. mouseUp([button]) Release the mouse button previously pressed. Parameters: button one or a combination of the button constants Button.LEFT, Button.MIDDLE, Button.RIGHT. Returns: Number 1 if the operation is performed successfully, and zero if otherwise. mouseMove(PSRML) Move the mouse pointer to a location indicated by PSRML. Parameters: PSMRL a pattern, a string, a match, a region or a location that evaluates to a click point.

Sikuli

Satish Gorripotu

Returns: Number 1 if the operation could be performed. If using PS (which invokes an implicity find operation), find fails and you have switched off FindFailed exception, a 0 (integer null) is returned. Otherwise, the script is stopped with a FindFailed exception. wheel(PSRML, WHEEL_DOWN | WHEEL_UP, steps) Move the mouse pointer to a location indicated by PSRML and turn the mouse wheel in the specified direction by the specified number of steps. Parameters: PSMRL a pattern, a string, a match, a region or a location that evaluates to a click point. WHEEL_DOWN|WHEEL_UP one of the two constants denoting the wheeling direction. steps an integer indicating the amoung of wheeling. keyDown(key | list-of-keys) Press and hold the specified key(s) until released by a later call to Region.keyUp(). Parameters: key|list-of-keys one or more keys (use the consts of class Key). A list of keys is a concatenation of several key constants using +. Returns: Number 1 if the operation could be performed and 0 if otherwise. keyUp([key | list-of-keys]) Release given keys. If no key is given, all currently pressed keys are released. Parameters: key|list-of-keys one or more keys (use the consts of class Key). A list of keys is a concatenation of several key constants using +. Returns: Number 1 if the operation could be performed and 0 if otherwise. Exception FindFailed These are the possibilities to handle not found situations: generally abort a script, if not handled with try: ... except: ... (the default setting or using setThrowException(True) or setFindFailedResponse(ABORT)) generally ignore all not found situations (using setThrowException(False) or setFindFailedResponse(SKIP)), want to be prompted in such a case (using setFindFailedResponse(PROMPT)) advise Sikuli to wait forever (be careful with that!) (using setFindFailedResponse(RETRY)) Using PROMPT:

Sikuli

Satish Gorripotu

Example: # --- nice and easy if exists("path-to-image"): # no exception, returns None when not found pass # it is there else: pass # we miss it # --- using exception handling # every not found in the try block will switch to the except block try: find("path-to-image") pass # it is there except FindFailed: pass # we miss it # --- using setFindFailedResponse setFindFailedResponse(SKIP) # no exception raised, not found returns None if find("path-to-image"): setFindFailedResponse(ABORT) # reset to default pass # it is there else: setFindFailedResponse(ABORT) # reset to default pass # we miss it # --- using setThrowException setThrowException(False) # no exception raised, not found returns None if find("path-to-image"): setThrowException(True) # reset to default pass # it is there else: setThrowException(True) # reset to default pass # we miss it

setFindFailedResponse(ABORT | SKIP | PROMPT | RETRY) For the specified region set the option, how Sikuli should handle not found situations. The option stays in effect until changed by another setFindFailedResponse().

Sikuli
Parameters:

Satish Gorripotu

ABORT all subsequent find failed operations (explicit or implicit) will raise exception FindFailed (which is the default when a script is started). SKIP all subsequent find operations will not raise exception FindFailed. Instead, explicit find operations such as Region.find() will return None. Implicit find operations (action functions) such as Region.click() will do nothing and return 0. PROMPT all subsequent find operations will not raise exception FindFailed. Instead you will be prompted for the way to handle the situation (see using PROMPT). Only the current find operation is affected by your response to the prompt. RETRY all subsequent find operations will not raise exception FindFailed. Instead, Sikuli will try to find the target until it gets visible. This is equivalent to using wait( ... , FOREVER) instead of find() or using setAutoWaitTimeout(FOREVER). getFindFailedResponse() Get the current setting in this region. Returns: ABORT or SKIP or PROMPT or RETRY Usage: val = getFindFailedResponse() if val == ABORT: print "not found will stop script with Exception FindFailed" elif val == SKIP: print "not found will be ignored" elif val == PROMPT: print "when not found you will be prompted" elif val == RETRY: print "we will always wait forever" else: print val, ": this is a bug :-(" Note: It is recommended to use set/getFindFailedResponse() instead of set/getThrowException() since the latter ones might be deprecated in the future. setThrowException(False | True) By using this method you control, how Sikuli should handle not found situations in this region. Parameters: True all subsequent find operations (explicit or implicit) will raise exception FindFailed (which is the default when a script is started) in case of not found. False all subsequent find operations will not raise exception FindFailed. Instead, explicit find operations such as Region.find() will return None. Implicit find operations (action functions) such as Region.click() will do nothing and return 0. getThrowException() Returns: True or False

Sikuli

Satish Gorripotu

Grouping Method Calls ( with Region: ) Instead of: # reg is a region object if not reg.exists(image1): reg.click(image2) reg.wait(image3, 10) reg.doubleClick(image4) you can group methods applied to the same region using Pythons with syntax: # reg is a region object with reg: if not exists(image1): click(image2) wait(image3, 10) doubleClick(image4) All methods inside the with block that have the region omitted are redirected to the region object specified at the with statement. Note for Sikuli X versions up to rc3 This should work for Regions and Region-like objects (Region, Screen, App.window()). Match objects have to be made a Region explicitly using Region(some_match). This is a general workaround when facing any problems using with some_region:. Special Methods for Developers getRegionFromPSRM(PSRM) Returns a new Region object derived from the given parameter. In case of PS, internally a find() is done inside this region. If found, the match is returned. In case RM, just a copy of the given region is returned. Params PSRM: Pattern, String, Region or Match object getLocationFromPSRML(PSRML) Returns a new Location object derived from the given parameter. In case of PS, internally a find() is done inside this region. If found, the matchs target offset position is returned. In case RM, just a copy of the given regions respective location (center or target offset) is returned. Params PSRML: Pattern, String, Region, Match or Location object

} Screen

Sikuli
Screen is a class available in Sikuli Package. This class has several methods continuation to above methods.

Satish Gorripotu

Since Screen extends class Region, all methods of class Region can be used with a screen object. Class Screen { Screen: Setting, Getting Attributes and Information Screen([id]) Create a new Screen object Parameters: id an integer number indicating which monitor in a multi-monitor environment. Returns: a new screen object. It creates a new screen object, that represents the default/primary monitor (whose id is 0), if id is omitted. Numbers 1 and higher represent additional monitors that are available at the time, the script is running. Using numbers that do not represent an existing monitor, will stop the script with an error. So we may either use getNumberScreens() or exception handling, to avoid this. Note: If we want to access the default/primary monitor ( Screen(0) ) without creating a new screen object, use the constant reference SCREEN, that is initiated when your script starts: SCREEN=Screen(0). getNumberScreens() Get the number of screens in a multi-monitor environment at the time the script is running getBounds() Get the dimensions of monitor (a rectangle object) represented by the screen object. Capturing Capturing is the feature, that allows to grab a rectangle from a screenshot, to save it for later use. At each time, a capturing is initiated, a new screenshot is taken. There are two different versions: Screen.capture() saves the content of the selected rectangle in a file and returns its file name, Screen.selectRegion() returns the position and dimension of the selected rectangle. capture([region | rectangle | text]) capture(x, y, w, h) Returns the path to the file, where the captured image was saved. In interactive mode, the user may cancel the capturing, in which case None is returned. Parameters: region an existing region object. rectangle an existing rectangle object (e.g., as a return value of another region method). text text to display in the middle of the screen in the interactive mode. x x position of the rectangle to capture y y position of the rectangle to capture w width of the rectangle to capture h height of the rectangle to capture

Sikuli

Satish Gorripotu

Interactive Mode: The script enters the screen-capture mode like when clicking the button in the IDE, enabling the user to capture a rectangle on the screen. If no text is given, the default Select a region on the screen is displayed. selectRegion([text]) Select a region on the screen interactively. Returns a new Region object or None, if the user cancels the capturing process Text is displayed for about 2 seconds in the middle of the screen. If text is omitted, the default Select a region on the screen is displayed. The interactive capture mode is entered and allows the user to select a region the same way as using the selection tool in the IDE. Parameters: text Text to display in the middle of the screen. } Multi-Monitor Environments If more than one monitor is available, Sikuli is able to manage regions and click points on these monitors.

getNumberScreens() returns the number of available screens. getBounds() returns the rectangle covered by the default/primary monitor. Screen.getBounds() returns the rectangle covered by a screen object created using Screen(id). The monitor, that is the first one based on hardware mapping (e.g. the laptop monitor), will always be Screen(0). In the Windows settings it is possible to place the taskbar on one of the secondary monitors, which makes it the primary monitor getting the base coordinates (0,0). The other available monitors are mapped around based on your settings. But the Sikuli internal mapping is not changed, so the primary monitor might be any of your Screen() objects. Sikuli takes care for that and maps SCREEN always to the primary monitor (the one with the (0,0) coordinates). So for example you have a laptop with an external monitor, that shows the taskbar (is primary monitor): SCREEN maps to Screen(1) Screen(0) is your laptop monitor With its rectangle, a screen object is always identical with the monitor it was created using Screen(id). Using Region.setROI() to restrict the region of interest for find operations has no effect on the base rectangle of the screen object. On the other hand region objects and location objects can be positioned anywhere in the coordinate system. Only when a find operation or a click action has to be performed, the objects rectangle or point has to be inside the rectangle of an existing monitor (basically repersented by

Sikuli

Satish Gorripotu

Screen(0), Screen(1), ...). When this condition is met, everything works as expected and known from a single monitor system. With finding and acting there are the following exceptions: Point Outside: a click point is outside any monitor rectangle. The clickpoint will be mapped to the edges or corners of the primary monitor according to the relative position: to the edges if its x or y value is in the range of the respective edge (right, left, above, below)i to the corners, if x and y are outside any range of any edge (left/above -> upper left corner, ...) Region Outside: a region is completely outside any monitor a click action is handled in the same way as Point Outside a find operation will always fail Region Partially Outside: a region is partially outside a monitor but not overlapping another monitor a click action is handled in the same way as Point Outside a find operation will be carried out only in the part of region within the bounds of the monitor, excluding the area outside the monitor. Region Across Monitors: a region lies across multiple monitors: a click action is handled in the same way as Point Outside a find operation will be restricted to the region within the bounds of the monitor that has a smaller id. An interactive capture (the user is asked to select an image or a rectangle via Screen.capture() or Screen.selectRegion()) will automatically be restricted to the monitor, where it was started. A scripted capture using a rectangle or a region (i.e. Screen.capture( region | rectangle )), will be handled accordingly: Region Outside: no image is captured, None is returned Region Partially Outside: the returned image will only cover the part inside the monitor Region Across Monitors: the returned image will only cover the part inside the monitor with the smallest id.

Location Location is a class available in Sikuli Package. Handle single points on the screen directly by its position (x, y). This class has several methods continuation to above methods. Class Location { Location(x, y) Returns a new location object representing the position (x,y) on the screen Parameters: x x position y y position getX() getY() Get the x or y value of a location object

Sikuli

Satish Gorripotu

It is possible to get the values directly by location.x or location.y. It is also possible to set these values directly by location.x = value or location.y = value. setLocation(x, y) Set the location of this object to the specified coordinates. offset(dx, dy) Get a new location which is dx and dy pixels away horizontally and vertically from the current location. above(dy) Get a new location which is dy pixels vertically above the current location. below(dy) Get a new location which is dy pixels vertically below the current location. left(dx) Get a new location which is dx pixels horizontally to the left of the current location. right(dx) Get a new location which is dx pixels horizontally to the right of the current location. }

Match Match is a class available in Sikuli Package. class Match extends class Region, all methods of class Region can be used with a match object. An object of class Match represents the result of a successful find operation. It has the rectangle dimension of the image, that was used to search. It knows the point of its upper left corner on an existing monitor, where it was found. This class has several methods continuation to above methods. Class Match { Creating a Match, Getting Attributes A match object is created as the result of an explicit find operation. It can be saved in a variable for later use with actions like click(). # m is a reference to a match object, if found m = find(Some Image) print m # message area: Match[10,0 30x22] score=1.00, target=center

Sikuli

Satish Gorripotu
# m is a reference to a match object, if found m = find(Pattern(Some Image).similar(0.5).targetOffset(100,0)) print m # message area: Match[10,0 30x22] score=1.00, target=(105,11)

getScore() Get the similarity score the image or pattern was found. The value is between 0 and 1. getTarget() Get the location object that will be used as the click point. If no offset was specified by Pattern.targetOffset(),click point is the center of the matched region. If an offset was given, the click point is the offset relative to the center. Iterating over Matches after findAll() find operation Region.findAll() returns an iterator object that can be used to fetch all found matches as match objects one by one. A reference to the iterator is stored in the respective region and can be accessed using Region.getLastMatches(). Note: per definition, an iterator can be stepped through only once - it is empty afterwards To save contained matches for later use, you can convert them to list.

findAll() # find all matches mm = list(getLastMatches()) Example: using while: with default screen findAll() # find all matches mm = SCREEN.getLastMatches() while mm.hasNext(): # loop as long there is a first and more matches print "found: ", mm.next() # access the next match in the row print mm.hasNext() # is False, because mm is empty now print mm.next() # is None, because mm is empty now print SCREEN.getLastMatches().hasNext() # is False also ;-) Example: using with: with default screen with findAll() as mm: while mm.hasNext(): # loop as long there is a first and more matches print "found: ", mm.next() # access the next match # mm will be None afterwards (destroyed automatically)

Sikuli

Satish Gorripotu

Pattern Pattern is a class available in Sikuli Package. A pattern is used, to associate an image file with additional attributes used in find operations and when acting on a match object. Normally when clicking on a match, the center pixel of the associated rectangle is used. With a pattern object, we can define a different click point relative to the center using targetOffset(). Class Pattern { Pattern(string) This will initialize a new pattern object without any additional attributes. As long as no pattern methods are used additionally, it is the same as just using the image file name itself in the find operation Parameters: string a path to an image file similar(similarity) Return a new Pattern object containing the same attributes (image, click point) with the minimum similarity set to the specified value. Parameters: similarity the minimum similarity to use in a find operation. The value should be between 0 and 1. exact() Return a new Pattern object containing the same attributes (image, click point) with the minimum similarity set to 1.0, which means exact match is required. targetOffset(dx, dy)

Sikuli

Satish Gorripotu

Return a new Pattern object containing the same attributes (image, similarity), but a different definition for the click. By default, the click point is the center of the found match. By setting the target offset, it is possible to specify a click point other than the center. dx and dy will be used to calculate the position relative to the center. Parameters: dx x offset from the center dy y offset from the center getFilename() Get the filename of the image as a string contained in the Pattern object. getTargetOffset() Get a Location object as the target offset of the Pattern object. } Finder Finder is a class available in Sikuli Package. A Finder object implements an iterator of matches and allows to search for a visual object in an image file that you provide (e.g. a screenshot taken and saved in a file before). After setting up the finder object and doing a find operation, you can iterate through the found matches if any. Note: 1. per definition, an iterator can be stepped through only once - it is empty afterwards 2. it has to be destroyed using finder.destroy(), especially when used with for: or while: 3. When used in a with: construct, it is destroyed automatically Compared with the region based find operation, no exception FindFailed is raised in case nothing is found at all (use hasNext() to check). The finder object can be compared to what you get with region.getLastMatches() when using findAll(). Note: With this version, there is no chance, to get the number of matches in advance. If you would iterate through to count, afterwards your finder would be empty. So in this case, you have to save your matches somehow The workflow always is, that you first do a find operation and afterwards go through the matches found. After a complete iteration, the finder object is empty. So we could start a new find operation again. Class Finder { Finder(path-to-imagefile) Create a new finder object. Parameters: path-to-imagefile filename to a source image to search within find(path-to-imagefile[, similarity])

Sikuli

Satish Gorripotu

Find a given image within a source image previously specified in the constructor of the finder object. Parameters: path-to-imagefile the target image to search for similarity the minimum similarity a match should have. If omitted, the default is used. hasNext() Check whether there are more matches available that satisfy the minimum similarity requirement. next() Get the next match object. The returnd reference to a match object is no longer available in the finder object afterwards. So if it is needed later on, it has to be saved to another variable. Example 1: basic operations using a Finder # create a Finder with your saved screenshot f = Finder() img= # the image you are searching f.find(img) # find all matches while f.hasNext(): # loop as long there is a first and more matches print "found: ", f.next() # access the next match in the row print f.hasNext() # is False, because f is empty now f.destroy() # release the memory used by finder Example 2: we want to know how many matches (based on the previous example). # create a Finder with your saved screenshot f = Finder() img= # the image you are searching f.find(img) # find all matches mm = [] # an empty list while f.hasNext(): # loop as long there is a first and more matches mm.append(f.next()) # access next match and add to mm print f.hasNext() # is False, because f is empty now f.destroy() # release the memory used by finder # now we have our matches saved in the list mm print len(mm) # the number of matches # we want to use our matches for m in mm: print m }

Sikuli

Satish Gorripotu

Key Constants class Key { Applicable usage situations for these predefined constants of special keys and key modifiers can be found in Acting on a Region and Low Level Mouse and Keyboard Actions. Special Keys The methods supporting the use of special keys are type(), keyDown(), and keyUp(). Key.CONSTANT (where CONSTANT is one of the following key names). String concatenation with other text or other key constants is possible using +. type("some text" + Key.TAB + "more text" + Key.TAB + Key.ENTER) # or eqivalent type("some text\tmore text\n") miscellanous keys ENTER, TAB, ESC, BACKSPACE, DELETE, INSERT,SPACE function keys F1, F2, F3, F4, F5, F6, F7, F8, F9, F10, F11, F12, F13, F14, F15 navigation keys HOME, END, LEFT, RIGHT, DOWN, UP, PAGE_DOWN, PAGE_UP special keys PRINTSCREEN, PAUSE, CAPS_LOCK, SCROLL_LOCK, NUM_LOCK Note: The status ( on / off ) of the keys Key.CAPS_LOCK, Key.NUM_LOCK and Key.SCROLL_LOCK can be evaluated with the method Env.isLockOn(). numpad keys NUM0, NUM1, NUM2, NUM3, NUM4, NUM5, NUM6, NUM7, NUM8, NUM9 SEPARATOR, ADD, MINUS, MULTIPLY, DIVIDE modifier keys ALT, CMD, CTRL, META, SHIFT, WIN

Sikuli

Satish Gorripotu

These modifier keys cannot be used as a key modifier with functions like type(), rightClick(), etc. They can only be used with keyDown() and keyUp(). If you need key modifiers, use KeyModifier instead. Key Modifiers Methods where key modifiers can be used include: click(), dragDrop() , doubleClick() , rightClick(), type(). system specific Win KEY_WIN, KEY_CMD KEY_META (a synonym for KEY_WIN or KEY_CMD on Windows). The old modifiers with a KEY_ prefix are deprecated. Use KeyModifier.CTRL, KeyModifier.ALT, KeyModifier.SHIFT, KeyModifier.META instead. KeyModifier class KeyModifier KeyModifier.CONSTANT (where CONSTANT is one of the following key names). CTRL equivalent to the old KEY_CTRL SHIFT equivalent to the old KEY_SHIFT ALT equivalent to the old KEY_ALT META equivalent to the old KEY_META CMD equivalent to the old KEY_CMD (and KEY_META) WIN equivalent to the old KEY_WIN (and KEY_META) The modifier constants can be combined to the modifier parameter by either using + or |, if more than one key modifier is needed. type(Key.ESC, KeyModifier.CTRL + KeyModifier.ALT) # or equivalent type(Key.ESC, KeyModifier.CTRL | KeyModifier.ALT) They should only be used in the modifiers parameter with functions like type(), rightClick(), etc. They should never be used with keyDown() or keyUp().

Constants related to Mouse and Keyboard Actions

Sikuli
Mouse related Class Button : Button.LEFT - Button.MIDDLE - Button.RIGHT related to mouse wheel actions: WHEEL_DOWN - WHEEL_UP Keyboard related

Satish Gorripotu

Key Modifiers KEY_ALT - KEY_CMD - KEY_CTRL - KEY_META - KEY_SHIFT - KEY_WIN A combination of these keys has to be set up using "+" or "|", e.g. KEY_SHIFT + KEY_ALT or KEY_SHIFT | KEY_ALT | KEY_CMD (used with action methods) Special Keys To be used with type() or keyDown()/keyUp(): ADD - ALT - BACKSPACE - CAPS_LOCK - DIVIDE - DOWN - END - ENTER - ESC - DELETE - F1 - F2 - F3 - F4 - F5 - F6 - F7 - F8 - F9 - F10 - F11 - F12 - F13 - F14 - F15 - HOME - INSERT- LEFT MINUS - MULTIPLY - NUM_LOCK - NUM0 - NUM1 - NUM2 - NUM3 - NUM4 - NUM5 - NUM6 - NUM7 NUM8 - NUM9 - PAGE_DOWN - PAGE_UP - PAUSE - PRINTSCREEN - RIGHT - SCROLL_LOCK SEPARATOR - TAB UP To be used with keyDown()/keyUp() only: CMD - CTRL - META - SHIFT - TAB- WIN To use these constants, write e.g. Key.ENTER. String concatenation with "+" is possible, e.g. "some text" + Key.ENTER Constants FOREVER: can be used with exists - observe - wait - waitVanish SCREEN : a constant reference to a screen object created using Screen(0) at startup, The default/primary monitor used as the default region for methods not qualified by an instance/object/class.

Extensions Download an Extension The download of an extension is supported by the IDE through the menu Tools -> Extensions.

Sikuli

Satish Gorripotu

We get a popup, that lists the available and already installed extensions and allows to download new packages or updates for installed ones. This popup shows a new package not yet installed:

Use an Extension: To use the features of an installed extension in one of your scripts, just say from extension-name import *. Technical Details Extensions are Java JAR files containing some Java classes (usually the core functions) and/or Python modules, which define the API to be used in a script. Sikuli maintains a local extensions directory, where downloaded extensions are stored together with a hidden list of the installed extensions %APPDATA%\Sikuli\extensions Once an extension is imported using import extension-name, Sikuli automatically searches and loads the JAR file of that extension into the current context with load(path-to-jar-file). Develop an extension The source structure of an extension named extension-name looks like this: Java - org/com -- your-organization-or-company --- extension-name ---- yourClass1.java ---- yourClass2.java ---- .... more classes python - extension-name -- __init__.py -- extension-name.py The final structure of a JAR (filename extension-name-X.Y where X.Y is the version string) looks like this: org/com - your-organization-or-company -- extension-name --- yourClass1.class --- yourClass2.class --- .... more classes extension-name - __init__.py

Sikuli
- extension-name.py META-INF

Satish Gorripotu

- MANIFEST.MFThe file __init__.py contains at least from extension-name import * to avoid one qualification level. So in a script you might either use: import extension-name extension-name.functionXYZ() or: from extension-name import * functionXYZ() The second case requires more investement in a naming convention, that avoids naming conflicts. The file extension-name.py contains the classes and methods, that represent the API, that one might use in a Sikuli script. Name the extensions Sikuli extensions can be Python/Jython modules or Java classes. For Java classes, following the reverse URL convention of Java is a good idea (for example, org.foo.your-extension). However, DO NOT use Javas convention for Python/Jython modules. You need to come up with a unique extension name that does not conflict with existing Python modules and other Sikuli extensions. Test the extension While developing your extensions, you can put the JAR file in Sikulis extension directory or in the same .sikuli folder as your test script. The JAR file should not have a version number in its file name, e.g. extension-name.jar. Because Sikuli starts to search extensions in the .sikuli folder of the running script and then in the Sikuli extensions folder, it is usually a good idea to put your developing extensions in the .sikuli folder of your test script. Another option is to use the load() function with an absolute path to your extension-name.jar. If this fails, Sikuli goes on searching in the current .sikuli folder and then in the Sikuli extensions folder. If load() succeeds, it returns True and puts absolute-path-to-your-extension-name.jar into sys.path, so you can use import extension-name afterwards. Guide:

Sikuli

Satish Gorripotu

Example: from guide import * text(<Some Image on the screen>, "This is Sikuli's logo") show(5) Adding Multiple Annotations The script uses the function tooltip(text) to add tooltips to three links in addition to the text annotation from guide import * text(, "This is Sikuli's logo") tooltip(,"Previous") tooltip(,"Next") tooltip(,"Index") show(5) from guide import *

Sikuli
while True: tooltip(,"Previous") show(3) tooltip(,"Next") show(3) tooltip(,"Index") show(3) Adding Interaction from guide import * text(,"Use these to jump to other parts") dialog("Part 1: Navigation Links") show() text(,"Use this to go back to Home") dialog("Part 2: Logo") show() Making a Region clickable from guide import * logo = find() text(logo, "To proceed click this red ...") clickable(logo) index = logo.above().right().find() text(index, "... or click this red") clickable(index) show()

Satish Gorripotu

Static Annotations guide.rectangle(PSRM) Add a rectangular overlay in red on the specified targets region. Parameters: PSRM a pattern, string, region or match guide.circle(PSRM) Add a red circle around the specified targets region. Parameters: PSRM a pattern, string, region or match guide.text(PSRM, text) Add some text (white large letters on dark grey background) left justified below the specified targets region, which is additionally highlighted. Parameters: PSRM a pattern, string, region or match text a string as text to display

Sikuli

Satish Gorripotu

guide.tooltip(PSRM, text) Add a tooltip (small text in a light yellow box) left justified below the specified targets region. Parameters: PSRM a pattern, string, region or match text a string as text to display Interactive Elements guide.dialog(text) Add a dialog box displaying the given text in the middle of the screen above all othe windows. Parameters: text a string as text to display guide.clickable(PSRM) Add a clickable element corresponding to the specified targets region. Parameters: PSRM a pattern, string, region or match Control guide.show([seconds]) Show static and interactive components added so far for the specified amount of time. Parameters: seconds a decimal number as display duration in seconds The default duration is 10 seconds. If interactive elements (either one or more clickable elements or a dialog box) were previously added, it waits until the user interacts with one of these elements. At this time all elements vanish and are discarded. Note: If a dialog() element is present, other interactive elements are visible, but not clickable. If the dialog element is clicked, all elements vanish and are discarded. Summary of Class and Methods Region Methods of Region above - below - click - drag - dragDrop - dropAt - doubleClick - exists - find - findAll - getCenter getH - getLastMatch - getLastMatches - getRect - getROI - getScreen - getW - getX - getY highlight - hover - inside - keyDown - keyUp - left - mouseDown - mouseMove - mouseUp - nearby - observe - onAppear - onVanish - onChange - paste - right - rightClick - selectRegion setAutoWaitTimeout - setH - setRect - setROI - setThrowException - setW - setX - setY stopObserver - text - type - wait waitVanish Creating a Region, Setting and Getting Attributes getCenter - getH - getLastMatch - getLastMatches - getRect - getROI - getScreen - getW - getX getY - selectRegion - setAutoWaitTimeout - setH - setRect - setROI - setW - setX - setY Extending a Region above - below - inside - left - nearby - right Finding inside a Region and Waiting for a Visual Event exists - find - findAll - wait - waitVanish Observing Visual Events in a Region

Sikuli
observe - onAppear - onVanish - onChange - stopObserver

Satish Gorripotu

Acting on a Region click - drag - dragDrop - dropAt - doubleClick - highlight - hover - paste - rightClick - type Extracting Text from a Region text Low Level Mouse and Keyboard Actions getMouseLocation - keyDown - keyUpmouseDown - mouseMove - mouseUp - wheel

Screen click - drag - dragDrop - dropAt - doubleClick - exists - find - findAll - getCenter - getH getLastMatch - getLastMatches - getROI - getW - getX - getY - hover - keyDown - keyUp mouseDown - mouseMove - mouseUp - observe - onAppear - onVanish - onChange - paste rightClick - setAutoWaitTimeout - setH - setROI - setThrowException - setW - setX - setY stopObserver - text - type - wait waitVanish capture - getBounds - getNumberScreens - selectRegion Setting, Getting Attributes and Information getBounds - getNumberScreens Screen as (Default) Region Capturing capture - selectRegion Location above - below - getsetL - getsetL - left - offset - right - getsetL - getsetL Match above - below - click - drag - dragDrop - dropAt - doubleClick - exists - find - findAll - getCenter getH - getLastMatch - getLastMatches - getScreen - getW - getX - getY - hover - inside - left mouseMove - nearby - observe - onAppear - onVanish - onChange - paste - right - rightClick selectRegion - setAutoWaitTimeout - setThrowException - stopObserver - type - wait waitVanish

Sikuli
getScore - getTarget Finder find - hasNext - next Pattern exact - getFilename - getTargetOffset - similar - targetOffset VDict

Satish Gorripotu

vdict[ ] - vdict[ ] = - [ not in vdict] - del vdict[ ] - get -get1 - get_exact - len( ) - keys Env getOS() returns information about the system, the script is running on ( -> General Information and Settings ) getOSVersion() returns the system's version number, the script is running on ( -> General Information and Settings ) getMouseLocation() returns the current location of the mouse pointer ( -> Low Level Mouse and Keyboard Actions ) getClipboard() returns the content of the Clipboard if it is text, otherwise an empty string. ( -> General Information and Settings )

Sikuli

Satish Gorripotu

Sikuli Class and methods

Class Region

Method

Parameter

Return value Create a region object Create a region object Create a region object

Usage 1

Usage 2

Region(x, y, w, h) Region(region) Region(Rectangle) setX(number) setY(number) setW(number) setH(number) moveTo(location) setROI(x, y, w, h) setROI(rectangle) setRect(x, y, w, h) setRect(rectangle) morphTo(region) getX() getY() getW() getH() getCenter() getTopLeft() getTopRight() getBottomLeft() getBottomRight() getScreen() getLastMatch() getLastMatches() setAutoWaitTimeout(seconds) setAutoWaitTimeout(seconds) getAutoWaitTimeout() offset(location) inside()

Sikuli
nearby([range]) above([range]) below([range]) left([range]) right([range]) find(PS) findAll(PS) wait([PS][, seconds]) waitVanish(PS[, seconds]) exists(PS[, seconds]) onAppear(PS, handler) onVanish(PS, handler) onChange([minChangedSize], handler) observe([seconds][, background = False | True]) stopObserver() click(PSMRL[, modifiers]) doubleClick(PSMRL[, modifiers]) rightClick(PSMRL[, modifiers]) highlight([seconds]) hover(PSMRL) dragDrop(PSMRL, PSMRL[, modifiers]) drag(PSMRL) dropAt(PSMRL[, delay]) type([PSMRL], text[, modifiers]) paste([PSMRL], text) text() mouseDown(button) mouseUp([button]) mouseMove(PSRML) wheel(PSRML, WHEEL_DOWN | WHEEL_UP, steps) keyDown(key | list-of-keys) keyUp([key | list-of-keys]) setFindFailedResponse(ABOR T | SKIP | PROMPT | RETRY) getFindFailedResponse() setThrowException(False | True) getThrowException() getRegionFromPSRM(PSRM) getLocationFromPSRML(PSRM L) SikuliEvent type pattern region match changes

Satish Gorripotu

Sikuli
Screen Screen([id]) getNumberScreens() getBounds() capture([region | rectangle | text]) capture(x, y, w, h) selectRegion([text]) Location Location(x, y) getX() getY() setLocation(x, y) offset(dx, dy) above(dy) below(dy) left(dx) right(dx) Match getScore() getTarget() Pattern Pattern(string) similar(similarity) exact() targetOffset(dx, dy) getFilename() getTargetOffset() Finder Finder(path-to-imagefile) find(path-to-imagefile[, similarity]) hasNext() next() Button left middle right App (Application) Env (Environment) Key (Key Constants) VDict (Visual Dictionary)

Satish Gorripotu

Sikuli consolidated Methods:

Sikuli

Satish Gorripotu

S.No command line option --args <arguments> -h,--help "-I" -r,--run <sikuli-file> -s,--stderr -t,--test <sikuli-file>

Methods

Examples

Page Numbers

A 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 above() (Location method) Above (Region method) ActionLogs (Settings attribute) addHotkey() (Env method) addImagePath() (built-in function) ALT (built-in variable) App (built-in class), [1] B below() (Location method) below (Region method) Bundle Path C capture() (Screen method) changes (SikuliEvent attribute) circle() (in module guide) click() (Region method) clickable() (in module guide) close() (App class method) Close (App method) closeApp() (built-in function) CMD (built-in variable) --args <arguments> -h,--help "-I" -r,--run <sikuli-file> -s,--stderr -t,--test <sikuli-file> CTRL (built-in variable) D DebugLogs (Settings attribute) DelayAfterDrag (Settings attribute) Done

Sikuli
29 30 31 32 33 34 DelayBeforeDrop (Settings attribute) dialog() (in module guide) doubleClick() (Region method) drag() (Region method) dragDrop() (Region method) dropAt() (Region method) destroy E 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 exact() (Pattern method) exists() (Region method) exit() (built-in function) F find() (Finder method) find (Region method) findAll() (Region method) Finder (built-in class), [1] Finder() (Finder method) focus() (App class method) focus (App method) focusedWindow() (App class method) G getAutoWaitTimeout() (Region method) getBottomLeft() (Region method) getBottomRight() (Region method) getBounds() (Screen method) getBundlePath() (built-in function) getCenter() (Region method) getClipboard() (Env method) getFilename() (Pattern method) getFindFailedResponse() (Region method) getH() (Region method) getImagePath() (built-in function) getLastMatch() (Region method) getLastMatches() (Region method) getLocationFromPSRML() (Region method) getMouseLocation() (Env method) getNumberScreens() (Screen method) getOS() (Env method) getOSVersion() (Env method) getParameter() (Vision method) getRegionFromPSRM() (Region method) getScore() (Match method) getScreen() (Region method) Done Done Done Done

Satish Gorripotu

Sikuli
68 69 70 71 72 73 74 75 76 77 78 79 getSikuliVersion() (Env method) getTarget() (Match method) getTargetOffset() (Pattern method) getThrowException() (Region method) getTopLeft() (Region method) getTopRight() (Region method) getW() (Region method) getX() (Location method) getX(Region method) getY() (Location method) getY(Region method) guide (module) getRect getROI H 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 hasNext() (Finder method) highlight() (Region method) hover() (Region method) I Image Search Path SIKULI_IMAGE_PATH import .sikuli InfoLogs (Settings attribute) input() (built-in function) inside() (Region method) isLockOn() (Env method) K Key (built-in class) keyDown() (Region method) KeyModifier (built-in class) keyUp() (Region method) L left() (Location method) Left (Region method) load() (built-in function) Location (built-in class) Location() (Location method) M Match (built-in class), [1] match (SikuliEvent attribute) META (built-in variable) MinSimilarity (Settings attribute) Done

Satish Gorripotu

Sikuli
103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 morphTo() (Region method) mouseDown() (Region method) mouseMove() (Region method) mouseUp() (Region method) MoveMouseDelay (Settings attribute) moveTo() (Region method) N nearby() (Region method) next() (Finder method) O observe() (Region method) ObserveMinChangedPixels (Settings attribute) ObserveScanRate (Settings attribute) offset() (Location method) offset(Region method) onAppear() (Region method) onChange() (Region method) onVanish() (Region method) open() (App class method) open(App method) openApp() (built-in function) P paste() (Region method) Pattern (built-in class), [1] pattern (SikuliEvent attribute) Pattern() (Pattern method) popup() (built-in function) R rectangle() (in module guide) Region (built-in class), [1], [2], [3], [4], [5], [6], [7], [8], [9] region (SikuliEvent attribute) Region() (Region method) removeHotkey() (Env method) removeImagePath() (built-in function) right() (Location method) right(Region method) rightClick() (Region method) run() (built-in function) S Screen (built-in class), [1], [2] Screen() (Screen method) selectRegion() (Screen method) Done Done

Satish Gorripotu

Sikuli
140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 setAutoWaitTimeout() (Region method) setBundlePath() (built-in function) setFindFailedResponse() (Region method) setH() (Region method) setLocation() (Location method) setParameter() (Vision method) setRect() (Region method) setROI() (Region method) setShowActions() (built-in function) setThrowException() (Region method) Settings (built-in class) setW() (Region method) setX() (Region method) setY() (Region method) SHIFT (built-in variable) show() (in module guide) SIKULI_IMAGE_PATH Image Search Path SikuliEvent (built-in class) similar() (Pattern method) SlowMotionDelay (Settings attribute) stopObserver() (Region method) switchApp() (built-in function) setX (Location) setY (Location) T 163 164 165 166 167 168 169 170 171 172 173 174 targetOffset() (Pattern method) text() (in module guide) text(Region method) tooltip() (in module guide) type (SikuliEvent attribute) type() (Region method) W wait() (Region method) WaitScanRate (Settings attribute) waitVanish() (Region method) wheel() (Region method) WIN (built-in variable) window() (App method) Done Done

Satish Gorripotu

Sikuli

Satish Gorripotu

Java Docs http://sikuli.org/doc/java-x/ S.No 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 App AppearEvent AppNotFound Button CapturePrompt ChangeEvent Clipboard Clipboard.Charset Clipboard.TextType Clipboard.TransferType Constants Debug DesktopRobot DummyOSUtil Env EventManager ExtensionManager Finder FindFailed FindFailedResponse HotkeyEvent HotkeyListener ImageLocator IRobot All Classes Animator

Sikuli
26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 IRobot.KeyMode IScreen Key KeyModifier LinuxUtil Location MacUtil Match Observer OpenCV OS OSUtil Pattern Region ResourceExtractor Screen ScreenHighlighter ScreenImage ScriptRunner Settings SikuliAction SikuliActionListener SikuliActionManager SikuliEvent SikuliEvent.Type SikuliEventAdapter SikuliEventObserver SikuliException SikuliScript Subject TextRecognizer TextRecognizer.ListTextMode TransparentWindow UnionScreen Util VanishEvent VDictProxy Win32Util

Satish Gorripotu

Sikuli

Satish Gorripotu

Basics of Jython Python for java Platform Sikuli Using IDE as Eclipse then, we have two choices. 1. Using Java as a scripting language 2. Using Python as scripting language by downloading Python or Jython plugIn to Eclipse. If we use default Sikuli Ide then we have to use Python or Jython as scripting language. http://jythonpodcast.hostjava.net/jythonbook/en/1.0/ Declare Variable Java int x = 0; Python x=0 Reserved Words and assert break class global or pass print raise Coding Structure Java if-statement x = 100; if(x > 0) { System.out.println("Wow, this is Java"); } else { System.out.println("Java likes curly braces"); } Python if-statement x = 100 if x > 0: print 'Wow, this is elegant' else: print 'Organization is the key' continue def del elif return try while with else yield except exec finally for from

Sikuli

Satish Gorripotu

Functions Example 1: def my_function_name(parameter_list): implementation Example 2: def my_simple_function(): ... print 'This is a really basic function'

my_simple_function() Example 3: def multiply_nums(x, y): ... return x * y multiply_nums(25, 7) 175 Example 4: def perform_math(oper): ... return oper(5, 6) perform_math(multiply_nums) 30 Classes Java - this Python - self >>> class my_object: ... def __init__(self, x, y): ... self.x = x ... self.y = y ... ... def mult(self): ... print self.x * self.y ... ... def add(self): ... print self.x + self.y ...

Sikuli
>>> obj1 = my_object(7, 8) >>> obj1.mult() 56 >>> obj1.add() 15 Statements Statement Keywords if-elif-else for while

Satish Gorripotu

continue break try-except-finally assert def print del raise import

if-elif-else Statement if <an expression to test>: perform an action else: perform a different action >>> x = 3 >>> y = 2 >>> if x == y: ... print 'x is equal to y' ... elif x > y: ... print 'x is greater than y' ... else: ... print 'x is less than y' ... x is greater than y print Statement Java Print Output Example System.out.println("This text will be printed to the command line"); Python Print Output Example print 'This text will be printed to the command line'

>>> my_value = 'Satish Gorripotu' >>> print my_value Satish Gorripotu >>> print 'My Name is Satish, but people called me as ' + my_value My Name is Satish, but people called me as Satish Gorripotu >>> z = 10 >>> print 'I am a fan of the number: ' + z Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: cannot concatenate 'str' and 'int' objects

Sikuli
To resolve, we have to use below formatting operators * %s - String * %d - Decimal * %f - Float

Satish Gorripotu

print 'String of text goes here %d %s %f' % (decimalValue, stringValue, floatValue) >>> string_value = 'hello world' >>> float_value = 3.998 >>> decimal_value = 5 >>> print 'Here is a test of the print statement using the values: %d, %s, and %f' % (decimal_value, string_value, float_value) Here is a test of the print statement using the values: 5, hello world, and 3.998000

>>> x = 1 >>> y = 2 >>> print 'The value of x + y is: %d' % (x + y) The value of x + y is: 3 >>> x = 2.3456 >>> print '%s' % x 2.3456 >>> print '%d' % x 2 >>> print '%f' % x 2.345600 try-except-finally >>> # Suppose we've calculated a value and assigned it to x >>> x 8.97 >>> y = 0 >>> try: ... print 'Just Print: %f' % (x/y) ... except: ... print Satti, we have a problem.... Satti, we have a problem. raise Statement the raise statement is used to throw or raise an exception in Python >>> raise NameError Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError

Sikuli

Satish Gorripotu

>>> raise Exception('Custom Exception') Traceback (most recent call last): File "<stdin>", line 1, in <module> Exception: Custom Exception

import Statement # Import a module named TipCalculator import TipCalculator # Import a function tipCalculator from within a module called ExternalModule.py from ExternalModule import tipCalculator To avode over riding import tipCalculator as tip Iteration Index start from zero >>> my_numbers = [1, 2, 3, 4, 5] >>> my_numbers [1, 2, 3, 4, 5] >>> my_numbers[1] 2 >>> for value in my_numbers: ... print value ... 1 2 3 4 5 >>> x = 0 >>> while x < len(my_numbers): ... print my_numbers[x] ... x = x + 1 ... 1 2 3 4 5

While Loop

Sikuli
while True perform operation

Satish Gorripotu

Example of a Java While Loop int x = 9; int y = 2; int z = x y; while (y < x) { System.out.println("y is " + z + " less than x"); y = y++; } Example of a Python While Loop >>> x = 9 >>> y = 2 >>> while y < x: ... print 'y is %d less than x' % (x-y) ... y += 1 ... y is 7 less than x y is 6 less than x y is 5 less than x y is 4 less than x y is 3 less than x y is 2 less than x y is 1 less than x For Loop for each value in this defined set: perform suite of operations Example of Java For Loop for (x = 0; x <= 6; x++){ System.out.println(x); } Example of Python For Loop >>> for x in range(10): ... print x ... 0 1 2 3 4 5

Sikuli
Basic Keyboard Input 1. raw_input() 2. input()

Satish Gorripotu

# The text within the function is optional, and it is used as a prompt to the user >>> name = raw_input("Enter Your Name:") Enter Your Name:satish >>> print name satish # Use the input function to evaluate an expression entered in by the user >>> val = input ('Please provide an expression: ') Please provide an expression: 9 * 3 >>> val 27 # The input function raises an error if an expression is not provided >>> val = input ('Please provide an expression: ') Please provide an expression: My Name is Satti Traceback (most recent call last): File "<stdin>", line 1, in <module> File "<string>", line 1 My Name is Satti ^ SyntaxError: invalid syntax

String Methods Method capitalize() center (width[,fill]) count(sub[,start[,end]]) decode([encoding[,errors]]) encode([encoding[,errors]]) endswith(suffix[,start[,end]]) expandtabs([tabsize]) find(sub[,start[,end]]) index(sub[,start[,end]) isalnum() isalpha() isdigit() islower() Description of Functionality Returns a capitalized copy of string Returns a repositioned string with specified width and provide optional padding filler character Count the number of distinct times the substring occurs within the string Decodes and returns Unicode string Returns an encoded version of a string Returns a boolean to state whether the string ends in a given pattern Converts tabs within a string into spaces Returns the index of the position where the first occurrence of the given substring begins Returns the index of the position where the first occurrence of the given substring begins. Raises a ValueError with the substring is not found. Returns a boolean to state whether the string contain only alphabetic and numeric characters Returns a boolean to state whether the string contains all alphabetic characters Returns a boolean to state whether the string contains all numeric characters Returns a boolean to state whether a string contains all lowercase characters

Sikuli
isspace() istitle() isupper() join(sequence) ljust(width[,fillchar]) lower()

Satish Gorripotu
Returns a boolean to state whether the string consists of all whitespace Returns a boolean to state whether the first character of each word in the string is capitalized Returns a boolean to state whether all characters within the string are uppercase Returns a copy of sequence joined together with the original string placed between each element Returns a string of the specified width along with a copy of the original string at the leftmost bit. (Optionally padding empty space with fillchar) Returns a copy of the original string with all characters in the string converted to lowercase Removes the first found characters in the string from the left that match the given characters. Also removes whitespace from the left. Whitespace removal is default when specified with no arguments. Returns a partitioned string starting from the left using the provided separator Returns a copy of the original string replacing the portion of string given in old with the portion given in new Searches string from right to left and finds the first occurrence of the given string and returns highest index where sub is found Searches string from right to left and finds the first occurrence of the given string and either returns highest index where sub is found or raises an exception Returns copy of string Aligned to the right by width Returns a copy of stringPartitioned starting from the right using the provided separator object Returns list of words in string and splits the string from the right side and uses the given separator as a delimiter. If maxsplit is specified then at most maxsplit splits are done (from the right). Returns copy of string removing the first found characters in the string from the right that match those given. Also removes whitespace from the right when no argument is specified. Returns a list of words in string and splits the string from the left side and uses the given separator as a delimiter. Splits the string into a list of lines. Keepends denotes if newline delimiters are removed. Returns the list of lines in the string. Returns a boolean to state whether the string starts with the given prefix Returns a copy of string with the given characters removed from the string. If no argument is specified then whitespace is removed. Returns a copy of the string the case of each character in the string converted. Returns a copy of the string with the first character in each word uppercase. Returns a copy of the string using the given character translation table to translate the string. All characters occurring in optional deletechars argument are removed. Returns a copy of string with all of the characters in the string converted to uppercase Returns a numeric string padded from the left with zeros for the specified width.

lstrip([chars]) partition(separator) replace(old,new[,count]) rfind(sub[,start[,end]]) rindex(sub[,start[,end]]) rjust(width[,fillchar]) rpartition(separator) rsplit([separator[,maxsplit]]) rstrip([chars]) split([separator[,maxsplit]]) splitlines([keepends]) startswith(prefix[,start[,end]]) strip([chars]) swapcase() title() translate(table[,deletechars]) upper() zfill(width)

Sikuli

Satish Gorripotu

Examples: our_string='python is the best language ever' # Capitalize first character of a String >>> our_string.capitalize() 'Python is the best language ever' # Center string >>> our_string.center(50) ' python is the best language ever ' >>> our_string.center(50,'-') '---------python is the best language ever---------' # Count substring within a string >>> our_string.count('a') 2 # Count occurrences of substrings >>> state = 'Mississippi' >>> state.count('ss') 2 # Partition a string returning a 3-tuple including the portion of string # prior to separator, the separator # and the portion of string after the separator >>> x = "Hello, my name is Josh" >>> x.partition('n') ('Hello, my ', 'n', 'ame is Josh') # Assuming the same x as above, split the string using 'l' as the separator >>> x.split('l') ['He', '', 'o, my name is Josh'] # As you can see, the tuple returned does not contain the separator value # Now if we add maxsplits value of 1, you can see that the right-most split is # taken. If we specify maxsplits value of 2, the two right-most splits are taken

Sikuli
>>> x.split('l',1) ['He', 'lo, my name is Josh'] >>> x.split('l',2) ['He', '', 'o, my name is Josh']

Satish Gorripotu

Lists Defining Lists # Define an empty list my_list = [] my_list = list() # rarely used # Single Item List >>> my_list = [1] >>> my_list in the >>> # interpreter [1] # Define a list of string values my_string_list = ['Hello', 'Jython' ,'Lists'] # Define a list containing mulitple data types multi_list = [1, 2, 'three', 4, 'five', 'six'] # Define a list containing a list combo_list = [1, my_string_list, multi_list] # Define a list containing a list inline >>> my_new_list = ['new_item1', 'new_item2', [1, 2, 3, 4], 'new_item3'] >>> print my_new_list ['new_item1', 'new_item2', [1, 2, 3, 4], 'new_item3'] # note that there is no need to use print to display a variable

Sikuli

Satish Gorripotu

Accessing a List # Obtain elements in the list >>> my_string_list[0] 'Hello' >>> my_string_list[2] 'Lists' # Negative indexes start with the last element in the list and work back towards the first # item >>> my_string_list[-1] 'Lists' >>> my_string_list[-2] 'Jython' # Using slicing (Note that slice includes element at starting index and excludes the end) >>> my_string_list[0:2] ['Hello', 'Jython'] # Create a shallow copy of a list using slice >>> my_string_list_copy = my_string_list[:] >>> my_string_list_copy ['Hello', 'Jython', 'Lists'] # Return every other element in a list >>> new_list=[2, 4, 6, 8, 10, 12, 14, 16, 18, 20] # Using a third parameter in the slice will cause a stepping action to take place # In this example we step by one >>> new_list[0:10:1]

Sikuli
[2, 4, 6, 8, 10, 12, 14, 16, 18, 20] # And here we step by two >>> new_list[0:10:2] [2, 6, 10, 14, 18]

Satish Gorripotu

# Leaving a positional index blank will also work as the default is 0 for the start, and the length of the string for the end. >>> new_list[::2] [2, 6, 10, 14, 18]

Modifying a List # Modify an element in a list. In this case we'll modify the element in the 9th position >>> new_list[9] = 25 >>> new_list [2, 4, 6, 8, 10, 12, 14, 16, 18, 25] Boolean Expressions Example: Testing a String >>> mystr = '' >>> if mystr: ... 'Now I contain the following: %s' % (mystr) ... else: ... 'I do not contain anything' ... 'I do not contain anything' >>> mystr = 'Now I have a value' >>> if mystr: ... 'Now I contain the following: %s' % (mystr) ... else: ... 'I do not contain anything' ... 'Now I contain the following: Now I have a value'

Using Expressions to Control Program Flow if-elif-else Statement

Sikuli
Example: Standard if-elif-else

Satish Gorripotu

# terminal symbols are left out of this example so that you can see the precise indentation pi =3.14 x = 2.7 * 1.45 if x == pi: print 'The number is pi' elif x > pi: print 'The number is greater than pi' else: print 'The number is less than pi'

Example: Evaluate Empty List # Use an if-statement to determine whether a list is empty # Suppose mylist is going to be a list of names >>> mylist = [] >>> if mylist: ... for person in mylist: ... print person ... else: ... print 'The list is empty' ... The list is empty while Loop Python while Statement >>> x = 0 >>> y = 10 >>> while x <= y: ... print 'The current value of x is: %d' % (x) ... x += 1 ... else: ... print 'Processing Complete...' ... The current value of x is: 0 The current value of x is: 1 The current value of x is: 2 The current value of x is: 3 The current value of x is: 4 The current value of x is: 5 The current value of x is: 6 The current value of x is: 7 The current value of x is: 8 The current value of x is: 9 The current value of x is: 10 Processing Complete...

Sikuli
esetting Counter Using with-else >>> total = 0 >>> x = 0 >>> y = 20 >>> while x <= y: ... total += x ... x += 1 ... else: ... print total ... total = 0 ... 210

Satish Gorripotu

continue Statement # Iterate over range and print out only the positive numbers >>> x = 0 >>> while x < 10: ... x += 1 ... if x % 2 != 0: ... continue ... print x ... 2 4 6 8 10

break Statement >>> x = 10 >>> while True: ... if x == 0: ... print 'x is now equal to zero!' ... break ... if x % 2 == 0: ... print x ... x -= 1 ... 10 8 6 4 2 x is now equal to zero! for Loop

Sikuli
Comparing Java and Python for-loop Example of Java for-loop for(x = 0; x <= myList.size(); x++){ // processing statements iterating through myList System.out.println("The current index is: " + x); } Example of Python for-loop my_list = [1,2,3,4,5] >>> for value in my_list: # processing statements using value as the current item in my_list ... print 'The current value is %s' % (value) ... The current value is 1 The current value is 2 The current value is 3 The current value is 4 The current value is 5

Satish Gorripotu

Enumerate() Functionality >>> myList = ['jython','java','python','jruby','groovy'] >>> for index, value in enumerate(myList): ... print index, value ... 0 jython 1 java 2 python 3 jruby 4 groovy If we do not require the use of an index, it can be removed and the syntax can be cleaned up a bit. >>> myList = ['jython', 'java', 'python', 'jruby', 'groovy'] >>> for item in myList: ... print item ... jython java python jruby groovy

Exception Handling - Need to work on it JDBC - Need to work on Data Base conncetions

Sikuli

Satish Gorripotu

from __future__ import with_statement from com.ziclix.python.sql import zxJDBC # for example jdbc_url = "jdbc:postgresql:test" username = "postgres" password = "jython25" driver = "org.postgresql.Driver" # obtain a connection using the with-statment with zxJDBC.connect(jdbc_url, username, password, driver) as conn: with conn: with conn.cursor() as c: c.execute("select name from country") c.fetchone() Excel Handling - Need to work on Test tada and Test results

Referenced and consolidated from below sites: http://sikuli.org/trac/wiki http://sikuli.org/trac/wiki/TitleIndex http://sikuli.org/trac/wiki/IPAddressMac-0.10?action=history http://sikuli.org/trac/wiki/WikiStart http://sikuli.org/trac/wiki/reference-0.10 http://sikuli.org/trac/wiki/RaiMansExamples http://sikuli.org/trac/wiki/The%20Complete%20Guide%20To%20Sikuli%20X http://sikuli.org/trac/wiki/A%20Quick%20Tour%20of%20Sikuli%20Script%20IDE?action=diff&ver sion=7 http://sikuli.org/trac/prefs http://sikuli.org/docx/ http://sikuli.org/documentation.shtml http://script.sikuli.org/documentation.shtml#doc/pythondocpython.edu.mit.csail.uid.Sikuli.html#python.edu.mit.csail.uid.Sikuli.type-function http://sikuli.csail.mit.edu/trac/wiki/TitleIndex http://www.csail.mit.edu/search/node/sikuli

Sikuli

Satish Gorripotu

http://www.makeuseof.com/techfun/search/?cx=009717636731598800244%3Aqhe4rh7wuxs&cof=FORID%3A11&q=sikuli#773 https://launchpad.net/sikuli/trunk/0.10 http://jythonpodcast.hostjava.net/jythonbook/en/1.0/ http://www.jython.org/jythonbook/en/1.0/LangSyntax.html#while-loop http://www.jython.org/jythonbook/en/1.0/LangSyntax.html#for-loop http://www.jython.org/jythonbook/en/1.0/InputOutput.html#file-i-o AutoHotkey : same like Sikuli, But takes Keyboard actions. http://www.autohotkey.com/docs/Tutorial.htm AutoHotkey supports image recognition and macro recording. I don't see much here that is new. The bus proximity and lasso search examples are both great ideas, but could each be easily implemented using either AutoHotkey or AutoIt.