PyAutoGUI - graphical user interface automation (detailed explanation)

PyAutoGUI module determines the target position through the screen xy coordinate system, controls the mouse and keyboard to send virtual keystrokes and mouse clicks, and completes operations such as clicking buttons and filling in forms. When there is no module corresponding to the boring tasks you want to operate, it is also a good choice.


Coordinate axis system of PyAutoGUI

The mouse function of pyautogui uses x and y coordinates. The origin is in the upper left corner of the screen. The x coordinate increases to the right and the y coordinate increases to the down. All coordinates are positive integers without negative coordinates.

pyautogui.size()     #Returns the tuple of screen width and height pixels
                     #For example, if the screen resolution is 1920 * 1080, the coordinates of the upper left corner are (0,0),
                    #The coordinates in the lower right corner are (19191079)

1. Control mouse

(1) Determines the current position of the mouse

pyautogui.position()                 #Determine the current position of the mouse and return the tuple of X and Y coordinates


(2) Move

pyautogui.moveTo(x,y[,duration = t])  # Move the mouse to the specified position on the screen,
                                      #x. Y is the abscissa and ordinate of the target position, and duration specifies that the mouse cursor moves to the target position
                                      #The number of seconds required, t can be an integer or floating-point number, and the duration parameter is omitted
                                      #Immediately move the cursor to the specified position (in PyAutoGUI function, all duration s)
                                      #Keyword parameters are optional)
                                      #Attention: coordinates x and y can be used for all places where x and Y coordinates are passed in
                                      #Tuple or list substitution, (x,y)/[x,y]

pyautogui.moveRel(x,y[,duration = t]) # moves the cursor relative to the current position,
#X and Y here are no longer the coordinates of the target position, but the offset,
#For example, pyautogui moveRel(100,0,duration=0.25)
#Indicates that the cursor moves 100 pixels to the right relative to the current position


(3) Click

Complete Click: press the mouse button and release it without moving the position

pyautogui.mouseDown()   #Press the mouse button (left button)

pyautogui. Mouseup # release the mouse button (left button)

pyautogui.click() # sends virtual mouse clicks to the computer (the click() function is just a convenient encapsulation of the previous two function calls)
#The default is at the current cursor position. Click with the left mouse button

pyautogui.click([x,y,button = 'left/right/middle']) # click the left, right and middle mouse buttons at (x,y)
#However, this method is not recommended. The following method is better
#pyautogui.moveTo(x,y,duration=t)
#pyautogui.click()
pyautogui.doubleClick() # double click the left mouse button

pyautogui. Rightclick # right click

pyautogui.middleClick() # middle click

(4) Drag

Drag: hold down a key while moving the mouse

pyautogui.dragTo(x,y[,duration=t)      #Drag the mouse to the specified position
#x. Y: X coordinate, y coordinate

pyautogui.dragRel(x,y[,duration=t]) # drag the mouse to a position relative to the current position
#x. Y: horizontal movement, vertical movement

(5) Rolling

pyautogui.scroll()         #Control window scrolling up and down (scrolling occurs at the current position of the mouse)
#Positive numbers scroll up, negative numbers scroll down,
#The size of the scrolling unit requires a specific attempt


#eg
sleep(2)
click()
moveTo((1418,12),duration=2)
click()
moveTo([1392,47],duration=1)
click()
typewrite('https://wwww.baidu.com')
typewrite(['enter'])

II. Control keyboard

(1) Input string

pyautogui.typewrite(s[,duration=t])      #Send string to text box
#The optional duration parameter adds a short pause between entering a single character
#Attention: can only be used to enter English


(2) Enter key string

Not all keys are easily represented by a single text character. For example, how do I represent Shift or left arrow keys as a single string? In PyAutoGUI, these keys are represented as short string values. For example, 'ESC' represents ESC key and 'Enter' represents enter. We call these strings key strings.


pyautogui.typewrite([Keyboard key string])      #In addition to a single string, you can also pass a list of key strings to the typewrite() function
#Such as pyautogui typewrite([‘a’,‘b’,‘left’,‘left’,‘X’,‘Y’])
#Press the 'a' key, the 'b' key, then press the left arrow twice, and then press' X 'and' Y '
#The output is XYab

pyautogui.keyDown() # sends a virtual key to the computer according to the passed in key string (press)

pyautogui.keyUp() # sends a virtual release (release) to the computer according to the passed in key string

pyautogui.press() # the encapsulation of the first two functions, simulating the complete keystroke (press and release)


give an example:

pyautogui.keyDown('shift');pyautogui.press('4');pyautogui.keyUp('shift')
#Press Shift, press and release 4, and then release Shift


The complete key string is as follows:

Keyboard key string                        meaning
'a','b','c','A','C','1','2','3', Single character key
'!','@','#'etc

'Enter' enter

'ESC' ESC key

Shift left, shift right

'Alt left', 'Alt right' left and right Alt keys

'Ctrl left' and 'Ctrl right' left and right Ctrl keys

'Tab '(or' \ t ') Tab key

'Backspace', 'Delete' Backspace key and delete key

'pageup', 'pagedown' Page Up and Page Down keys

'Home', 'end' Home and end keys

'up', 'down', 'left', 'right' up, down, left and right arrow keys

'F1', 'f2', 'f3' and other F1 to F12 keys

'volumemute', 'volumeup', volumedown 'mute, volume up and volume down keys

'pause' key

'capslock', 'numlock', 'scrolllock' Caps Lock, Num Lock and Scroll Lock keys

'Insert' key

'printscreen' Prtsc or Print Screen key

'winleft', 'winright' or so Win keys (on windows)

'Command' Command key (on OS X)

'Option' Option key (on OS X)


(3) Shortcut key combination

pyautogui.hotkey()           #Receive multiple string parameters, press them in sequence, and then release them in reverse order


give an example:

pyautogui.hotkey('ctrl','c')  #Press Ctrl and C, and then release
Equivalent to
pyautogui.keyDown('ctrl')
pyautogui.keyDown('c')
pyautogui.keyUp('c')
pyautogui.keyUp('ctrl')

Tags: Python

Posted by x_maras on Mon, 18 Apr 2022 22:33:20 +0930