Thank you… motion sensor hand towel machine. You never work, so I just end up looking like I’m waving hello to a wall robot.
-Jimmy Fallon via BrainyQuote
I’ve been interested in the automation of our tasks. One of my most memorable inspirations was the scene in the “Green Hornet” where Jay Chou, playing tech-savvy Kato, makes an amazing cup of latte that you can practically taste and smell through the screen, using a beautiful, complicated-looking coffee machine (that he built!) by pushing a few buttons and pulling a few levers!
I also remember when I came out of college and worked a temporary job that primarily had me entering data, which I found very boring. I wish there was a program to do that work for me! (there are but we won’t cover that here).
I recently picked up a book called “Automate the Boring Stuff with Python,” by a very funny and smart guy named Al Sweigart. In his book, he talks about how you can do really cool things using Python like creating watermarks on a large number of photos at the same time and controlling your mouse and keyboard through a GUI. I will be talking about the latter topic in this post.
In this book, Sweigart covers controlling the mouse and keyboard with a module that he created called “PyAutoGui.” With this module, you can program your computer to do things like move your mouse to a specified spot, open programs for you at a chosen time, scroll up and down pages and many other applications. The first time I messed with PyAutoGui, I was mind-blown when my cursor instantaneously moved to a coordinate on my screen, all by itself! It almost felt like I was sharing my screen with someone that had full access to my computer. From there I was hooked- now I wanted to see what else I could do it. How about creating a script that likes all of someone’s Instagram posts?
First I wanted to understand how the computer screen coordinates work. On Sweigart’s PyAutoGui site, the docsclearly explains how to use it. On a 1440x900 screen, coordinates (0,0) would be the top left corner of your screen and (1439,899) would be the bottom right corner. One important note is that when you pull the curser down the screen, the coordinate increases. Here is a diagram of the screen and how to read the coordinates on a 1920x1080 screen taken from the site.
0,0 X increases -->
| | Y increases
| | |
| 1920 x 1080 screen | |
| | V
+---------------------------+ 1919, 1079
When I first started thinking about how I would do this, I tested its applications by manually inputting the coordinates of where to click and how many times it would click. I had a friend’s Instagram pulled up on my laptop.
This code block essentially moved to the Instagram heart’s coordinates, clicked it, and then moved to the next page. Pyautogui.moveTo() tells the mouse to go to that coordinate. Pyautogui.click(), where there are adjustable parameters, clicks the specified number of times and waits 0.5 between each click. Pyautogui.press() presses the specified key. Very intuitive! Because this code was written manually, this method will too long especially when the location of the Instagram heart moved after every next photo. The next step shows you how Pyautogui locates each heart automatically using Pyautogui.locateOnScreen().
With this method, you need have a photo saved on your computer where you can access it by inserting it into its parameter. Essentially it searches for the exact picture down to the pixel. If even one pixel is off, it will return None and the code will break. First I screen-shotted the heart, then I entered the file name like so Pyautogui.locateOnScreen(heart.png). After the heart is located, I used another method called Pyautogui.center(‘heart.png’) that centered the cursor on the heart. Afterwards I created a loop that clicks the heart and presses the ‘right’ direction key to get to the next photo.
Now that it can like all of someone’s posts, I could potentially have it leave a comment at the end of the posts, leave the screen, go to your mutual friends, and find another person’s Instagram page to like. Although this was a simple project using only python, it’s interesting to note that people would pay for this type of bot service to automatically follow people, leave many many comments, and post pictures on its own.