Automating my finances with Puppeteer (logins and acct balances)

Disclaimer: not a tutorial

Repo

This is using Puppeteer on NodeJS with Google Spreadsheet API to automate website logins and get data eg. current balance and fill in a Google Spreadsheet that’s used as a DB for other personal APIs.

Demo

Here’s a video

Yeah I filmed this with a camera facing my monitor like a noob

This is ran locally but at this point you would just move the code to a server/tie it to a CRON job. This example is hitting up bankofamerica.com and logging in as me, waiting for me to provide the 2FA sms code through Twilio and then getting the balance to enter into the spreadsheet on the right.

Background

Back in 2015 or so my finances tanked. My credit score dropped to the low 400’s or even lower. Although I’m still broke AF, right now I’m above mid 700’s but it took me a bit to fix that. One of the fixes is building up my credit with a lot of “shit” cards. So as part of my sanity (what my worth is, am I F’d if I get fired, etc..) I tabulate my finances. So usually at any given time I know what my “worth” is (still in debt).

So here you can see that I have 19 things to look up and fill out. Which ultimately boils down to long-term view on debt/burn rate(how many months I have to live if I was fired), etc..

I timed myself and it takes about 10 minutes to fill out the full row (19 columns or websites). But the beauty in this is I use Google spreadsheet as an API database for other sub apps.

Workflow

The TL;DR is this is a manual process (at first). There are Chrome extensions I believe that record your DOM events eg. clicks/values entered into fields so that’s easier than manually checking out the DOM make up of a website.

Mostly I wanted to point out this diagram here so it is understood what is happening. Just before the 2FA request is done in the bofa example, I notify myself of an incoming 2FA sms text from bofa. Once I get it, I send it to the Twilio number which forwards it to my own endpoint. Then the auth-looper that is continuously checking every 5 seconds for an auth code suddenly gets one and returns it to the login script that’s blocked by the 2FA prompt. Then the login script enters the SMS auth code and proceeds to login then get the balance.

Tasks

• ✓ install Puppeteer
• ✓ do a basic test and try to login to one of the bank/cc sites
• do the rest of the websites in a way that is a config, maybe like a J̵S̵O̵N̵ ̵f̵i̵l̵e̵ actually just decided to use single files in the interest of development speed
• ✓ add to Spreadsheet/update what I currently have
• tie to an app button or schedule

Extras

• add a prompt if there is a problem like a captcha, should be able to send me a screenshot of it and I can send the answer back (not sure how well that would work)
• ✓ method to address 2FA
• add error alerting system in case UI changed or something on site

General process of a login per site

Here’s an example process on how to write a login procedure for BofA. Open their page up in Incognito mode so that there’s no auto-fill based on your past history. You can also see I opened the page in 1280x720 due to responsive issues i.e. some things are missing in mobile.

Pretty much open your page, inspect it, find ids for target elements or use DOM navigation if no ids present

First problems (2FA)

So for Bank of America if you login from some new device (or script in this case) you’ll get a page like below: (can see I’m running this by code through Puppeteer).

So for any site that has 2FA you’ll have to factor this in.

Below you can see code process and the interaction using Android SMS and a Twilio number.

I just manually updated the Twilio Auth API since it’s not developed yet as part of the full bofa process and it works. So I gotta develop the Twilio API now. Theme is Earth Song

Closing thoughts

Well this is pretty amazing, I mean you don’t have to worry about sessions… I remember that was a problem last time I tried stuff like this eg. with PhantomJS but that was a while back. Lots of room for improvement but I wanted to get it done in the weekend. I actually have been working on a hardware project with a Seeeduino for the last two weeks but I wanted to do this real quick. I’ll post a write up on that as well next.

At this time I have not tried this on a remote server yet, I will deploy it but I’m kinda spent now. Been working on this 14 hours+ straight like a mad man.