Trying to upload transactions to Mint using a CSV and Python? Find the source code and examples on Github.
- Why You Might Need to Add Transactions into Mint
- My Search for Tools that do Mint Import Transactions
- My Python 3 Import CSV Tool
Why You Might Need to Add Transactions into Mint
To create a budget you are going to want data to base spending habits on. While mint.com allows for up to a 3-month auto-import, it is not guaranteed. This also neglects periodic purchases like new brakes or tires for your car.
The whole reason to have Mint is to be able to see all the data in one place. All in all, let’s face it: The software’s “from 30 to 90 days” just doesn’t cut it for new users.
My Search for Tools that do Mint Import Transactions
Having worked in technology for as long as I have, I knew I could not be the only one with this problem. So I immediately went on Google to see if there was anyone else who had solved this issue for me.
Low and behold, I did find a YouTube video of a proof of concept approach for automating manual transactions with a plug for their premium product. There was no code available though, so I would have to reverse engineer that myself.
Not wanting to totally start from scratch, I figured I’d check out GitHub too. I stumbled upon some similar code specifically for the UK locale and date formats. It was clearly written by someone else like me who had a one-off problem and decided to solve it themselves.
The code did lack some features and process clarity that I needed, so I figured I’d write my own version with the documentation where other people could easily extend or augment it for themselves in the future.
My Python 3 Import CSV Tool
I chose Python for the ease of use of an interpreted language (one that is compiled on the fly by the machine, not the user) and the ease of PIP for extending functionality for things like CSVs.
Python Package Dependencies
We are going to need the following Python packages for our script. For those not familiar with installing packages via PIP, we have included instructions in the import.py documentation.
""" ################################# Pre-requisites needed ################################# If you are missing any of the following you can install with: pip install $name Example: pip install csv OR if you are using pip3 pip3 install $name Example: pip3 install csv """ import csv import datetime import os import random import requests import time import urllib.parse
- CSV – used to parse the input file
- datetime – used to manipulate date strings and objects
- os – used to interact with our local machine
- random – used to implement random numbers
- requests – used to submit HTTP requests (note we are actually just using curl right now)
- time – used to interact with time strings and objects
- urllib – used to HTTP encode strings so they can be submitted over POST and form data
Process to Importing Transactions
- Upload CSV data to python
- Process date for correct format and HTTP encode the result
- Process merchant text for HTTP encode
- Process categories and change your bank’s category name into a Mint category ID (limited in scope based on the categories I needed when I wrote this)
- Process amount for positive or negative value, indicating the transaction data is income or an expense
- Create CURL string post request
- Send POST Request to Mint as new transaction data.
- Force Randomized Wait Time before starting next request
Setting Up a Mint CSV Import Client
Because there is no public Mint API, we are going to need to find a way to mimic manual transaction adds using the UI’s form post.
William Lorfing (Intuit Developer Group)3 years ago
There are no APIs to access Mint data.Mint HelpDesk as of 05/04/20
To do this we are going to need to use your browsers DevTools:
""" ################################# Mint Client Credentials ################################# You will need the tags, cookie, and token to simulate a UI form submission. You can get these by opening developer tools > network analysis tab and doing a test submission in mint.com. From there look for the post request to "updateTransaction.xevent" and grab the credentials from the header and body """ account = 'XXXXXXX' # grab from POST request form body in devtools tag1 = 'tagXXXXXX' # in form of tagXXXXXXX tag2 = 'tagXXXXXXX' # in form of tagXXXXXXX tag3 = 'tagXXXXXXX' # in form of tagXXXXXXX cookie = 'XXXXXXX' # grab from POST request header in devtools referrer = 'XXXXXXX' # grab from POST request header in devtools token = 'XXXXXXX' # grab from POST request form body in devtools
- Go to Mint and login
- Go to transactions and click “+transaction”
- Open your DevTools (F12 on Chrome) and go to Network Tab
- Submit a TEST transaction for $0.01, the network analysis tab will record the POST submission under “updateTransaction.xevent”
- Click on “updateTransaction.xevent” and you can see the headers of your request. Specifically, you will want the “request headers” for the cookie and the “form data” for things like tags and your token.
Setting Up a Bank’s Transactions CSV
Each bank will have a different way to get your transactions as CSV data. If your bank really doesn’t have this functionality,
- Get a new bank 😉
- Reach out to support as ask for a CSV file
- If they won’t do that, you can manually extract and copy the data into a spreadsheet, transform it into CSV format, then load it in (this is beyond the scope of this article, but we’ll have an article on scraping and text transformations later).
If you are uploading thousands of transactions like I did, you are going to want to sort your CSV by transaction date. This will allow you to retry from where it left off in the event of a failure.
What is important is understanding the format that the program expects. My script is based on a Chase.com CSVs export. If your bank uses a different format, you can adjust the column maps OR scrub it to match.
To those not familiar with reading Python, row[X] means the item X from left to right of that row starting with number 0. In my script, the first column must be the date of the transactions, followed by post date, merchant memo, categoryID, typeID, and amount of transaction (signed +-).
# Initialize Variables date = (row) postDate = (row) merchant = (row) catID = (row) typeID = (row) amount = (float(row))
Adding Transaction Categories
One of the main areas I did not fully develop is in category mapping. Mint requires you to use categoryIDs and names defined by them, which means the ones from your bank will not work. To deal with this, I have implemented two functions:
# Category ID Mapping Function def category_id_switch(import_category):
category_id_switch maps the Chase named categories to Mint category ids. To make this match your bank’s categories, you just change the keys like “Credit Card Payment” to whatever your bank calls it. To add additional category support, I have included Mint’s category menu in html format as of 05/04/20 where you can grab the category names and ids from the markup.
# Category NAME Mapping Function def category_name_switch(mint_id):
def category_name_switch maps Mint category ids to their Mint names. To add additional category support I have included Mint’s category menu in html format as of 05/04/20 where you can grab the category names and ids from the markup.
There was only one special case in my 1,800 transactions, and that is when Chase’s type = “payment”. In these scenarios, we had to handle them as catID 2101 in Mint. If your bank uses a different payment category, you can either comment this code out or map it to a category of your choosing.
# typeID payment overrides all categories if typeID == "Payment": catID = '2101' # Since I was importing credit cards I have mine set to credit card payment. If you are doing bank accounts you many want to change this to payment general
Running CSV Import
Once you’ve got your client all setup and your category maps set all you need to do is run the following command from the directory your code and the CSV are in:
python3 import.py OR if you are not using the python3 namespace python import.py
That is it! You can sit back and watch the transactions add into your profile.