TLDR: To track my Coinbase portfolio for free, I Dockerized an existing script that generates portfolio metrics and ran it continuously on AWS. Read on to learn about my motivations and the process. Code is located on Github. Thanks to Matias B for providing the original tool.
I generally have a good experience using Coinbase to HODL cryptocurrency, but it is completely missing portfolio metrics. Portfolio metrics may not seem important when you buy your first batch of crypto. But after compulsively refreshing the Coinbase app to watch crypto prices yo-yo, you will ask yourself a simple question.
Am I up or down?
At first, you may be able to answer this question before that bead of sweat fully forms on your forehead. Maybe you simply remember the price at which you bought the crypto. Maybe you find Coinbase’s transaction email notification. Maybe you learn about getting your transaction history via Coinbase Reports. Either way, you’re able to determine whether you’re up or down, and life goes on.
That is, life goes on until your portfolio becomes more complicated. As you buy and sell various cryptocurrencies at various prices, this question becomes more elusive. Eventually, it will become impossible without some tooling.
Understanding your portfolio’s performance is really important for two reasons:
- You need to understand your unrealized gains/losses to profit from trading crypto assets.
- You need to include your crypto earnings in your taxes.
When I failed to find Coinbase portfolio metrics, I was convinced that I was missing a hidden tab in the Coinbase interface, stumped by a clunky user experience. Then I learned that I was not alone. I found multiple discussions about this hole in the crypto trading journey. It was pandemonium. Reddit power users lamented that noobs did not have enough braincells to instrument an auto-updating performance spreadsheet, while official Coinbase documentation shoved users towards a paid service called CoinTracker.
I discovered that CoinTracker offered a shiny interface out of the box, but most features would cost at least $15/mo, and the tax report would be at least $48 for anyone with over 25 transactions. Meanwhile, a Coinbase competitor called Voyager seemed to offer better metrics, but I did not feel like moving all of my crypto holdings just yet. Instead, I wanted to find a free, DIY solution to get some high-level portfolio metrics.
Eventually, I found a very detailed blog post offering a Python script that could read information from my Coinbase account and dump the results in a Google Spreadsheet. I was delighted to find that the whole thing worked as expected.
But those metrics were frozen in time, and I wanted to continuously update the data without entering a daily terminal command. While there are lots of ways to do this, I decided to use AWS to continuously update my portfolio metrics. This approach would not require my own computer to be online, and it would dovetail nicely with my separate goal of gaining more AWS experience (see prior post).
Continuously Updating Portfolio Metrics on AWS
To continuously update my Coinbase portfolio metrics, I would require just two components:
- A scheduler that continuously triggered a portfolio update
- An AWS function or module that would run the original script
For the scheduler, I chose to use AWS EventBridge. For running the script, I decided to package the code into a Docker container and run it with AWS Fargate on AWS Elastic Container Service. The whole setup looks like this:
This setup is pretty straightforward, but I will describe two more interesting pieces of the instrumentation in detail: improving the security of the script and debugging EventBridge failures.
Improving the Security of the Script
While the original script works, it has some security vulnerabilities that make it undesirable for the cloud:
- It puts the Coinbase API Key and Secret directly into the Python code. You should generally never put sensitive information like passwords or keys directly in code. There are a lot of good reasons for this, and the link explains them better than I can.
- It passes the Google Service Account credential to the script via a local file. Whoever possesses this credential has the ability to act as the service account. While nobody can access the file on a local machine, they actually can access the file once it is packaged into a Docker container. So, passing the credential via file works locally but not on AWS.
- The Google Service Account is given Project Owner permissions in the original instructions. This is a very powerful permission that allows the Service Account to do anything possible in Google Cloud. In the wrong hands, the credential could be used to spin up virtual machines and mine crypto, leaving you with a staggering credit card bill. Some lucky bandits would eventually benefit from this post, but that is beside the point.
The first two problems can be solved with environment variables and AWS Parameter Store. Environment variables allow a developer to place secret information outside of the code. The code can still dynamically fetch this information via an interface. In Python, the language used for this script, it looks like this:
import os
key = os.environ.get('COINBASE_KEY')
scrt = os.environ.get('COINBASE_SECRET')
google_creds = os.environ.get('GOOGLE_COINBASE_CREDS')When running the update script on your local machine, which is useful for development, you can simply make sure these variables are exported.
export COINBASE_KEY="XXXXXX"
export COINBASE_SECRET="YYYYYY"On AWS, we can put the secret information into the Parameter Store. Then, when we set up our ECS Task Definition, we can use Parameter Store to pass the environment variables to the Docker container.
To make passing these variables as secure as possible, we can restrict the System Parameter permissions given to the Task Execution role (which executes the Docker Container on ECS) to include only these specific parameters. This granularity is nice because it allows us to follow the principle of least privilege when setting up the ECS task role.
Note that to securely pass the Service Account credential, I dumped the entire JSON blob into a System Parameter value. This string can be used instead of passing a file to the code. Because I found this change somewhat tricky to get right, I think it is worth pasting the code sample:
import json
import os
from google.oauth2.service_account import Credentials
google_creds = os.environ.get('GOOGLE_COINBASE_CREDS')
scopes = ['https://spreadsheets.google.com/feeds', 'https://www.googleapis.com/auth/drive']
service_account_info = json.loads(google_creds)
credentials = (Credentials.from_service_account_info(service_account_info, scopes=scopes))Remember, the final issue was the sweeping Project Owner role that was granted to the Service Account. It turns out that this role is totally unnecessary. It is a confusing story, but the permissions model used by Google Cloud (Google Cloud IAM) is completely separate from the permissions model used by Google Apps (Drive Sharing). The Service Account is actually given permissions to write to the Google Sheet via Drive Sharing (the little share button in the Docs UI). It will successfully write to the spreadsheet even with no IAM permissions at all! So, this issue is easily solved by giving the Service Account zero IAM permissions. This ensures that even if a hacker got ahold of the credential, the worst they could do is overwrite your Coinbase Portfolio spreadsheet. More details on these two permission models can be found here.
Debugging EventBridge Failures
After setting up my ECS Task Definition and EventBridge Rule, I sadly discovered that my spreadsheet was not being updated. I poked around in the EventBridge metrics, but all I could find was a graph of failed invocations over time.
This graph let me know that something was wrong, but it did not answer the obvious next question of why. As I dug around AWS and searched the web, I found that many users shared this consternation. Some claimed that scheduling a Fargate task from EventBridge was simply not possible. To make matters even more confusing, it seems that scheduling Fargate on EventBridge only got support in 2018, so it actually was impossible in the past.
Fortunately, I learned that AWS CloudTrail retains an event history, and I was able to find instances of the RunTask event that is triggered by the EventBridge scheduler.
Clicking one of the RunTask entries revealed a lot more detail about the event, including a very suspicious error message.
"errorCode": "InvalidParameterException",
"errorMessage": "Network Configuration must be provided when networkMode 'awsvpc' is specified.",I learned that Fargate always runs with awsvpc network mode, and this means a corresponding network configuration must be provided in the EventBridge Rule (more details in this helpful post from Jimena Mora). It turns out that the EventBridge UI subtly hints that you must specify network configuration when using awsvpc, but it does not spell out that this always applies to Fargate, and it certainly does not use this information for form validation.
And so finally, with correct network configuration and secure variables, I discovered that my Coinbase portfolio was updated continuously! Mission accomplished.
Conclusion
This exercise taught me a lot about AWS debugging and Docker best practices. Some takeaways I had:
- The AWS UI is pretty clunky, but it usually has what you want, somewhere…
- Crypto taxes actually seems pretty complicated, so I would still consider paying for the CoinTracker tax report
- The best way to avoid all of these shennanigans is to simply keep HODLing!
Finally, I should note that while my goal was to make a tool that was completely free, I have observed that the continuous updates seem to cost a few cents a day, so I did not entirely meet my goal. But, I think this is a good compromise for the learning I got and the decoupling of the job from my personal computer.