As the title implies, this is the new version of a project I built a few years ago, called Goldwatch. It tracks, tweets, and adds to a Spotify playlist the songs that get played on the Melbourne radio station Gold 104. Head back and read the post about that for some more background. If you don’t have time to read two articles, here is the key takeaway:
“If I was going to build something like this now, I would do it very differently…I might resurrect it one day, probably as a Rails app.”
I was right. I would do it very differently. But I was wrong. I wouldn’t use Rails. That’s far too heavy duty for what it needs to do. In mid 2019 it occurred to me the backend didn’t need to do much, and only needed to run periodically, so it could be a serverless function. Around that time Gold 104.3 started playing some really wack shit they’d never played before, and I decided the world really needed a database keeping track of it. So I decided to get to work on Version 2.0, and it would be much simpler than the thing I hammered together last time. There are two halves:
- the backend, whose algorithm is basically “check which song is playing, and tweet about it and save it in the database if it isn’t the same song as last time we checked”
- the frontend, which queries the database
I put it together in Google Cloud, mainly because I wanted Jeff Bezos – just for once – to not win this round. Google is definitely the Pepsi to Amazon’s Coke in the cloud world but from what I could see Google had a worthy competitor to AWS Lambda that also ran functions in the cloud, and they called it…you guessed it…Cloud Functions. The frontend was in Node, because I hadn’t tried that for a while and wanted a refresher. My only regret is my choice of database. Google seem to steer you toward the technologies they want you to use by making them free, and as my cloud function would sneak in under the free tier limit, I really wanted a free database too if I could make it work. For this reason I used Google Firestore. It’s not Firebase. It’s not Datastore. It’s Firestore. All these APIs with crazily similar names were starting to remind me of Amazon. It’s a NOSQL database, which I didn’t know much about, but after a bit of reading I decided it would work. I’d regret that decision later, as I started to really miss SQL when I realised there were some really basic SQL things that just wouldn’t work in NOSQL. Here’s one of my favourites.
“Oh, it can’t do joins? I can’t get the songs and join the artist names from the artist table? I just have to store everything everywhere? Duplicating data is ok all of a sudden?”
You also can’t group by one thing and then sort by another. Or sum things. You have to do that yourself after you fetch all the data. A tutorial on YouTube told me I should design my database according the queries I wanted to make. I disagreed, and told him I shouldn’t have to. But he couldn’t hear me. It was a video. I decided that either NOSQL (or just Firestore, or both) sucks or it’s just not the right tool for this job. Or both. But the result of using Firestore was that I couldn’t make the queries I wanted to. My dream of a chart showing what time of day Dreams by Van Halen gets played, and whether it gets played more than Dreams by Fleetwood Mac, was going to stay a dream. But I was already pretty far down the ladder, and figured that maybe one day I could build version 3.0, bringing SQL back into the fray.
My advice if you want to get into Google Cloud is name your project carefully! You can’t change it later. That’s why I’m stuck with this URL. I might move it one day but for now it lives here: