It looks like I first started talking about MongoDB online back in 2011 - We’ve come a long way baby.
Right, need to finish watching the MongoDB vids (easier than I thought) and then do an Amazon ELB setup for kicks.
— Conor O'Neill (@conoro) May 28, 2011
Mongo has remained one of my favourite tools for getting things done. I always think that’s why it succeeded. Whilst the factions argued it out on HN, developers just got on with using it for practical purposes.
One of my big frustrations on various personal projects in recent years was in failing to find an affordable easily-accessible scalable online datastore that could also be queried. The usual suspects like DynamoDB and Cosmos DB ended up being dead-ends where, with a bunch of SDKs, you could get data in, but you had to then move it somewhere else to do anything useful with it. Some of the newer Low-Code platforms also looked attractive initially until their pricing or performance went completely out of whack with barely a few thousand rows of data.
To my shame I have to admit to using SQLite on Dropbox and Google Sheets as my de-facto databases for many of these scenarios. Hell, I even investigated using the GitHub GIST API as some sort of JSON database. Don’t worry, I gave up.
But when using LCNC tools for more general automations, I often find myself wanting to store output data somewhere. That might be low-volume IOT data like home-office CO2 levels or specific Google Alerts or a Slack event history. That’s how I ended up on Google Sheets as a pseudo-DB. It’s accessible over REST, it uses standard Auth protocols, it’s cheap and it scales to thousands of rows. However of course it does not do ACID, it blatantly overwrites rows, it’s slow and you can’t even easily find out if a Sheet already has a specific row of data. Because it’s not a DB, it’s a spreadsheet!
And then I saw last week’s announcement of the MongoDB Atlas Data API and you could not wipe the smile off my face, despite the vicious throat infection that currently has me in bed.
This isn’t just close to what I need, it’s exactly what I need. REST access to a full-blown online database that can then be used for whatever querying, analytics or visualisations I want to do afterwards 🤯
Even better - like everything historically with MongoDB, it just works! 5 minutes after reading the announcement I was writing and reading to/from a collection using cURL against my Atlas instance.
curl --request POST \
'https://data.mongodb-api.com/app/{{.RESOURCE.mongodb_data_api_app_id}}/endpoint/data/beta/action/insertOne' \
--header 'Content-Type: application/json' \
--header 'Access-Control-Request-Headers: *' \
--header 'api-key: {{.CREDENTIAL.mongodb_data_api_key}}' \
--data-raw '{
"dataSource": "{{.RESOURCE.mongodb_cluster_name}}",
"database": "learn-data-api",
"collection": "people",
"document": {
"name": "John Sample",
"age": 42
}
}'
Shortly after that, I had copied those cURL commands as Node-RED HTTP requests and was reading and writing to MongoDB in a No-Code Automation platform. I don’t think I’ve stopped grinning even whilst asleep.
I never thought I’d get this excited by a database API but it’s a step-change moment for the growth of Low-Code and No-Code. The frustration of trying to do anything meaty with most of the toy online pseudo-databases in the LCNC world has been infuriating. I wonder how many projects started well and then hit a brick wall as soon as they tried to scale? I suspect many of the “Is it a Spreadsheet? Or is it a Database?” tools may start fading away as their limitations become more obvious. The brilliant trick with MongoDB Atlas is that it’s suitable for both dabblers and hard-core fast-scaling apps and companies.
I’ve started switching the output of all of my GSheets-based automations to MongoDB Atlas and I’ll fiiiiiiiinally start doing some decent queries on the data. I have months of office CO2 data that I have yet to crunch. But switching to storing that data in MongoDB literally took me another 5 minutes:
Next up - all those Google Alerts into a proper database where they belong.