The Personal Data Pipeline Automate your reporting in Google Sheets

TL;DR: If you export CSV reports from software tools, and do some post-processing in spreadsheets, you should think about building a personal data pipeline to automate it.  Start yours here.


Since the dawn of time, there’s been a gate blocking you from doing your job.

The export button.

It seems friendly at first – you can export all your data!  By just clicking this one button.

But like an iceberg, the risk is in what you *can’t* see.

The exported data’s never in the exact right format.  You have to delete this line, copy-paste just these rows.

And you have to repeat that process every day, week or month, until the end of the universe.  For every software tool that you use.

You *become* the process that you built.

Q: “What do you do for work?”

A: “I download CSV reports and format them.  It pays pretty well.”

Not very inspiring – it makes building a Jump to Conclusions Mat look like a good career alternative.

And the great paradox?

Everyone I know who runs reporting – marketers, founders, and project managers – are *really* bright people with much better things to do with their time.

Reporting is the part of their job that they hate the most, without fail – it’s comically beneath their skillset, while being absolutely critical.

So how to exercise this demon?

The best companies build data pipelines – engineered processes to deliver data to the fingertips of the non-technical folks, without risking anyone having to get cooties from *touching* the data.

But pipelines are certainly not lean – like an oil & gas pipeline, they’re a piece of infrastructure that are constructed and maintained.

They’re built for companies, not individuals.

But as individuals, we can learn a lot from *how* they’re built.  At the center of a data pipeline is an ETL process, that does 3 things:

1) Extracts data from a source: whether that’s your own database, or something like SEMrush using an API

2) Transforms data into a usable format: removing unnecessary columns, reformatting dates, etc

3) Loads data into a database, acting as a single source of truth for the company

What I propose, to cure the corrosive time-sucking void of reporting, is a *personal data pipeline*.

A systematic process, that does this skullduggery for us – so that we can spend time actually taking action on what our reporting tells us.

How the Personal Data Pipeline works:

1) Pull exactly the data you need (from APIs) into Google Sheets, by pressing a button in a Sheet

2) Use Google Sheets Queries to transform your raw data into metrics

3) Press a button to snapshot those metrics and produce a report (or push the metrics to a dashboard like Google Data Studio)

I truly believe we’ll be able to place the tyranny of manual reporting on the scrap heap of history, where it belongs.

Are you in?  Drop your email below to start building your pipeline with the help of CIFL.

Submit a comment