Skip to content
All Articles
Automation·By ··8 min read

Cost to Automate a Manual Process: 5 Hours to 15 Minutes

What it costs to automate a manual process, from a real build: where the money actually goes, what the estimate missed, and how to size the payback first.

AutomationDagsterPythonCostGoogle Sheets

Requests for quote arrived hourly. Each one had to be read, priced against stock and supplier data, and answered. Pricing sat in one team, sourcing in another, and the working copy of everything was a spreadsheet that several people had open at once.

The backlog was not dramatic. It was just permanent. Quotes went out a day late, and two people quoting the same part could reach two different numbers, because they were looking at the sheet at different moments.

Handling one request took about five hours end to end. After the rebuild it took under fifteen minutes. That is the honest headline, and the useful part of this post is not the number. It is where the cost to automate a manual process actually sits, because it is almost never where the estimate puts it.

First, price the process you already have

Before anything is built there is a number worth calculating, and most teams never do.

Take the time one pass through the process takes. Multiply by how often it runs, then by the loaded hourly cost of whoever performs it. Add the rework: corrections, chasing, the meeting where two people reconcile conflicting versions. That last part is usually a third of the total and it hides because nobody logs it as process time.

For the quoting workflow above, five hours a request against a steady hourly arrival rate is not a productivity problem. It is a capacity ceiling. Every request beyond what the team could physically process became a slower answer to a customer, and in a competitive market a slow quote is functionally a lost one.

That framing matters more than any efficiency argument. Finance teams reliably fund things that remove a ceiling and reliably defer things that improve a percentage.

The spreadsheet ceiling is real and it is documented

A lot of manual processes are already semi-automated inside a spreadsheet, and the question is not whether to automate but when to move off it. There are published limits that answer that, and they are worth knowing before you spend anything.

Google's own documentation puts a hard cap on a Sheets file at 10 million cells or 18,278 columns. That sounds enormous until a process appends rows hourly and nobody prunes history.

The tighter constraint is the scripting layer. The Apps Script quota documentation sets script runtime at 6 minutes per execution. Triggers get 90 minutes of total runtime per day on a consumer account and 6 hours on Workspace. Outbound calls are capped at 20,000 a day on consumer accounts, 100,000 on Workspace.

Read those together and the ceiling becomes obvious. A sync that grows past six minutes cannot finish, no matter how it is written. A process that needs to run every fifteen minutes will exhaust a consumer trigger budget partway through the day. There is no configuration that fixes either. You have hit the edge of the tool.

That was the situation with the quoting workflow. The sheet could not fetch dynamically, could not deduplicate reliably, and had no way to tell anyone a run had failed. It had not been a bad choice. It had simply been outgrown.

Where the money actually goes

Three buckets, roughly in the order people underestimate them.

1. Understanding the process, including its exceptions

This is the expensive part and it looks like talking.

The documented process is never the real one. The real one has an exception for a particular supplier, a rule about which customers get manual review, and a step that exists because of something that went wrong in 2023 and was never written down. Every one of those has to surface before it can be encoded, and they surface by asking the people who do the work, repeatedly, until the answers stop changing.

Skip this and you build a system that automates the happy path and pushes every exception back to a human, which produces a smaller version of the original problem plus a maintenance burden.

2. The last mile into somebody's existing workflow

More on this below, because it is the one that broke my own estimate.

3. Running it afterwards

An automation is a system now. It needs somewhere to run, monitoring so failures are noticed, and someone to fix it when an upstream format changes. Hosting is usually trivial. The attention is not, and it is the line most often left out entirely.

The design decision that keeps this bucket small is isolation. In the quoting rebuild the jobs were decoupled and config-driven, so any single job could fail and be re-run on its own without replaying the whole chain. That sounds like a detail. It is the difference between a five-minute fix and an afternoon.

What my estimate missed

Phase one delivered the pipeline. Dagster orchestration, scheduled jobs, a visual UI showing every run and its status. Data flowed. Failures were visible instead of silent. By any engineering measure it was finished.

It was not adopted.

The team's work still lived in the spreadsheet. A pipeline that ran correctly somewhere they did not look was, from their side, roughly as useful as the backlog they already had. They had not asked for a data platform. They had asked to stop retyping things.

Phase two is what actually closed it: a Flask REST API updating the sheets directly, plus an Apps Script sidebar so someone could trigger a refresh from inside the spreadsheet without learning a new tool. Rolling sheets synced to the database hourly instead of weekly.

Same pipeline. Completely different outcome. Quote handling dropped to under fifteen minutes, data errors went to near zero, and the 80% time saving is measured from the point people actually used it.

Sizing it before you commit

A method that takes an afternoon and is far more reliable than a vendor calculator.

  1. Time one real pass, end to end. Include the waiting and the corrections. Not the version people describe from memory.
  2. Count the frequency honestly. Hourly arrival is a different problem from a weekly batch, and it changes the architecture rather than just the schedule.
  3. List the exceptions. Ask the person who does the job what makes a request unusual. However many they name, there are more. Anything under about 20% exception rate automates cleanly. Above roughly half, you are looking at a decision-support tool instead, and you should scope it as one.
  4. Name where the output has to land. The existing system, the existing sheet, the existing inbox. If the answer is "a new dashboard", check that anyone will open it.
  5. Add the operating line. Hosting plus a realistic maintenance allowance. Small, and not zero.

Then compare the annual manual cost from the first section against the build plus a year of operating. If payback lands inside a year the decision is usually easy. Past two years it needs a reason beyond cost, and "we cannot answer customers fast enough" is a perfectly good one.

When not to automate

Worth saying, because the honest answer is sometimes no.

Processes that run a handful of times a year rarely repay the effort, however irritating they are. Neither do processes about to be replaced by a system migration already on the roadmap. And a process nobody can describe consistently is not ready to be automated. It is ready to be agreed on first.

The same logic applies to buying versus building. If an off-the-shelf tool covers the workflow and you are inside its pricing tiers, use it. Building becomes the better economics when volume makes per-task pricing hurt, or when the process has enough exceptions that no product models it. I wrote up that trade-off in more detail for cloud spend in in-house, a tool, or a consultant, and the decision shape is the same here.

Frequently asked questions

How long does automating a process usually take?

Less time than the discovery around it, in my experience. The build is bounded once the rules are known. What varies is how long it takes to pin down the exceptions and to get the output into the tool people already use, and that depends on access to the people doing the work rather than on engineering.

Should we start with the biggest process?

Usually not. Start with the one that is high-frequency and well-understood. It pays back sooner, and it teaches you how your own exceptions behave before you take on something where a mistake is expensive.

What if the process changes after we automate it?

Assume it will. That is why isolation matters. Decoupled, config-driven jobs mean a change touches one step and gets re-run on its own. A single monolithic script means every change is a full redeployment and a full retest.

Can we just use a no-code tool?

Often, and you should if it fits. The ceiling arrives through volume-based pricing or through an exception the tool cannot express. The documented Apps Script and Sheets limits above are a concrete example of the same ceiling in a tool most teams already have.

What the number was worth

Five hours to under fifteen minutes, weekly syncing to hourly, near-zero errors on data that used to be retyped. The pipeline produced maybe half of that. The other half came from putting the result somewhere people already worked, which was the piece I had priced as an afterthought.

The full write-up of that engagement is on the RFQ operations automation case study, and there is a similar shape in the distributor ETL work for Unilever.

If you have a process eating hours every week and you want a straight answer on whether it is worth automating, describe it to me. I will tell you if it is a three-day job, a three-month one, or something you should leave alone.

MH

Mirza Hammad Tariq

Data & Automation Engineer with 5+ years on AWS: ETL pipelines, backend APIs and automation workflows in Python, SQL and FastAPI, built to cost less to run.

Work With Me

Get the next one by email

Occasional write-ups from real AWS data work: what the bill actually did, what broke, and how long it took. No roundups, no thought leadership. Unsubscribe whenever.

The proof

Related case studies

AutomationDelivered

RFQ Operations Automation — Joy Aviation

From a 5-hour manual backlog to sub-15-minute automated quotes

A two-phase automation for an aerospace-logistics supplier that replaced manual RFQ handling and rolling spreadsheets with Dagster orchestration plus…

< 15 minRFQ processing time
DagsterFlaskGoogle Sheets APIApps Script
View Case Study
Data EngineeringDelivered

Automated Distributor ETL for Unilever

Hands-off SFTP-to-analytics for daily sales & stock data

An automated ELT pipeline that detects distributor files landing on SFTP, validates and homologates them, and lands analytics-ready data in S3 for rea…

ZeroManual intervention
DagsterdbtPostgreSQLAWS S3
View Case Study
Keep reading

Continue reading

Data Engineering

AWS Cost Optimisation: In-House, a Tool, or a Consultant?

Fix your AWS bill in-house, buy an automated cost tool, or hire someone? A straight comparison of what each option reaches, and when each is wasted money.

AWSCost OptimizationFinOpsHiring
Data Engineering

How to Schedule a Python Script to Run Automatically

How to schedule a Python script to run automatically: cron on Mac and Linux, Task Scheduler on Windows, the schedule library, and the gotchas that break them.

PythonAutomationCronData Engineering
Data Engineering

AWS Lambda Deployment Limits and How to Deal With Them

AWS Lambda deployment limits explained: layers don't raise the 250 MB cap. What actually counts, what to delete first, and when a container image wins instead.

AWS LambdaPythonServerlessAWS
Taking on new engagements · Remote worldwide

Have a data, automation or AI decision worth getting right?

Whether that is ongoing technical ownership, one independent read on a problem, or someone to run the project. Let's scope it — I reply within one business day.