AWS Lambda Deployment Limits and How to Deal With Them

You wrote forty lines of Python. It reads a CSV off S3, reshapes it, writes Parquet back. Then the deploy fails:

Unzipped size must be smaller than 262144000 bytes

An AWS Lambda deployment package too large error is almost never about your code. It is about what pip dragged in behind it. I measured a plain pip install pandas pyarrow boto3 this week and it came to 297 MB before I wrote a single line. The limit is 250.

The good news is that most of those bytes are things Lambda will never execute.

straighten What "deployment package too large" actually means

There is not one AWS Lambda deployment limit. There are five in the quota table, they apply at different moments, and the error message does not tell you which one you hit.

The number in the error, 262144000, is exactly 250 × 1024², so it is 250 MiB rather than 250 MB. AWS concedes the point in a footnote to its own quota table: "The Lambda documentation, log messages, and console use the abbreviation MB (rather than MiB) to refer to 1,024 KB." That distinction costs you about 12 MB of headroom you might have assumed you had.

The one that catches people is the second row against the third. You can zip a package down under 50 MB and still fail, because the check that matters happens after Lambda unzips it. Compression is not a solution here. It only hides the problem until deploy time.

layers Why Lambda layers do not fix this

This is the part worth reading twice, because a lot of advice gets it backwards.

Lambda layers do not increase the 250 MB limit. AWS defines the quota as the maximum size of a deployment package's contents including layers and custom runtimes, unzipped. You can attach five layers. All five count. Moving pandas out of your deployment package and into a layer changes precisely nothing about whether you fit.

Layers solve a different problem, and they solve it well:

• Deploy speed. Your function zip shrinks to just your code, so uploads take seconds instead of minutes.
• Reuse. Ten functions share one dependency set instead of ten copies.
• Separation. Dependencies version independently of your handler.

Those are real benefits. Capacity is not one of them. If you are reaching for a layer because you are over 250 MB, you are about to spend an afternoon and arrive at the same error.

search Find out what is actually big

Before deleting anything, look. Most people guess, and most people guess wrong.

pip install --target ./pkg \
  --platform manylinux2014_x86_64 \
  --implementation cp --python-version 3.12 \
  --only-binary=:all: \
  pandas pyarrow boto3
 
du -sm ./pkg/* | sort -rn | head -10

The --platform manylinux2014_x86_64 flag matters. Lambda runs Linux, so if you build on a Mac or Windows machine without it you get the wrong wheels, and the sizes you measure are not the sizes you deploy.

Here is what that returned on my machine (pandas 2.3.2, pyarrow 20.0.0, numpy 2.2.6, boto3 1.43.74):

I expected pandas to be the problem. It is not even close. PyArrow is 47% of the package on its own, and it is there because I asked for Parquet support. Nobody warns you about that one, because everyone writes about pandas.

content_cut What to delete, in order of payoff

I ran these five steps in order on that 297 MB package and measured after each one. Real numbers, one machine, same afternoon:

297 MB to 204 MB. Nothing removed that runs at execution time.

The ordering surprised me. Deleting test suites saved twice what dropping the AWS SDK did. pandas and pyarrow both ship their full test suites inside the wheel, and on a data stack that is 49 MB of code that exists to test the library, not to run it.

On the boto3 advice you have read elsewhere

Every article on this topic tells you to delete boto3, and they are right: the Lambda Python runtime already includes the SDK for Python, so bundling your own is usually waste. The exception worth knowing is that AWS pins no particular version, and the one you get varies by runtime release and by Region. If you depend on a recently shipped API, keep your own copy and spend those bytes deliberately. You will also see a figure of around 90% savings attached to that advice.

That number is real but it is not yours. It comes from stripping the SDK out of a layer that was mostly SDK. On a package built around pandas and pyarrow, removing boto3 and botocore saved me 24 MB out of 297, which is 8%. Worth doing. Not the thing that saves you.

One caveat before you delete it. The runtime's bundled boto3 lags the current release, sometimes by months. If you depend on a recently added API, bundle it deliberately and accept the bytes. Check what you are actually getting:

import boto3, botocore, sys
print(sys.version)
print("boto3   ", boto3.__version__)
print("botocore", botocore.__version__)

Deploy that as a throwaway function and read the log. Do not trust the documented version. Trust the one your function prints.

The cleanup itself

cd ./pkg
 
# already in the runtime
rm -rf boto3* botocore* s3transfer* jmespath* urllib3* six* dateutil python_dateutil*
 
# test suites: the biggest single win on a data stack
find . -type d -name "tests" -prune -exec rm -rf {} +
find . -type d -name "test"  -prune -exec rm -rf {} +
 
# bytecode, rebuilt at runtime anyway
find . -type d -name "__pycache__" -prune -exec rm -rf {} +
find . -name "*.pyc" -delete
 
# packaging metadata and C build headers
rm -rf *.dist-info
find . -type d -name "include" -prune -exec rm -rf {} +
find . \( -name "*.pyi" -o -name "*.h" -o -name "*.a" \) -delete
 
du -sm . | cut -f1

data_object The PyArrow question

After stripping, my package was 204 MB. PyArrow was 127 MB of it, so 62% of the surviving package was one dependency.

Drop it and the same package falls to 77 MB.

So the question that actually decides this is not "how do I make pandas smaller." It is: do you need Parquet in this function?

• Reading or writing Parquet, or handing data to Athena or Glue? You need PyArrow. Keep it and plan around 130 MB.
• Reading CSV or JSON and writing to a database? You probably do not. Check your imports before you carry it.

A note on version drift here, because this changes soon. In pandas 2.x, PyArrow is optional. From pandas 3.0 it becomes a required dependency, so pip install pandas alone will pull it. If you are pinned to pandas 2.x today and doing CSV work, that pin is buying you real headroom. Know that before you upgrade casually.

If you are writing Parquet at volume, that is also a signal worth reading. I go through the partitioning and file-format side of this in AWS Athena query optimization, and the honest answer is sometimes that the work belongs in a Glue job instead. More on that below.

inventory_2 The managed layer shortcut

You do not have to build any of this yourself. AWS publishes a managed layer, AWS SDK for pandas (previously awswrangler), that ships pandas, numpy, pyarrow and boto3 already stripped and packaged for Lambda. Add the ARN for your region and runtime, and skip the whole exercise.

The trade is version control. You get the versions in the layer, not the ones you pinned. On a team where requirements.txt is reviewed and locked, that is often a dealbreaker. On a quick pipeline where you just need pandas to exist, it saves you an afternoon.

Use the managed layer when you do not care about exact versions. Build your own when you do.

deployed_code When to stop deleting and use a container

There is a point where stripping bytes stops being optimisation and starts being a warning sign. Here is the line I use.

Move to a container image when any of these are true:

• You are under 250 MB but with less than about 30 MB of headroom. One dependency upgrade puts you back over, and you will be doing this again in three months.
• You need something that is not a Python package at all: a system binary, a font, ffmpeg, a custom shared library.
• Your zipped package is over 50 MB, so you are already going through S3 and have lost the simple upload path anyway.

That last one bit me here. My stripped 204 MB package zipped down to 62 MB. It fits the unzipped limit but not the direct-upload limit, so it has to go via S3 regardless. At that point most of the simplicity argument for zip has already gone.

Container images give you 10 GB of uncompressed image, all layers included, and the old objection to them is out of date. Cold starts for container-packaged functions used to be meaningfully worse. AWS has since done substantial work on container image loading, and for large Python packages the gap has closed or reversed. If you are carrying 200 MB of dependencies, do not rule out containers on cold-start grounds you last checked in 2021. Measure it.

The real cost of containers is workflow, not runtime: a Dockerfile, an ECR repository, and a CI pipeline that builds and pushes images. That is a genuine step up in complexity. It is worth it once you are fighting the limit rather than tidying up.

payments Package size is a billing question now

One thing changed in August 2025 that makes this worth caring about beyond the deploy error.

Lambda now bills the INIT phase for on-demand functions on managed runtimes. Previously the initialisation time, which includes importing your dependencies, was not charged. Now it is. A fat package is slower to import, and that import now appears on the invoice.

For a function invoked a few times an hour, where most invocations are cold, this is not a rounding error. The exercise above is no longer only about getting past a deploy failure. It is the same shape of work as cutting a Glue bill by right-sizing what you actually run.

alt_route When the answer is "this is not a Lambda job"

Worth saying plainly, because the fix is sometimes not a smaller package.

If you are pulling 200 MB of Spark-adjacent tooling into a function to process a file that arrives once a night, you are using the wrong service. Lambda is very good at short, event-driven work. It has a hard 15-minute ceiling and a memory cap, and a heavy dependency stack is usually a sign the job outgrew both.

The rule I apply: if the function processes more than about 1 GB of data, or takes more than a couple of minutes, it belongs in Glue. If it validates a file, transforms a small payload or calls an API, Lambda is right and the dependencies should be light enough to prove it. I use exactly this split on a pipeline that moves millions of records a day: Lambda handles arrival and validation, the heavy transforms run elsewhere.

quiz Frequently asked questions

The takeaway

The 250 MB error looks like a packaging problem and it usually is. Delete the test suites, drop the SDK the runtime already gives you, clear the bytecode, and a typical pandas stack loses about a third of its weight without giving up anything that runs.

But watch which way the numbers point. If one dependency is 62% of your package, the real decision is whether you need that dependency in this function. If you are stripping bytes just to squeeze under the line, you have outgrown the zip format and a container will cost you less over the next year than the same exercise repeated every quarter.

And if the package is heavy because the job is heavy, the honest fix is a different service.

If you are building Python data pipelines on AWS and want a second opinion on where a job should actually run, tell me what you are moving and I will tell you where it belongs. For the layer above this one, building your first ETL pipeline in Python covers the shape of the job itself, and scheduling a Python script covers getting it to run on time.