Split a list into chunks
Splits any list into fixed-size chunks using a slicing generator, so the last chunk can be shorter without any special-case code.
Last updated
def chunk_list(items, size):
for i in range(0, len(items), size):
yield items[i:i + size]
# list(chunk_list([1, 2, 3, 4, 5], 2)) -> [[1, 2], [3, 4], [5]]How it works
range(0, len(items), size) walks the start index of each chunk in steps of size (0, 2, 4, ... for size=2), and Python slicing (items[i:i+size]) is naturally forgiving about running past the end of the list — it just returns however many elements are left, which is exactly why the last chunk comes out shorter with no extra logic.
It's written as a generator (yield, not return a list of lists) so chunking a very large list doesn't build every chunk in memory upfront — chunks are produced one at a time as the caller iterates, and list(...) at the call site is what materializes them when you actually want the full list.
Edge cases to know
- →A size of 0 raises ValueError: range() arg 3 must not be zero. A negative size is the quieter problem — it yields nothing at all, an empty result with no error — so validate size > 0 if it isn't a hard-coded constant.
- →The input needs to support slicing (lists, tuples, strings); a plain iterator or generator passed as items won't work since items[i:i+size] requires random access.
- →This only handles one level — chunking, not flattening. It works on any sliceable sequence type, not just lists, despite the name.
Part by part
- range(0, len(items), size)
- A stepped range producing the start index of each chunk — 0, size, 2*size and so on — which is what makes the slicing arithmetic trivial.
- yield
- Makes this a generator, so chunks are produced on demand rather than all at once. That matters when the input is large.
- items[i:i + size]
- Python slicing clamps to the end of the sequence, so the final chunk is simply shorter rather than needing a special case.
Common mistakes
for i in range(0, len(items), size): yield items[i:size]The end of the slice must be i + size, not size. As written, every chunk after the first is empty, because the start index has already passed the fixed endpoint.
yield items[i:i + size]chunk_list(items, 0)A step of zero makes range raise ValueError. The function does not validate its input, so this surfaces as an error from deep inside rather than at the call.
Calling it and expecting a listIt is a generator, so it returns lazily and printing it shows a generator object rather than the chunks. Wrap it in list() when you want them all at once, as the example does.
Questions
Is there a standard library equivalent?
itertools.batched, added in Python 3.12, does exactly this and returns tuples. Before that version there was no built-in, which is why this pattern is written out so often.
Does it work on any sequence?
On anything supporting slicing — lists, tuples, strings. It does not work on a generator or a set, because those cannot be indexed; itertools.islice is the tool for those.
Why is the last chunk shorter?
Because the input rarely divides evenly, and Python's slicing clamps rather than erroring. If every chunk must be the same size, pad the last one or drop it deliberately.
What is a good chunk size for batch API calls?
Whatever the API documents as its maximum, minus a little headroom. Larger batches mean fewer round trips, and a batch that fails takes more work with it — so match the limit rather than guessing.