Why do tuples take less space in memory than lists?

JON

A tuple takes less memory space in Python:

>>> a = (1,2,3)
>>> a.__sizeof__()
48

whereas lists takes more memory space:

>>> b = [1,2,3]
>>> b.__sizeof__()
64

What happens internally on the Python memory management?

MSeifert

I assume you're using CPython and with 64bits (I got the same results on my CPython 2.7 64-bit). There could be differences in other Python implementations or if you have a 32bit Python.

Regardless of the implementation, lists are variable-sized while tuples are fixed-size.

So tuples can store the elements directly inside the struct, lists on the other hand need a layer of indirection (it stores a pointer to the elements). This layer of indirection is a pointer, on 64bit systems that's 64bit, hence 8bytes.

But there's another thing that lists do: They over-allocate. Otherwise list.append would be an O(n) operation always - to make it amortized O(1) (much faster!!!) it over-allocates. But now it has to keep track of the allocated size and the filled size (tuples only need to store one size, because allocated and filled size are always identical). That means each list has to store another "size" which on 64bit systems is a 64bit integer, again 8 bytes.

So lists need at least 16 bytes more memory than tuples. Why did I say "at least"? Because of the over-allocation. Over-allocation means it allocates more space than needed. However, the amount of over-allocation depends on "how" you create the list and the append/deletion history:

>>> l = [1,2,3]
>>> l.__sizeof__()
64
>>> l.append(4)  # triggers re-allocation (with over-allocation), because the original list is full
>>> l.__sizeof__()
96

>>> l = []
>>> l.__sizeof__()
40
>>> l.append(1)  # re-allocation with over-allocation
>>> l.__sizeof__()
72
>>> l.append(2)  # no re-alloc
>>> l.append(3)  # no re-alloc
>>> l.__sizeof__()
72
>>> l.append(4)  # still has room, so no over-allocation needed (yet)
>>> l.__sizeof__()
72

Images

I decided to create some images to accompany the explanation above. Maybe these are helpful

This is how it (schematically) is stored in memory in your example. I highlighted the differences with red (free-hand) cycles:

enter image description here

That's actually just an approximation because int objects are also Python objects and CPython even reuses small integers, so a probably more accurate representation (although not as readable) of the objects in memory would be:

enter image description here

Useful links:

Note that __sizeof__ doesn't really return the "correct" size! It only returns the size of the stored values. However when you use sys.getsizeof the result is different:

>>> import sys
>>> l = [1,2,3]
>>> t = (1, 2, 3)
>>> sys.getsizeof(l)
88
>>> sys.getsizeof(t)
72

There are 24 "extra" bytes. These are real, that's the garbage collector overhead that isn't accounted for in the __sizeof__ method. That's because you're generally not supposed to use magic methods directly - use the functions that know how to handle them, in this case: sys.getsizeof (which actually adds the GC overhead to the value returned from __sizeof__).

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at
0

Comments

0 comments
Login to comment

Related

From Dev

Why do tuples take less space in memory than lists?

From Dev

Why do bools take less memory in an array?

From Dev

Why do tab delimited files take less space than comma separated?

From Dev

Why do tab delimited files take less space than comma separated?

From Dev

OpenGL: Why do square textures take less memory

From Dev

Why do microSD cards have less than advertised space?

From Dev

Why do macros take more space than an equivalently defined function?

From Dev

Why memory usage is less than Xms?

From Dev

Do properties/methods in classes take space in memory?

From Dev

How much memory space do compilers take?

From Dev

Why do some lists of tuples are sorted and not others?

From Dev

Why Memory size is less than the one Specified on Memory Storage Devices?

From Dev

Why the available memory is less than the free memory in free command?

From Dev

Why is the structure showing less memory than actual memory in the following programming

From Dev

Why are tuples slower than lists at evaluating generators into themselves?

From Dev

Why are tuples slower than lists at evaluating generators into themselves?

From Dev

Filter list of tuples to removes lists with all odd number or sum less than 80

From Dev

Why does this list implementation take more space than stl list?

From Dev

Why does a char seem to take more space in an array than by itself

From Dev

Scala - why Double consume less memory than Floats in this case?

From Java

Does the count-min sketch take less space than a typical sparse vector format?

From Dev

Why is Used + Available disk space always less than Total disk space?

From Dev

Why do simple programs take up so much storage space?

From Dev

Why do simple programs take up so much storage space?

From Dev

Why do hard links seem to take the same space as the originals?

From Dev

Why do Grunt/Gulp plugins take up so much space?

From Dev

Why Generators are exhaustive and Lists/Tuples are not?

From Dev

Why do most scripting languages use less memory?

From Dev

Why does an object allocated in boost interprocess shared memory take up more memory than required?

Related Related

  1. 1

    Why do tuples take less space in memory than lists?

  2. 2

    Why do bools take less memory in an array?

  3. 3

    Why do tab delimited files take less space than comma separated?

  4. 4

    Why do tab delimited files take less space than comma separated?

  5. 5

    OpenGL: Why do square textures take less memory

  6. 6

    Why do microSD cards have less than advertised space?

  7. 7

    Why do macros take more space than an equivalently defined function?

  8. 8

    Why memory usage is less than Xms?

  9. 9

    Do properties/methods in classes take space in memory?

  10. 10

    How much memory space do compilers take?

  11. 11

    Why do some lists of tuples are sorted and not others?

  12. 12

    Why Memory size is less than the one Specified on Memory Storage Devices?

  13. 13

    Why the available memory is less than the free memory in free command?

  14. 14

    Why is the structure showing less memory than actual memory in the following programming

  15. 15

    Why are tuples slower than lists at evaluating generators into themselves?

  16. 16

    Why are tuples slower than lists at evaluating generators into themselves?

  17. 17

    Filter list of tuples to removes lists with all odd number or sum less than 80

  18. 18

    Why does this list implementation take more space than stl list?

  19. 19

    Why does a char seem to take more space in an array than by itself

  20. 20

    Scala - why Double consume less memory than Floats in this case?

  21. 21

    Does the count-min sketch take less space than a typical sparse vector format?

  22. 22

    Why is Used + Available disk space always less than Total disk space?

  23. 23

    Why do simple programs take up so much storage space?

  24. 24

    Why do simple programs take up so much storage space?

  25. 25

    Why do hard links seem to take the same space as the originals?

  26. 26

    Why do Grunt/Gulp plugins take up so much space?

  27. 27

    Why Generators are exhaustive and Lists/Tuples are not?

  28. 28

    Why do most scripting languages use less memory?

  29. 29

    Why does an object allocated in boost interprocess shared memory take up more memory than required?

HotTag

Archive