Why is coreutils sort slower than Python?

debugcn Published at Dev

augurar

I wrote the following script to test the speed of Python's sort functionality:

from sys import stdin, stdout
lines = list(stdin)
lines.sort()
stdout.writelines(lines)

I then compared this to the coreutils sort command on a file containing 10 million lines:

$ time python sort.py <numbers.txt >s1.txt
real    0m16.707s
user    0m16.288s
sys     0m0.420s

$ time sort <numbers.txt >s2.txt 
real    0m45.141s
user    2m28.304s
sys     0m0.380s

The built-in command used all four CPUs (Python only used one) but took about 3 times as long to run! What gives?

I am using Ubuntu 12.04.5 (32-bit), Python 2.7.3, and sort 8.13

augurar

Izkata's comment revealed the answer: locale-specific comparisons. The sort command uses the locale indicated by the environment, whereas Python defaults to a byte order comparison. Comparing UTF-8 strings is harder than comparing byte strings.

$ time (LC_ALL=C sort <numbers.txt >s2.txt)
real    0m5.485s
user    0m14.028s
sys     0m0.404s

How about that.

Collected from the Internet

Please contact [email protected] to delete if infringement.

edited at2021-06-20

Comments

0 comments

From Java

Why is max slower than sort?

From Java

Why is Collections.sort() much slower than Arrays.sort()?

From Dev

Why is my merge sort code slower than insertion sort

From Dev

Why is Collections.sort() much slower than Arrays.sort()?

From Dev

Why my selection sort is slower than insertion sort

From Dev

Why is nested `if` in Python much slower than parallel `and`?

From Dev

Python: Why is list comprehension slower than for loop

From Dev

why readline() is much slower than readlines() in Python?

From Dev

lambda is slower than function call in python, why

From Dev

Why is this Rust slower than my similar Python?

From Java

Why is heap slower than sort for K Closest Points to Origin?

From Dev

Why is std::shuffle as slow (or even slower than) std::sort?

From Dev

Why is heap slower than sort for K Closest Points to Origin?

From Dev

Why is "or" slower than "and" in Java?

From Dev

QuickSort is slower than std::sort

From Dev

Why is Python 3 is considerably slower than Python 2?

From Dev

Parallel sort slower than serial sort

From Dev

Python: Why is threaded function slower than non thread

From Dev

Why is using a Python generator much slower to traverse binary tree than not?

From Dev

Why is this Haskell program so much slower than an equivalent Python one?

From Dev

Python | Why is accessing instance attribute slower than local?

From Dev

Why my bisection search is slower than linear search in python?

From Dev

Why mesh python code slower than decomposed one?

From Dev

Why is this implementation of binary heap slower than that of Python's stdlib?

From Dev

Why python multiprocess pool is slower than one process only?

From Dev

Why is calling float() on a number slower than adding 0.0 in Python?

From Dev

Why is this generator pipeline slower than a traditional loop in Python?

From Dev

Why python broadcasting in the example below is slower than a simple loop?

From Dev

Why is NumPy sometimes slower than NumPy + plain Python loop?

Related Related

Article