ℹ️ Skipped - page is already crawled
| Filter | Status | Condition | Details |
|---|---|---|---|
| HTTP status | PASS | download_http_code = 200 | HTTP 200 |
| Age cutoff | PASS | download_stamp > now() - 6 MONTH | 1.5 months ago |
| History drop | PASS | isNull(history_drop_reason) | No drop reason |
| Spam/ban | PASS | fh_dont_index != 1 AND ml_spam_score = 0 | ml_spam_score=0 |
| Canonical | PASS | meta_canonical IS NULL OR = '' OR = src_unparsed | Not set |
| Property | Value |
|---|---|
| URL | https://labs.ub.uni-frankfurt.de/post/asyncio-and-multiprocessing-1/ |
| Last Crawled | 2026-02-28 00:28:07 (1 month ago) |
| First Indexed | 2023-07-06 09:42:27 (2 years ago) |
| HTTP Status Code | 200 |
| Meta Title | CPU-intensive Python Web Backends with asyncio and multiprocessing, Part I - UBLabs |
| Meta Description | Python, popular though it is, has a few well-known weaknesses. One of the most well known among serious users of the language is the lack of multicore support for in-process threads. This is because CPython, Python’s standard implementation has a global interpreter lock (often referred to as the GIL). The GIL locks each instance of the interpreter to a single core—a common approach to avoid race conditions in the implementation of language interpreters. |
| Meta Canonical | null |
| Boilerpipe Text | Python, popular though it is, has a few well-known weaknesses. One of
the most well known among serious users of the language is the lack of
multicore support for in-process threads. This is because CPython,
Python’s standard implementation has a global interpreter lock (often
referred to as the GIL ). The GIL locks
each instance of the interpreter to a single core—a common approach
to avoid race conditions in the implementation of language
interpreters. This series of posts will discuss the approach I used for parallel
execution in the implementation of the Revrit
API . The Revrit API is a tool for
reconstructing original Hebrew script from transcribed metadata—but
the approach is quite CPU intensive! The difficulty is coming up with
an implementation that can utilize multiple cores effectively while
still maintaining a responsive web server. This first post will provide an introduction to the multiprocessing
module, especially using Pool instances, as well as the challenges
using it with common Python WSGI web frameworks and looking at a way
forward with asynchronous HTTP servers. Then next post will cover asynchronous programming in Python with
async and await , and the final post will show how to combine these
things. Python does have a threading
module , which
offers a form of concurrency. However, Python threads are deceptive to
new users coming from a language like Java. Because all threads run on
a single CPU core in each instance of the Python interpreter, they do
not execute in parallel. Threads in Python are mostly useful for a
small number of workers which are I/O bound. CPU intensive tasks will
not benefit from this approach. If one really wants to benefit from multiple cores in Python, the only
way to do it is with OS processes. OS processes have some pros and
cons because they (normally) do not share memory. On the plus side, it is impossible to create a data race without
shared memory. It’s a much safer, saner way to program, which is why
Erlang
borrowed this paradigm for its own abstraction on concurrency. On the minus side, communication and data transfer has to be
organized in another way by the programmer—normally through pipes
or sockets. This requires defining binary or text serializations for
any objects you wish to pass between processes. Luckily, Python mostly mitigates the downside of having to implement
serialization and transports with the multiprocessing
module in the
standard library. This module defines types and functions for
transporting serialized Python objects between processes (and even
across the internet, if you need). It defines types for processes,
queues, pipes, locks, shared memory and other goodies, but my
preferred way to work with processes is as a worker pool. Using
multiprocessing.Pool ,
you create a pool of worker processes. You can send work to the pool,
which will distribute it to any available worker, and you can come
back to the result later. 1
2
3
4
5
6
7
8
# With no arguments, Pool will create as # many workers as your system has cores. pool = multiprocessing . Pool () result = pool . apply_async ( func = str . upper , args = [ "spam" ]) print ( result . get ()) # SPAM In this example, we use Pool.apply_async to create an
AsyncResult
object which represents work being done in a worker process. When we
call its .get method, the main process will block until the result
is available and return its value. This abstraction isn’t perfect. Not every type of Python object can be
serialized automatically. Functions and classes, for example, are not
serialized, but are looked up in the worker process. This is normally
fine, since the memory will have been copied from the parent when the
pool was created. However, it can be a problem if you are trying to
use functions or classes defined after the pool was created. In
practice, the main issue with this is that you can’t use closures to
create jobs in the pool. Likewise, while most pure Python objects can be serialized
automatically (with some small exceptions), instances of types defined
in C as part of native extension modules must explicitly define their
serialized data representation, which not all extension modules do. Of course, serialization and transport of objects also has a cost. It
will not be as efficient as the shared memory situation one has with
threads in a language with better multicore support. Despite these
limitations, the multiprocessing module greatly reduces the burden
of traditional multiprocess programming with its helpful, albeit
imperfect abstractions for interprocesses communication. We’ve seen how to use the pool to create jobs one at a time, but one
of the best features of the process pool is the way it can abstract
over iteration. 1
2
3
4
5
6
7
8
9
10
11
12
import multiprocessing import os import random import time def report ( n ): time . sleep ( random . random () / 2 ) return f "job { n } in process { os . getpid () } " pool = multiprocessing . Pool () for string in pool . imap ( report , range ( 1 , 11 )): print ( string ) 1
2
3
4
5
6
7
8
9
10
job 1 in process 216701
job 2 in process 216702
job 3 in process 216703
job 4 in process 216704
job 5 in process 216705
job 6 in process 216706
job 7 in process 216707
job 8 in process 216708
job 9 in process 216702
job 10 in process 216708
The .map and .imap methods will efficiently distribute the work to the
pool. If the order doesn’t matter, we can also use .imap_unordered
to add a little more efficiency: 1
2
for string in pool . imap_unordered ( report , range ( 1 , 11 )): print ( string ) 1
2
3
4
5
6
7
8
9
10
job 3 in process 216946
job 8 in process 216951
job 7 in process 216950
job 4 in process 216947
job 6 in process 216949
job 2 in process 216945
job 9 in process 216946
job 5 in process 216948
job 1 in process 216944
job 10 in process 216951
These are my favorite abstractions for parallel execution in Python. This strategy works well for the CPU-intensive transliteration
conversion process on my local machine, but how will it work on a web
server? It depends, but if you’re using the most popular Python web frameworks
like Django, Flask or Pyramid, you run into some issues. These
frameworks are all implemented in terms of WSGI, a common protocol
used for Python backends to communicate with web servers. However,
WSGI does not allow the framework to spawn new processes—only the
http server is allowed to do that. One possible solution would be to have the web server communicate with
another network service over sockets or another communication system
like ZeroMQ , but at this point we’d just be
re-implementing what the multiprocessing library gives us for free
in terms of communication and serialization. If you want to save yourself the bother, what you need is a Python web
server that does not rely on WSGI. There are a few well-known options:
Twisted ,
Tornado , and
AIOHTTP . (There may be more
options now. The technical decision was made some time ago.) The thing all of these webservers have in common is use an
event-loop-based architecture to do I/O operations
asynchronously. This makes it possible to get maximum throughput on a
single thread in a way similar to how nginx
or Node.js work. Event-loop-based web servers were created as large companies needed to be
able to serve tens of thousands of connections per second, rather than
the hundreds which process and thread-based architectures of older
systems could accommodate. Our use case does not require this kind of density, but it does
require something that will allow us to perform CPU-intensive work
while still allowing the web server to be responsive. In the end, we
went with Tornado because it provides a relatively simple routing
system, with great performance. It is often benchmarked as the fastest
of all Python HTTP servers, along with a lot of extra industry-tested
features. Continue reading this series for an overview of how asynchronous
programming works in Python. The third and final post covers combined
asynicio with multiprocessing to create a web service that can keep
taking requests even when the CPU is at full capacity. |
| Markdown | [UBLabs](https://labs.ub.uni-frankfurt.de/)
- [Home](https://labs.ub.uni-frankfurt.de/)
- [Archives](https://labs.ub.uni-frankfurt.de/post/)
- [Tags](https://labs.ub.uni-frankfurt.de/tags/)
- [Projects](https://labs.ub.uni-frankfurt.de/projects/)
- [About](https://labs.ub.uni-frankfurt.de/about/)
[](http://www.uni-frankfurt.de/)
# [Universitätsbibliothek J.C. Senckenberg](http://www.ub.uni-frankfurt.de/) Labs
- [Home](https://labs.ub.uni-frankfurt.de/)
- [Archives](https://labs.ub.uni-frankfurt.de/post/)
- [Tags](https://labs.ub.uni-frankfurt.de/tags/)
- [Projects](https://labs.ub.uni-frankfurt.de/projects/)
- [About](https://labs.ub.uni-frankfurt.de/about/)
# CPU-intensive Python Web Backends with asyncio and multiprocessing, Part I
2021-09-15
[](https://labs.ub.uni-frankfurt.de/authors/jud/)
1351 words 7 mins read
Written by [Aaron Christiansson](https://labs.ub.uni-frankfurt.de/authors/ac/)
## Contents
- [Parallel execution with processes pools](https://labs.ub.uni-frankfurt.de/post/asyncio-and-multiprocessing-1/#parallel-execution-with-processes-pools)
- [Multiprocessing on a Python web backend?](https://labs.ub.uni-frankfurt.de/post/asyncio-and-multiprocessing-1/#multiprocessing-on-a-python-web-backend)
Python, popular though it is, has a few well-known weaknesses. One of the most well known among serious users of the language is the lack of multicore support for in-process threads. This is because CPython, Python’s standard implementation has a global interpreter lock (often referred to as the [GIL](https://wiki.python.org/moin/GlobalInterpreterLock)). The GIL locks each instance of the interpreter to a single core—a common approach to avoid race conditions in the implementation of language interpreters.
This series of posts will discuss the approach I used for parallel execution in the implementation of the [Revrit API](https://github.com/ubffm/arcapi). The Revrit API is a tool for reconstructing original Hebrew script from transcribed metadata—but the approach is quite CPU intensive! The difficulty is coming up with an implementation that can utilize multiple cores effectively while still maintaining a responsive web server.
This first post will provide an introduction to the multiprocessing module, especially using `Pool` instances, as well as the challenges using it with common Python WSGI web frameworks and looking at a way forward with asynchronous HTTP servers.
Then next post will cover asynchronous programming in Python with `async` and `await`, and the final post will show how to combine these things.
# Parallel execution with processes pools
Python does have a [threading module](https://docs.python.org/3/library/threading.html), which offers a form of concurrency. However, Python threads are deceptive to new users coming from a language like Java. Because all threads run on a single CPU core in each instance of the Python interpreter, they do not execute in parallel. Threads in Python are mostly useful for a small number of workers which are I/O bound. CPU intensive tasks will not benefit from this approach.
If one really wants to benefit from multiple cores in Python, the only way to do it is with OS processes. OS processes have some pros and cons because they (normally) do not share memory.
- On the plus side, it is impossible to create a data race without shared memory. It’s a much safer, saner way to program, which is why [Erlang](https://en.wikipedia.org/wiki/Erlang_\(programming_language\)) borrowed this paradigm for its own abstraction on concurrency.
- On the minus side, communication and data transfer has to be organized in another way by the programmer—normally through pipes or sockets. This requires defining binary or text serializations for any objects you wish to pass between processes.
Luckily, Python *mostly* mitigates the downside of having to implement serialization and transports with the [multiprocessing module](https://docs.python.org/3/library/multiprocessing.html) in the standard library. This module defines types and functions for transporting serialized Python objects between processes (and even across the internet, if you need). It defines types for processes, queues, pipes, locks, shared memory and other goodies, but my preferred way to work with processes is as a worker pool. Using [multiprocessing.Pool](https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool), you create a pool of worker processes. You can send work to the pool, which will distribute it to any available worker, and you can come back to the result later.
In this example, we use `Pool.apply_async` to create an [AsyncResult](https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.AsyncResult) object which represents work being done in a worker process. When we call its `.get` method, the main process will block until the result is available and return its value.
This abstraction isn’t perfect. Not every type of Python object can be serialized automatically. Functions and classes, for example, are not serialized, but are looked up in the worker process. This is normally fine, since the memory will have been copied from the parent when the pool was created. However, it can be a problem if you are trying to use functions or classes defined after the pool was created. In practice, the main issue with this is that you can’t use closures to create jobs in the pool.
Likewise, while most pure Python objects can be serialized automatically (with some small exceptions), instances of types defined in C as part of native extension modules must explicitly define their serialized data representation, which not all extension modules do.
Of course, serialization and transport of objects also has a cost. It will not be as efficient as the shared memory situation one has with threads in a language with better multicore support. Despite these limitations, the `multiprocessing` module greatly reduces the burden of traditional multiprocess programming with its helpful, albeit imperfect abstractions for interprocesses communication.
We’ve seen how to use the pool to create jobs one at a time, but one of the best features of the process pool is the way it can abstract over iteration.
The `.map` and `.imap` methods will efficiently distribute the work to the pool. If the order doesn’t matter, we can also use `.imap_unordered` to add a little more efficiency:
These are my favorite abstractions for parallel execution in Python.
# Multiprocessing on a Python web backend?
This strategy works well for the CPU-intensive transliteration conversion process on my local machine, but how will it work on a web server?
It depends, but if you’re using the most popular Python web frameworks like Django, Flask or Pyramid, you run into some issues. These frameworks are all implemented in terms of WSGI, a common protocol used for Python backends to communicate with web servers. However, WSGI does not allow the framework to spawn new processes—only the http server is allowed to do that.
One possible solution would be to have the web server communicate with another network service over sockets or another communication system like [ZeroMQ](https://zeromq.org/), but at this point we’d just be re-implementing what the `multiprocessing` library gives us for free in terms of communication and serialization.
If you want to save yourself the bother, what you need is a Python web server that does not rely on WSGI. There are a few well-known options: [Twisted](https://twistedmatrix.com/trac/), [Tornado](https://www.tornadoweb.org/en/stable/), and [AIOHTTP](https://docs.aiohttp.org/en/stable/). (There may be more options now. The technical decision was made some time ago.)
The thing all of these webservers have in common is use an event-loop-based architecture to do I/O operations asynchronously. This makes it possible to get maximum throughput on a single thread in a way similar to how [nginx](https://nginx.org/en/) or [Node.js](https://nodejs.org/) work.
Event-loop-based web servers were created as large companies needed to be able to serve tens of thousands of connections per second, rather than the hundreds which process and thread-based architectures of older systems could accommodate.
Our use case does not require this kind of density, but it does require something that will allow us to perform CPU-intensive work while still allowing the web server to be responsive. In the end, we went with Tornado because it provides a relatively simple routing system, with great performance. It is often benchmarked as the fastest of all Python HTTP servers, along with a lot of extra industry-tested features.
Continue reading this series for an overview of how asynchronous programming works in Python. The third and final post covers combined asynicio with multiprocessing to create a web service that can keep taking requests even when the CPU is at full capacity.
Last Modified on 2021-09-15.
[python](https://labs.ub.uni-frankfurt.de/tags/python/) [asyncio](https://labs.ub.uni-frankfurt.de/tags/asyncio/) [multiprocessing](https://labs.ub.uni-frankfurt.de/tags/multiprocessing/)
[CPU-intensive Python Web Backends with asyncio and multiprocessing, Part II Prev](https://labs.ub.uni-frankfurt.de/post/asyncio-and-multiprocessing-2/) [Getting the (Semantic) Sense out of a User Query Next](https://labs.ub.uni-frankfurt.de/post/getting-the-semantic-sense-out-of-a-user-query/)
© 2025 Universitätsbibliothek Frankfurt am Main \| [Privacy policy](http://www.ub.uni-frankfurt.de/benutzung/datenschutz.html) \| [Imprint](http://www2.uni-frankfurt.de/impressum)
All content is published under [CC-BY 4.0](https://creativecommons.org/licenses/by/4.0/) \| Powered by [Hugo](https://gohugo.io/) \| Theme Based on: [Even](https://github.com/olOwOlo/hugo-theme-even) |
| Readable Markdown | null |
| Shard | 65 (laksa) |
| Root Hash | 16531431506572871865 |
| Unparsed URL | de,uni-frankfurt!ub,labs,/post/asyncio-and-multiprocessing-1/ s443 |