cpython: a3f27289ec68 (original) (raw)

Mercurial > cpython

changeset 79746:a3f27289ec68 3.3

Tweak the threaded example in concurrent.futures

Nick Coghlan ncoghlan@gmail.com
date	Tue, 16 Oct 2012 22:50:04 +1000
parents	24d52d3060e8
children	f2a739d6fc86 903c276b814a
files	Doc/library/concurrent.futures.rst Misc/NEWS
diffstat	2 files changed, 19 insertions(+), 11 deletions(-)[+] [-] Doc/library/concurrent.futures.rst 27 Misc/NEWS 3

line wrap: on

line diff

--- a/Doc/library/concurrent.futures.rst +++ b/Doc/library/concurrent.futures.rst @@ -136,20 +136,25 @@ ThreadPoolExecutor Example 'http://www.bbc.co.uk/',[](#l1.4) 'http://some-made-up-domain.com/'][](#l1.5)

Retrieve a single page and report the url and contents
def load_url(url, timeout):

  return urllib.request.urlopen(url, timeout=timeout).read()[](#l1.9)

with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:

  future_to_url = dict((executor.submit(load_url, url, 60), url)[](#l1.12)

                       for url in URLS)[](#l1.13)

  conn = urllib.request.urlopen(url, timeout=timeout)[](#l1.14)

```
  return conn.readall()[](#l1.15)
```

  for future in concurrent.futures.as_completed(future_to_url):[](#l1.17)

      url = future_to_url[future][](#l1.18)

      if future.exception() is not None:[](#l1.19)

          print('%r generated an exception: %s' % (url,[](#l1.20)

                                                   future.exception()))[](#l1.21)

We can use a with statement to ensure threads are cleaned up promptly
with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:

  # Start the load operations and mark each future with its URL[](#l1.24)

  load_urls = [executor.submit(load_url, url, 60) for url in URLS][](#l1.25)

  for future, url in zip(load_urls, URLS):[](#l1.26)

```
      future.url = url[](#l1.27)
```

  for future in concurrent.futures.as_completed(load_urls):[](#l1.28)

```
      url = future.url[](#l1.29)
```
```
      try:[](#l1.30)
```

          data = future.result()[](#l1.31)

      except Exception as exc:[](#l1.32)

          print('%r generated an exception: %s' % (url, exc))[](#l1.33)
      else:[](#l1.34)

          print('%r page is %d bytes' % (url, len(future.result())))[](#l1.35)

          print('%r page is %d bytes' % (url, len(data)))[](#l1.36)

ProcessPoolExecutor

--- a/Misc/NEWS +++ b/Misc/NEWS @@ -143,6 +143,9 @@ Build Documentation ------------- +- Additional comments and some style changes in the concurrent.futures URL

retrieval example +

Issue #16115: Improve subprocess.Popen() documentation around args, shell, and executable arguments.

cpython: a3f27289ec68 (original) (raw)

Mercurial > cpython

changeset 79746:a3f27289ec68 3.3

Retrieve a single page and report the url and contents

We can use a with statement to ensure threads are cleaned up promptly