[Python-Dev] Do any of the 4 multiprocessing methods work using shared memory? (original) (raw)
Kevin Ar18 kevinar18 at hotmail.com
Tue Jul 27 02:20:35 CEST 2010
- Previous message: [Python-Dev] Readability of hex strings (Was: Use of coding cookie in 3.x stdlib)
- Next message: [Python-Dev] Do any of the 4 multiprocessing methods work using shared memory?
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Brief Summary: Can I share Python objects between multiple processes using shared memory (using the multiprocessing module)? In particular, do Queues or Pipes work using shared memory?
Details:
- I have several processes each on a separate CPU core (so they run in parallel).
- I want to share certain Python objects between the processes using shared memory (I can handle the locking myself -- only one process can access a variable or object at a time, so there is no conflict anyways).
- I strongly prefer that it be shared memory because I do not want to incur the costs of having to copy data back and forth between the processes.
I am aware of the multiprocessing module. It offers: Queues Pipes Shared Memory Map Server Process
So, let me ask a few questions about those 4 items:
Queues & Pipes These handle Python objects and variables.... However, my question is do they use shared memory or do they require copying the data or some other method that is more costly than shared memory?
Shared Memory Map Does not support Python objects or variables -- thus no good to me.
Server Process Looks like it requires another process that incurs extra processing just to handle the data exchange -- which, if true, is bad for me. Does this use shared memory? or does it require copying data back and forth between processes?
The New Busy is not the too busy. Combine all your e-mail accounts with Hotmail. http://www.windowslive.com/campaign/thenewbusy?tile=multiaccount&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_4
- Previous message: [Python-Dev] Readability of hex strings (Was: Use of coding cookie in 3.x stdlib)
- Next message: [Python-Dev] Do any of the 4 multiprocessing methods work using shared memory?
- Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]