How To Get Python's Multiprocessing Array'pointer And Pass It To Cpp Program?

I now need to request arrays in python, and pass them into a Cpp program, yet python still needs to process them. But I found that when I use multiprocessing, the array's adderess

Solution 1:

Do not create the RawArray outside of the "run once" main code or you are making different arrays. Create the RawArray once in the main process, and pass that RawArray as a parameter to the target function of the new processes. The virtual address each process "sees" will be different, but the physical memory will be the same.

Here's an example:

test.cpp:

This will display the pointer address and then change the specified index in the shared array.

#include<iostream>#include<cstdint>usingnamespace std;

#define API __declspec(dllexport)extern"C"API
voidset(double* data, int index, double value){
    cout << data << ' ' << index << ' ' << value << endl;
    data[index] = value;
}

test.py:

This passes the shared array to each process. The main process will also change an element. The lock is used because RawArray is not synchronized and the printing in the C++ code will mess up otherwise, so this code won't really run in parallel but it does illustrate that the processes get different virtual addresses but share the same data.

import multiprocessing as mp
from multiprocessing.sharedctypes import RawArray
from ctypes import *

dll = CDLL('./test')
dll.set.argtypes = POINTER(c_double),c_int,c_double
dll.set.restype = Nonedefcall(lock,data,index,value):
    with lock:
        dll.set(data,index,value)

if __name__ == '__main__':

    # This code runs once in the main process.# The lock and shared data are created once only and passed to other processes.

    lock = mp.Lock()
    data = RawArray(c_double, 3)
    data[0] = 0.5
    p1 = mp.Process(target=call, args=(lock,data,1,1.25))
    p2 = mp.Process(target=call, args=(lock,data,2,2.5))
    p1.start()
    p2.start()
    p1.join()
    p2.join()
    print(list(data))

Output (different addresses, same shared data):

00000269D66E0000 1 1.25
00000187F0B90000 2 2.5
[0.5, 1.25, 2.5]