Ctypes: Fast way to convert returned pointer to array or Python list

RTC222

I'm using ctypes to pass an array pointer to a dll and get back a pointer to an array of doubles created using malloc in the dll. When returning to Python, I need a fast way to convert a pointer to an array or Python list.

I can use this list component, but it's slow because there are 320,000 data points:

list_of_results = [ret_ptr[i] for i in range(320000)]

Ideally, I would create the array in Python and pass it to the dll, but I have to use malloc from the dll to create it, since this is a dynamic array, and before that I don't know how many data elements there will be ( Although returning a pointer also returns the number of data elements, so I know how many when I return to Python) - I use realloc to dynamically expand the array size in the dll; I can use realloc with Python arrays, but there is no guarantee that free( ) works fine.

Here is the relevant Python code:

CallTest = hDLL.Main_Entry_fn
CallTest.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.c_int64]
CallTest.restype = ctypes.POINTER(ctypes.c_double)
ret_ptr = CallTest(DataArray, number_of_data_points)
list_of_results = [ret_ptr[i] for i in range(320000)]

So my question is: what is the fastest way to convert the pointer returned by the dll to a Python list or array? The method shown above is too slow.

thank you very much.

user2357112 supports Monica

Slicing a ctypes array or pointer will automatically generate a list:

list_of_results = ret_ptr[:320000]

Depending on what you mean by "convert a pointer to an array" and what kind of output you can use, you might be able to do better. For example, you can back a NumPy array directly by a buffer without copying the data:

buffer_as_ctypes_array = ctypes.cast(ret_ptr, ctypes.POINTER(ctypes.c_double*320000))[0]
buffer_as_numpy_array = numpy.frombuffer(buffer_as_ctypes_array, numpy.float64)

This is of course shocking if you deallocate a buffer while it is still needed.

Related

Faster way to convert ctypes array to python list?

munieq11 I was wondering if anyone could suggest me if there is a better/faster way to read data from my C program that outputs two lists of size n . I use to call C programs.ctypes The loop I show below works by iterating over multiple scans. Each scan produc

Faster way to convert ctypes array to python list?

munieq11 I was wondering if anyone could suggest me if there is a better/faster way to read data from my C program that outputs two lists of size n . I use to call C programs.ctypes The loop I show below works by iterating over multiple scans. Each scan produc

Faster way to convert ctypes array to python list?

munieq11 I was wondering if anyone could suggest me if there is a better/faster way to read data from my C program that outputs two lists of size n . I use to call C programs.ctypes The loop I show below works by iterating over multiple scans. Each scan produc

Faster way to convert ctypes array to python list?

munieq11 I was wondering if anyone could suggest me if there is a better/faster way to read data from my C program that outputs two lists of size n . I use to call C programs.ctypes The loop I show below works by iterating over multiple scans. Each scan produc

Faster way to convert ctypes array to python list?

munieq11 I was wondering if anyone could suggest me if there is a better/faster way to read data from my C program that outputs two lists of size n . I use to call C programs.ctypes The loop I show below works by iterating over multiple scans. Each scan produc

Fast way to convert collection to array or list?

jack For each *Collection( HtmlNodeCollection, TreeNodeCollection, CookieCollectionetc.) class instance I need to pass to a method that accepts only an array or list (should there not be an accepting method TreeNodeCollectionin eg?) I have to write an extensio

Fast way to convert collection to array or list?

Jack For each *Collection( HtmlNodeCollection, TreeNodeCollection, CookieCollectionetc.) class instance I need to pass to a method that accepts only an array or list (should there not be an accepting method TreeNodeCollectionin eg?) I have to write an extensio

Fast way to convert collection to array or list?

jack For each *Collection( HtmlNodeCollection, TreeNodeCollection, CookieCollectionetc.) class instance I need to pass to a method that accepts only an array or list (should there not be an accepting method TreeNodeCollectionin eg?) I have to write an extensio

Fast way to convert collection to array or list?

Jack For each *Collection( HtmlNodeCollection, TreeNodeCollection, CookieCollectionetc.) class instance I need to pass to a method that accepts only an array or list (should there not be an accepting method TreeNodeCollectionin eg?) I have to write an extensio

Fast way to convert collection to array or list?

jack For each *Collection( HtmlNodeCollection, TreeNodeCollection, CookieCollectionetc.) class instance I need to pass to a method that accepts only an array or list (should there not be an accepting method TreeNodeCollectionin eg?) I have to write an extensio

Python Numpy array Ctypes pointer

Ventura Franklin I have a GetImageData function in a DLL: int GetImageData ( const char * name , int ID , const char * strLibPath , int Lvl , int rbeg , int rend , int cbeg , int cend , int ZLayer , unsigned char * ImgBuffer ); In Python, I import this DLL a

How to convert a Python list to a C array using ctypes?

Especially no one: If I have the following two sets of code, how can I glue them together? void c_function(void *ptr) { int i; for (i = 0; i < 10; i++) { printf("%p", ptr[i]); } return; } def python_routine(y): x = [] for e

How to convert a Python list to a C array using ctypes?

Especially no one: If I have the following two sets of code, how can I glue them together? void c_function(void *ptr) { int i; for (i = 0; i < 10; i++) { printf("%p", ptr[i]); } return; } def python_routine(y): x = [] for e