as_ctypes function for Lambdify

[isuruf]

@richardotis, can you use these ctypes functions? You shouldn't need to access symengine Cython layer and can do it in pure python.

[richardotis]

Can you post a simple example of how to use this? My earlier solution looked similar, but I couldn't figure out how to get the function pointer back in the downstream code.

[isuruf]

Here's how to do this in pure python.

In [1]: import numpy as np                                                                                                                                            

In [2]: from symengine import *                                                                                                                                       

In [3]: var("x y z")                                                                                                                                                  
Out[3]: (x, y, z)

In [4]: l = Lambdify([x, y, z], [x+y+z, x*y*z+1])                                                                                                                     

In [5]: addr1, addr2 = l.as_ctypes()                                                                                                                                  

In [6]: import ctypes                                                                                                                                                 

In [7]: inp = np.array([1,2,3], dtype=np.double)                                                                                                                      

In [8]: out = np.array([0, 0], dtype=np.double)                                                                                                                       

In [9]: addr1(out.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), inp.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), addr2)                                        

In [10]: out                                                                                                                                                          
Out[10]: array([6., 7.])

[isuruf]

To do this in cython,

In [11]: %load_ext cython 

In [12]: %%cython 
    ...:  
    ...: ctypedef void (*math_function_t)(double*, const double*, void* user_data) nogil 
    ...:  
    ...: import ctypes 
    ...:  
    ...: def call_lambdify(double[::1] out, double[::1] inp, l): 
    ...:     addr1, addr2 = l.as_ctypes() 
    ...:     cdef math_function_t cy_f_ptr = (<math_function_t*><size_t>ctypes.addressof(addr1))[0] 
    ...:  
    ...:     cdef void * data =  (<void**><size_t>ctypes.addressof(addr2))[0] 
    ...:     with nogil: 
    ...:         cy_f_ptr(&out[0], &inp[0], data) 
    ...:          
    ...:                                                                                                                                                              
warning: /home/isuru/.cache/ipython/cython/_cython_magic_b3ffdcd6b0cb2c599014b3a4acdd137a.pyx:12:21: Use boundscheck(False) for faster access
warning: /home/isuru/.cache/ipython/cython/_cython_magic_b3ffdcd6b0cb2c599014b3a4acdd137a.pyx:12:30: Use boundscheck(False) for faster access

[richardotis]

@isuruf Thanks, this works great! I'd support merging this and reverting #291 in light of the footguns involving build-time paths to symengine.

Here is the downstream implementation I've put together which uses this:

.pxd

ctypedef void (*math_function_t)(double*, const double*, void* user_data) nogil

cdef class FastFunction:
    cdef readonly object _objref
    cdef math_function_t f_ptr
    cdef void *func_data
    cdef void call(self, double *out, double *inp) nogil

.pyx

cdef class FastFunction:
    def __cinit__(self, object func):
        if func is None:
            self.f_ptr = NULL
            self.func_data = NULL
            return
        # Preserve reference to object to prevent garbage collection
        self._objref = func
        addr1, addr2 = func.as_ctypes()
        self.f_ptr = (<math_function_t*><size_t>ctypes.addressof(addr1))[0]
        self.func_data =  (<void**><size_t>ctypes.addressof(addr2))[0]
    cdef void call(self, double *out, double *inp) nogil:
        if self.f_ptr != NULL:
            self.f_ptr(out, inp, self.func_data)

[richardotis]

Upon re-reading I think we need to keep addr1 and addr2 around as well:

[isuruf]

Upon re-reading I think we need to keep addr1 and addr2 around as well

Are you sure? These are pointers to the function (which is not deleted) and the symengine object (which is kept alive by _objref)

[richardotis]

I am not sure. Some tests with trying to force deletion and collection of addr1 and addr2 seem to work fine, so I may just misunderstand how the pointer extraction code works (which isn't surprising, since I couldn't figure it out earlier on my own)

[richardotis]

I think I get it now. The [0] at the end immediately dereferences the pointer to addr1/addr2 (returned by ctypes.addressof()), so now you just have a pointer to the same memory that those objects pointed to, so they can be safely deleted.

[isuruf]

Yes, see https://stackoverflow.com/a/49635841/4768820 for a detailed answer

[richardotis]

What's needed to merge this? A test?

[isuruf]

Yes, and a revert of last PR

[certik]

That looks good to me.