Entry

How can I make a compiled-in extension function be a method of a Python object without having a python method wrapper? I have read doc/ext and doc/api

Jun 7th, 2001 02:07
Rich Dougherty, Fred Drake, Tom Grydeland,

Python functions implemented in C can be used as methods of Python
objects in a couple of different ways, but they boil down to essentially
the same data structures at the C level.

Python classes (not types) which are constructed at the C level can have
their __dict__ populated with unbound methods which wrap PyCFunction
objects; an example of this is found in the "exceptions" module.  See
the source for this module in the source distribution as
Python/exceptions.c.

For classes written directly in Python, unbound method objects can be
created from arbitrary callable objects, including PyCFunction objects,
using the instancemethod() function in the new module:

    import new
    import c_extension

    class Foo:
        pass

    Foo.meth = new.instancemethod(c_extension.function, None, Foo)

Note that the insertion of the new unbound method must occur after the
close of the "class" statement; the class object itself is required, but
has not been created until after the close of the class declaration. 
Documentation on the "new" module is available at:

    http://www.python.org/doc/current/lib/module-new.html

--------------

This is not an answer, more a clarification.

I have a library I've wrapped in Python, let's call it Spam.  Spam is
fairly well object-oriented, and there is a natural "Spam" object with
its methods taking a "Spam" object as its first argument.

Now, I've defined C interfaces to all of the methods I want using the
common prefix "PySpam", so I can now access the C function (method)
called "Spam_Eggs()" from Python as "PySpam.Eggs()".  This is all well,
except when I want to instantiate Python objects with these wrapped
methods as the methods of the object, I end up writing a whole rewrap of
the interface in Python.  It now looks like this:

import PySpam

class spam:
	def __init__(self):
		self.__py = PySpam.init()


	def eggs(self):
		PySpam.Eggs(self.__py)

	def bacon(self, howfried):
		PySpam.Bacon(self.__py, howfried)

etc.

This seems a bit wasteful to me, and since the wrapped methods have a
PyObject *self argument, it would appear as if it should be possible to
call my wrapped library methods directly as Python methods.

As I tried writing in the question, I have read doc/api and doc/ext,
which has brought me to this point, but I wouldn't mind a tip for the
last bit.

Thanks in advance,

//tom

--------------

I'm going to outline an approach which is slightly different to the
one in Python/exceptions.c [1]. It is based on the example module in
the Python source Modules/xxmodule.c [2] which I suggest you
look. Before I start going to make a few assumptions about your PySpam
example above. I'm guessing that in PySpammodule.c you have:

- PySpam_Type, a PyTypeObject. This is the object that represents the
  "class" PySpam. You want to give objects of this type a few methods.

- Spam_init(), a C function. This is the "constructor" for PySpam. At
  the moment you expose it as "init" so that you must call
  PySpam.init() to create a PySpam.

- Spam_Eggs(), a C function. You want this to be a method of PySpam
  objects.

- Spam_Bacon(), a C function. This is another method of PySpam
  objects.

You want to change all this so that instead of saying

  import PySpam
  s = PySpam.init()
  PySpam.Eggs(s)
  PySpam.Bacon(s)

you can say

  import PySpam
  s = PySpam.PySpam()
  s.Eggs()
  s.Bacon()

To get your "constructor" simply change the symbol you put in the
module dictionary for Spam_init() from "init" to "PySpam". This will
change PySpam.PySpam to refer to your function. And this means that
calling PySpam.PySpam() will create a new PySpam object. Voila! A
constructor.

Now lets see how we turn Spam_Eggs() and Spam_Bacon() into methods of
PySpam. To explore this we'll look at plain old Python for a moment.

What we've got is a class with no methods and two functions. Something
very much like

  class PySpam:
    pass

  def Eggs(self):
    ...

  def Bacon(self):
    ...

Now if we want to be able to say

  p = PySpam()
  p.Eggs()

then we need to make Eggs() and "unbound method" of PySpam. In Python
all you have to do is put a function into a class to create an
"unbound" method.

  PySpam.Eggs = Eggs

Simple. There is now an attribute in PySpam called "Eggs". This means
that you can call

  p.Eggs()

This is the technique that is used to create methods in
Python/exceptions.c [1]. Take a look at make_class() and notice how it
calls populate_methods() to put all the method definitions into the
class' internal dictionary.

The approach in Modules/xxmodule.c [2] is simpler. What happens here
is a play on the fact that

  p.Eggs()

is actually just

  getattr(PySpam, "Eggs")()

So what you want to do is create a C function to implement getattr()
for PySpam. You'll need to join it to PySpam with a pointer in the
PySpam_Type struct.

  static PyMethodDef PySpam_methods[] = {
    {"Eggs", (PyCFunction)Spam_Eggs, METH_VARARGS},
    {"Bacon", (PyCFunction)Spam_Bacon, METH_VARARGS},
    {NULL, NULL}
  };

  static PyObject *
  PySpam_getattr(PySpamObject *self, char *name)
  {
    return Py_FindMethod(PySpam_methods, (PyObject *)self, name);
  }

  statichere PyTypeObject PySpam_Type = {
    /* The ob_type field must be initialized in the module init function
     * to be portable to Windows without using C++. */
    PyObject_HEAD_INIT(NULL)
    0,			          /*ob_size*/
    "PySpam",			  /*tp_name*/
    sizeof(PySpamObject),         /*tp_basicsize*/
    0,			          /*tp_itemsize*/
    /* methods */
    0,                            /*tp_dealloc*/
    0,			          /*tp_print*/
    (getattrfunc)PySpam_getattr,  /*tp_getattr*/
    0,                            /*tp_setattr*/
    0,			          /*tp_compare*/
    0,			          /*tp_repr*/
    0,			          /*tp_as_number*/
    0,			          /*tp_as_sequence*/
    0,		        	  /*tp_as_mapping*/
    0,		        	  /*tp_hash*/
  };

Now any calls to getattr() on PySpam call PySpam_getattr(). They are
then delegated to Py_FindMethod which will look through the
PySpam_methods data structure to return the appropriate method. Voila!
Methods.

The getattr() code in Modeules/xxmodule.c is actually a bit more
complicated than this. The example XxoObject has an internal
dictionary which getattr() looks at before looking in the list of
method definitions. If you're feeling inspired by Python/exceptions.c
you could put the references to your methods in this dictionary
instead.

In conclusion, the key to creating methods for your own types is in
the getattr() method. You can either implement getattr() using
Py_FindMethod() or get references out of an internal dictionary.

Hope this helps.

Rich Dougherty

[1]
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/~checkout~/python/python/dist/src/Python/exceptions.c?rev=1.24&content-type=text/plain
[2]
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/~checkout~/python/python/dist/src/Modules/xxmodule.c?rev=2.22&content-type=text/plain