Website hosting and domain hosting service by Active-Venture.com
  

 Back to Index

18 Pymalloc: A Specialized Object Allocator

Pymalloc, a specialized object allocator written by Vladimir Marangozov, was a feature added to Python 2.1. Pymalloc is intended to be faster than the system malloc() and to have less memory overhead for allocation patterns typical of Python programs. The allocator uses C's malloc() function to get large pools of memory and then fulfills smaller memory requests from these pools.

In 2.1 and 2.2, pymalloc was an experimental feature and wasn't enabled by default; you had to explicitly enable it when compiling Python by providing the --with-pymalloc option to the configure script. In 2.3, pymalloc has had further enhancements and is now enabled by default; you'll have to supply --without-pymalloc to disable it.

This change is transparent to code written in Python; however, pymalloc may expose bugs in C extensions. Authors of C extension modules should test their code with pymalloc enabled, because some incorrect code may cause core dumps at runtime.

There's one particularly common error that causes problems. There are a number of memory allocation functions in Python's C API that have previously just been aliases for the C library's malloc() and free(), meaning that if you accidentally called mismatched functions the error wouldn't be noticeable. When the object allocator is enabled, these functions aren't aliases of malloc() and free() any more, and calling the wrong function to free memory may get you a core dump. For example, if memory was allocated using PyObject_Malloc(), it has to be freed using PyObject_Free(), not free(). A few modules included with Python fell afoul of this and had to be fixed; doubtless there are more third-party modules that will have the same problem.

As part of this change, the confusing multiple interfaces for allocating memory have been consolidated down into two API families. Memory allocated with one family must not be manipulated with functions from the other family. There is one family for allocating chunks of memory and another family of functions specifically for allocating Python objects.

 

  • To allocate and free an undistinguished chunk of memory use the ``raw memory'' family: PyMem_Malloc(), PyMem_Realloc(), and PyMem_Free().

     

  • The ``object memory'' family is the interface to the pymalloc facility described above and is biased towards a large number of ``small'' allocations: PyObject_Malloc, PyObject_Realloc, and PyObject_Free.

     

  • To allocate and free Python objects, use the ``object'' family PyObject_New(), PyObject_NewVar(), and PyObject_Del().

Thanks to lots of work by Tim Peters, pymalloc in 2.3 also provides debugging features to catch memory overwrites and doubled frees in both extension modules and in the interpreter itself. To enable this support, compile a debugging version of the Python interpreter by running configure with --with-pydebug.

To aid extension writers, a header file Misc/pymemcompat.h is distributed with the source to Python 2.3 that allows Python extensions to use the 2.3 interfaces to memory allocation while compiling against any version of Python since 1.5.2. You would copy the file from Python's source distribution and bundle it with the source of your extension.

 

  

 

2002-2004 Active-Venture.com Website Hosting Service

 

Disclaimer: This documentation is provided only for the benefits of our hosting customers.
For authoritative source of the documentation, please refer to http://python.org/doc/

 

Register domain name by Cheap Domain Registrar with free domain hosting services

 
Cheap domain name - Cheap domain name registration service including free domain hosting services   Active-Domain.com offers domain name registration, domain name transfer and domain search services