storm/w3m
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update to w3m-0.2.1-inu-1.6.

Browse Source
This commit is contained in:
Akinori Ito
2001-11-15 00:32:13 +00:00
parent 6c63633545
commit 85da7ee692
82 changed files with 10900 additions and 217 deletions
Download Patch File Download Diff File Expand all files Collapse all files
gc/alpha_mach_dep.s
+1 -1
View File
@@ -1,4 +1,4 @@
# $Id: alpha_mach_dep.s,v 1.2 2001/11/09 04:59:18 a-ito Exp $
# $Id: alpha_mach_dep.s,v 1.3 2001/11/15 00:32:13 a-ito Exp $
.arch ev6
.text
gc/config.guess
Vendored
+1121
View File
File diff suppressed because it is too large Load Diff
gc/config.sub
Vendored
+1232
View File
File diff suppressed because it is too large Load Diff
gc/doc/README
+617
View File
@@ -0,0 +1,617 @@
Copyright (c) 1988, 1989 Hans-J. Boehm, Alan J. Demers
Copyright (c) 1991-1996 by Xerox Corporation. All rights reserved.
Copyright (c) 1996-1999 by Silicon Graphics. All rights reserved.
Copyright (c) 1999-2001 by Hewlett-Packard Company. All rights reserved.
The file linux_threads.c is also
Copyright (c) 1998 by Fergus Henderson. All rights reserved.
The files Makefile.am, and configure.in are
Copyright (c) 2001 by Red Hat Inc. All rights reserved.
The files config.guess and a few others are copyrighted by the Free
Software Foundation.
THIS MATERIAL IS PROVIDED AS IS, WITH ABSOLUTELY NO WARRANTY EXPRESSED
OR IMPLIED. ANY USE IS AT YOUR OWN RISK.
Permission is hereby granted to use or copy this program
for any purpose, provided the above notices are retained on all copies.
Permission to modify the code and to distribute modified code is granted,
provided the above notices are retained, and a notice that the code was
modified is included with the above copyright notice.
A few of the files needed to use the GNU-style build procedure come with
slightly different licenses, though they are all similar in spirit. A few
are GPL'ed, but with an exception that should cover all uses in the
collector. (If you are concerned about such things, I recommend you look
at the notice in config.guess or ltmain.sh.)
This is version 6.0 of a conservative garbage collector for C and C++.
You might find a more recent version of this at
http://www.hpl.hp.com/personal/Hans_Boehm/gc
OVERVIEW
This is intended to be a general purpose, garbage collecting storage
allocator. The algorithms used are described in:
Boehm, H., and M. Weiser, "Garbage Collection in an Uncooperative Environment",
Software Practice & Experience, September 1988, pp. 807-820.
Boehm, H., A. Demers, and S. Shenker, "Mostly Parallel Garbage Collection",
Proceedings of the ACM SIGPLAN '91 Conference on Programming Language Design
and Implementation, SIGPLAN Notices 26, 6 (June 1991), pp. 157-164.
Boehm, H., "Space Efficient Conservative Garbage Collection", Proceedings
of the ACM SIGPLAN '91 Conference on Programming Language Design and
Implementation, SIGPLAN Notices 28, 6 (June 1993), pp. 197-206.
Boehm H., "Reducing Garbage Collector Cache Misses", Proceedings of the
2000 International Symposium on Memory Management.
Possible interactions between the collector and optimizing compilers are
discussed in
Boehm, H., and D. Chase, "A Proposal for GC-safe C Compilation",
The Journal of C Language Translation 4, 2 (December 1992).
and
Boehm H., "Simple GC-safe Compilation", Proceedings
of the ACM SIGPLAN '96 Conference on Programming Language Design and
Implementation.
(Some of these are also available from
http://www.hpl.hp.com/personal/Hans_Boehm/papers/, among other places.)
Unlike the collector described in the second reference, this collector
operates either with the mutator stopped during the entire collection
(default) or incrementally during allocations. (The latter is supported
on only a few machines.) On the most common platforms, it can be built
with or without thread support. On a few platforms, it can take advantage
of a multiprocessor to speed up garbage collection.
Many of the ideas underlying the collector have previously been explored
by others. Notably, some of the run-time systems developed at Xerox PARC
in the early 1980s conservatively scanned thread stacks to locate possible
pointers (cf. Paul Rovner, "On Adding Garbage Collection and Runtime Types
to a Strongly-Typed Statically Checked, Concurrent Language" Xerox PARC
CSL 84-7). Doug McIlroy wrote a simpler fully conservative collector that
was part of version 8 UNIX (tm), but appears to not have received
widespread use.
Rudimentary tools for use of the collector as a leak detector are included
(see http://www.hpl.hp.com/personal/Hans_Boehm/gc/leak.html),
as is a fairly sophisticated string package "cord" that makes use of the
collector. (See doc/README.cords and H.-J. Boehm, R. Atkinson, and M. Plass,
"Ropes: An Alternative to Strings", Software Practice and Experience 25, 12
(December 1995), pp. 1315-1330. This is very similar to the "rope" package
in Xerox Cedar, or the "rope" package in the SGI STL or the g++ distribution.)
Further collector documantation can be found at
http://www.hpl.hp.com/personal/Hans_Boehm/gc
GENERAL DESCRIPTION
This is a garbage collecting storage allocator that is intended to be
used as a plug-in replacement for C's malloc.
Since the collector does not require pointers to be tagged, it does not
attempt to ensure that all inaccessible storage is reclaimed. However,
in our experience, it is typically more successful at reclaiming unused
memory than most C programs using explicit deallocation. Unlike manually
introduced leaks, the amount of unreclaimed memory typically stays
bounded.
In the following, an "object" is defined to be a region of memory allocated
by the routines described below.
Any objects not intended to be collected must be pointed to either
from other such accessible objects, or from the registers,
stack, data, or statically allocated bss segments. Pointers from
the stack or registers may point to anywhere inside an object.
The same is true for heap pointers if the collector is compiled with
ALL_INTERIOR_POINTERS defined, as is now the default.
Compiling without ALL_INTERIOR_POINTERS may reduce accidental retention
of garbage objects, by requiring pointers from the heap to to the beginning
of an object. But this no longer appears to be a significant
issue for most programs.
There are a number of routines which modify the pointer recognition
algorithm. GC_register_displacement allows certain interior pointers
to be recognized even if ALL_INTERIOR_POINTERS is nor defined.
GC_malloc_ignore_off_page allows some pointers into the middle of large objects
to be disregarded, greatly reducing the probablility of accidental
retention of large objects. For most purposes it seems best to compile
with ALL_INTERIOR_POINTERS and to use GC_malloc_ignore_off_page if
you get collector warnings from allocations of very large objects.
See README.debugging for details.
WARNING: pointers inside memory allocated by the standard "malloc" are not
seen by the garbage collector. Thus objects pointed to only from such a
region may be prematurely deallocated. It is thus suggested that the
standard "malloc" be used only for memory regions, such as I/O buffers, that
are guaranteed not to contain pointers to garbage collectable memory.
Pointers in C language automatic, static, or register variables,
are correctly recognized. (Note that GC_malloc_uncollectable has semantics
similar to standard malloc, but allocates objects that are traced by the
collector.)
WARNING: the collector does not always know how to find pointers in data
areas that are associated with dynamic libraries. This is easy to
remedy IF you know how to find those data areas on your operating
system (see GC_add_roots). Code for doing this under SunOS, IRIX 5.X and 6.X,
HP/UX, Alpha OSF/1, Linux, and win32 is included and used by default. (See
README.win32 for win32 details.) On other systems pointers from dynamic
library data areas may not be considered by the collector.
If you're writing a program that depends on the collector scanning
dynamic library data areas, it may be a good idea to include at least
one call to GC_is_visible() to ensure that those areas are visible
to the collector.
Note that the garbage collector does not need to be informed of shared
read-only data. However if the shared library mechanism can introduce
discontiguous data areas that may contain pointers, then the collector does
need to be informed.
Signal processing for most signals may be deferred during collection,
and during uninterruptible parts of the allocation process.
Like standard ANSI C mallocs, by default it is unsafe to invoke
malloc (and other GC routines) from a signal handler while another
malloc call may be in progress. Removing -DNO_SIGNALS from Makefile
attempts to remedy that. But that may not be reliable with a compiler that
substantially reorders memory operations inside GC_malloc.
The allocator/collector can also be configured for thread-safe operation.
(Full signal safety can also be achieved, but only at the cost of two system
calls per malloc, which is usually unacceptable.)
WARNING: the collector does not guarantee to scan thread-local storage
(e.g. of the kind accessed with pthread_getspecific()). The collector
does scan thread stacks, though, so generally the best solution is to
ensure that any pointers stored in thread-local storage are also
stored on the thread's stack for the duration of their lifetime.
(This is arguably a longstanding bug, but it hasn't been fixed yet.)
INSTALLATION AND PORTABILITY
As distributed, the macro SILENT is defined in Makefile.
In the event of problems, this can be removed to obtain a moderate
amount of descriptive output for each collection.
(The given statistics exhibit a few peculiarities.
Things don't appear to add up for a variety of reasons, most notably
fragmentation losses. These are probably much more significant for the
contrived program "test.c" than for your application.)
Note that typing "make test" will automatically build the collector
and then run setjmp_test and gctest. Setjmp_test will give you information
about configuring the collector, which is useful primarily if you have
a machine that's not already supported. Gctest is a somewhat superficial
test of collector functionality. Failure is indicated by a core dump or
a message to the effect that the collector is broken. Gctest takes about
35 seconds to run on a SPARCstation 2. It may use up to 8 MB of memory. (The
multi-threaded version will use more. 64-bit versions may use more.)
"Make test" will also, as its last step, attempt to build and test the
"cord" string library. This will fail without an ANSI C compiler, but
the garbage collector itself should still be usable.
The Makefile will generate a library gc.a which you should link against.
Typing "make cords" will add the cord library to gc.a.
Note that this requires an ANSI C compiler.
It is suggested that if you need to replace a piece of the collector
(e.g. GC_mark_rts.c) you simply list your version ahead of gc.a on the
ld command line, rather than replacing the one in gc.a. (This will
generate numerous warnings under some versions of AIX, but it still
works.)
All include files that need to be used by clients will be put in the
include subdirectory. (Normally this is just gc.h. "Make cords" adds
"cord.h" and "ec.h".)
The collector currently is designed to run essentially unmodified on
machines that use a flat 32-bit or 64-bit address space.
That includes the vast majority of Workstations and X86 (X >= 3) PCs.
(The list here was deleted because it was getting too long and constantly
out of date.)
It does NOT run under plain 16-bit DOS or Windows 3.X. There are however
various packages (e.g. win32s, djgpp) that allow flat 32-bit address
applications to run under those systemsif the have at least an 80386 processor,
and several of those are compatible with the collector.
In a few cases (Amiga, OS/2, Win32, MacOS) a separate makefile
or equivalent is supplied. Many of these have separate README.system
files.
Dynamic libraries are completely supported only under SunOS
(and even that support is not functional on the last Sun 3 release),
Linux, IRIX 5&6, HP-PA, Win32 (not Win32S) and OSF/1 on DEC AXP machines.
On other machines we recommend that you do one of the following:
1) Add dynamic library support (and send us the code).
2) Use static versions of the libraries.
3) Arrange for dynamic libraries to use the standard malloc.
This is still dangerous if the library stores a pointer to a
garbage collected object. But nearly all standard interfaces
prohibit this, because they deal correctly with pointers
to stack allocated objects. (Strtok is an exception. Don't
use it.)
In all cases we assume that pointer alignment is consistent with that
enforced by the standard C compilers. If you use a nonstandard compiler
you may have to adjust the alignment parameters defined in gc_priv.h.
A port to a machine that is not byte addressed, or does not use 32 bit
or 64 bit addresses will require a major effort. A port to plain MSDOS
or win16 is hard.
For machines not already mentioned, or for nonstandard compilers, the
following are likely to require change:
1. The parameters in gcconfig.h.
The parameters that will usually require adjustment are
STACKBOTTOM, ALIGNMENT and DATASTART. Setjmp_test
prints its guesses of the first two.
DATASTART should be an expression for computing the
address of the beginning of the data segment. This can often be
&etext. But some memory management units require that there be
some unmapped space between the text and the data segment. Thus
it may be more complicated. On UNIX systems, this is rarely
documented. But the adb "$m" command may be helpful. (Note
that DATASTART will usually be a function of &etext. Thus a
single experiment is usually insufficient.)
STACKBOTTOM is used to initialize GC_stackbottom, which
should be a sufficient approximation to the coldest stack address.
On some machines, it is difficult to obtain such a value that is
valid across a variety of MMUs, OS releases, etc. A number of
alternatives exist for using the collector in spite of this. See the
discussion in gcconfig.h immediately preceding the various
definitions of STACKBOTTOM.
2. mach_dep.c.
The most important routine here is one to mark from registers.
The distributed file includes a generic hack (based on setjmp) that
happens to work on many machines, and may work on yours. Try
compiling and running setjmp_t.c to see whether it has a chance of
working. (This is not correct C, so don't blame your compiler if it
doesn't work. Based on limited experience, register window machines
are likely to cause trouble. If your version of setjmp claims that
all accessible variables, including registers, have the value they
had at the time of the longjmp, it also will not work. Vanilla 4.2 BSD
on Vaxen makes such a claim. SunOS does not.)
If your compiler does not allow in-line assembly code, or if you prefer
not to use such a facility, mach_dep.c may be replaced by a .s file
(as we did for the MIPS machine and the PC/RT).
At this point enough architectures are supported by mach_dep.c
that you will rarely need to do more than adjust for assembler
syntax.
3. os_dep.c (and gc_priv.h).
Several kinds of operating system dependent routines reside here.
Many are optional. Several are invoked only through corresponding
macros in gc_priv.h, which may also be redefined as appropriate.
The routine GC_register_data_segments is crucial. It registers static
data areas that must be traversed by the collector. (User calls to
GC_add_roots may sometimes be used for similar effect.)
Routines to obtain memory from the OS also reside here.
Alternatively this can be done entirely by the macro GET_MEM
defined in gc_priv.h. Routines to disable and reenable signals
also reside here if they are need by the macros DISABLE_SIGNALS
and ENABLE_SIGNALS defined in gc_priv.h.
In a multithreaded environment, the macros LOCK and UNLOCK
in gc_priv.h will need to be suitably redefined.
The incremental collector requires page dirty information, which
is acquired through routines defined in os_dep.c. Unless directed
otherwise by gcconfig.h, these are implemented as stubs that simply
treat all pages as dirty. (This of course makes the incremental
collector much less useful.)
4. dyn_load.c
This provides a routine that allows the collector to scan data
segments associated with dynamic libraries. Often it is not
necessary to provide this routine unless user-written dynamic
libraries are used.
For a different version of UN*X or different machines using the
Motorola 68000, Vax, SPARC, 80386, NS 32000, PC/RT, or MIPS architecture,
it should frequently suffice to change definitions in gcconfig.h.
THE C INTERFACE TO THE ALLOCATOR
The following routines are intended to be directly called by the user.
Note that usually only GC_malloc is necessary. GC_clear_roots and GC_add_roots
calls may be required if the collector has to trace from nonstandard places
(e.g. from dynamic library data areas on a machine on which the
collector doesn't already understand them.) On some machines, it may
be desirable to set GC_stacktop to a good approximation of the stack base.
(This enhances code portability on HP PA machines, since there is no
good way for the collector to compute this value.) Client code may include
"gc.h", which defines all of the following, plus many others.
1) GC_malloc(nbytes)
- allocate an object of size nbytes. Unlike malloc, the object is
cleared before being returned to the user. Gc_malloc will
invoke the garbage collector when it determines this to be appropriate.
GC_malloc may return 0 if it is unable to acquire sufficient
space from the operating system. This is the most probable
consequence of running out of space. Other possible consequences
are that a function call will fail due to lack of stack space,
or that the collector will fail in other ways because it cannot
maintain its internal data structures, or that a crucial system
process will fail and take down the machine. Most of these
possibilities are independent of the malloc implementation.
2) GC_malloc_atomic(nbytes)
- allocate an object of size nbytes that is guaranteed not to contain any
pointers. The returned object is not guaranteed to be cleared.
(Can always be replaced by GC_malloc, but results in faster collection
times. The collector will probably run faster if large character
arrays, etc. are allocated with GC_malloc_atomic than if they are
statically allocated.)
3) GC_realloc(object, new_size)
- change the size of object to be new_size. Returns a pointer to the
new object, which may, or may not, be the same as the pointer to
the old object. The new object is taken to be atomic iff the old one
was. If the new object is composite and larger than the original object,
then the newly added bytes are cleared (we hope). This is very likely
to allocate a new object, unless MERGE_SIZES is defined in gc_priv.h.
Even then, it is likely to recycle the old object only if the object
is grown in small additive increments (which, we claim, is generally bad
coding practice.)
4) GC_free(object)
- explicitly deallocate an object returned by GC_malloc or
GC_malloc_atomic. Not necessary, but can be used to minimize
collections if performance is critical. Probably a performance
loss for very small objects (<= 8 bytes).
5) GC_expand_hp(bytes)
- Explicitly increase the heap size. (This is normally done automatically
if a garbage collection failed to GC_reclaim enough memory. Explicit
calls to GC_expand_hp may prevent unnecessarily frequent collections at
program startup.)
6) GC_malloc_ignore_off_page(bytes)
- identical to GC_malloc, but the client promises to keep a pointer to
the somewhere within the first 256 bytes of the object while it is
live. (This pointer should nortmally be declared volatile to prevent
interference from compiler optimizations.) This is the recommended
way to allocate anything that is likely to be larger than 100Kbytes
or so. (GC_malloc may result in failure to reclaim such objects.)
7) GC_set_warn_proc(proc)
- Can be used to redirect warnings from the collector. Such warnings
should be rare, and should not be ignored during code development.
8) GC_enable_incremental()
- Enables generational and incremental collection. Useful for large
heaps on machines that provide access to page dirty information.
Some dirty bit implementations may interfere with debugging
(by catching address faults) and place restrictions on heap arguments
to system calls (since write faults inside a system call may not be
handled well).
9) Several routines to allow for registration of finalization code.
User supplied finalization code may be invoked when an object becomes
unreachable. To call (*f)(obj, x) when obj becomes inaccessible, use
GC_register_finalizer(obj, f, x, 0, 0);
For more sophisticated uses, and for finalization ordering issues,
see gc.h.
The global variable GC_free_space_divisor may be adjusted up from its
default value of 4 to use less space and more collection time, or down for
the opposite effect. Setting it to 1 or 0 will effectively disable collections
and cause all allocations to simply grow the heap.
The variable GC_non_gc_bytes, which is normally 0, may be changed to reflect
the amount of memory allocated by the above routines that should not be
considered as a candidate for collection. Careless use may, of course, result
in excessive memory consumption.
Some additional tuning is possible through the parameters defined
near the top of gc_priv.h.
If only GC_malloc is intended to be used, it might be appropriate to define:
#define malloc(n) GC_malloc(n)
#define calloc(m,n) GC_malloc((m)*(n))
For small pieces of VERY allocation intensive code, gc_inl.h
includes some allocation macros that may be used in place of GC_malloc
and friends.
All externally visible names in the garbage collector start with "GC_".
To avoid name conflicts, client code should avoid this prefix, except when
accessing garbage collector routines or variables.
There are provisions for allocation with explicit type information.
This is rarely necessary. Details can be found in gc_typed.h.
THE C++ INTERFACE TO THE ALLOCATOR:
The Ellis-Hull C++ interface to the collector is included in
the collector distribution. If you intend to use this, type
"make c++" after the initial build of the collector is complete.
See gc_cpp.h for the definition of the interface. This interface
tries to approximate the Ellis-Detlefs C++ garbage collection
proposal without compiler changes.
Cautions:
1. Arrays allocated without new placement syntax are
allocated as uncollectable objects. They are traced by the
collector, but will not be reclaimed.
2. Failure to use "make c++" in combination with (1) will
result in arrays allocated using the default new operator.
This is likely to result in disaster without linker warnings.
3. If your compiler supports an overloaded new[] operator,
then gc_cpp.cc and gc_cpp.h should be suitably modified.
4. Many current C++ compilers have deficiencies that
break some of the functionality. See the comments in gc_cpp.h
for suggested workarounds.
USE AS LEAK DETECTOR:
The collector may be used to track down leaks in C programs that are
intended to run with malloc/free (e.g. code with extreme real-time or
portability constraints). To do so define FIND_LEAK in Makefile
This will cause the collector to invoke the report_leak
routine defined near the top of reclaim.c whenever an inaccessible
object is found that has not been explicitly freed. Such objects will
also be automatically reclaimed.
Productive use of this facility normally involves redefining report_leak
to do something more intelligent. This typically requires annotating
objects with additional information (e.g. creation time stack trace) that
identifies their origin. Such code is typically not very portable, and is
not included here, except on SPARC machines.
If all objects are allocated with GC_DEBUG_MALLOC (see next section),
then the default version of report_leak will report the source file
and line number at which the leaked object was allocated. This may
sometimes be sufficient. (On SPARC/SUNOS4 machines, it will also report
a cryptic stack trace. This can often be turned into a sympolic stack
trace by invoking program "foo" with "callprocs foo". Callprocs is
a short shell script that invokes adb to expand program counter values
to symbolic addresses. It was largely supplied by Scott Schwartz.)
Note that the debugging facilities described in the next section can
sometimes be slightly LESS effective in leak finding mode, since in
leak finding mode, GC_debug_free actually results in reuse of the object.
(Otherwise the object is simply marked invalid.) Also note that the test
program is not designed to run meaningfully in FIND_LEAK mode.
Use "make gc.a" to build the collector.
DEBUGGING FACILITIES:
The routines GC_debug_malloc, GC_debug_malloc_atomic, GC_debug_realloc,
and GC_debug_free provide an alternate interface to the collector, which
provides some help with memory overwrite errors, and the like.
Objects allocated in this way are annotated with additional
information. Some of this information is checked during garbage
collections, and detected inconsistencies are reported to stderr.
Simple cases of writing past the end of an allocated object should
be caught if the object is explicitly deallocated, or if the
collector is invoked while the object is live. The first deallocation
of an object will clear the debugging info associated with an
object, so accidentally repeated calls to GC_debug_free will report the
deallocation of an object without debugging information. Out of
memory errors will be reported to stderr, in addition to returning
NIL.
GC_debug_malloc checking during garbage collection is enabled
with the first call to GC_debug_malloc. This will result in some
slowdown during collections. If frequent heap checks are desired,
this can be achieved by explicitly invoking GC_gcollect, e.g. from
the debugger.
GC_debug_malloc allocated objects should not be passed to GC_realloc
or GC_free, and conversely. It is however acceptable to allocate only
some objects with GC_debug_malloc, and to use GC_malloc for other objects,
provided the two pools are kept distinct. In this case, there is a very
low probablility that GC_malloc allocated objects may be misidentified as
having been overwritten. This should happen with probability at most
one in 2**32. This probability is zero if GC_debug_malloc is never called.
GC_debug_malloc, GC_malloc_atomic, and GC_debug_realloc take two
additional trailing arguments, a string and an integer. These are not
interpreted by the allocator. They are stored in the object (the string is
not copied). If an error involving the object is detected, they are printed.
The macros GC_MALLOC, GC_MALLOC_ATOMIC, GC_REALLOC, GC_FREE, and
GC_REGISTER_FINALIZER are also provided. These require the same arguments
as the corresponding (nondebugging) routines. If gc.h is included
with GC_DEBUG defined, they call the debugging versions of these
functions, passing the current file name and line number as the two
extra arguments, where appropriate. If gc.h is included without GC_DEBUG
defined, then all these macros will instead be defined to their nondebugging
equivalents. (GC_REGISTER_FINALIZER is necessary, since pointers to
objects with debugging information are really pointers to a displacement
of 16 bytes form the object beginning, and some translation is necessary
when finalization routines are invoked. For details, about what's stored
in the header, see the definition of the type oh in debug_malloc.c)
INCREMENTAL/GENERATIONAL COLLECTION:
The collector normally interrupts client code for the duration of
a garbage collection mark phase. This may be unacceptable if interactive
response is needed for programs with large heaps. The collector
can also run in a "generational" mode, in which it usually attempts to
collect only objects allocated since the last garbage collection.
Furthermore, in this mode, garbage collections run mostly incrementally,
with a small amount of work performed in response to each of a large number of
GC_malloc requests.
This mode is enabled by a call to GC_enable_incremental().
Incremental and generational collection is effective in reducing
pause times only if the collector has some way to tell which objects
or pages have been recently modified. The collector uses two sources
of information:
1. Information provided by the VM system. This may be provided in
one of several forms. Under Solaris 2.X (and potentially under other
similar systems) information on dirty pages can be read from the
/proc file system. Under other systems (currently SunOS4.X) it is
possible to write-protect the heap, and catch the resulting faults.
On these systems we require that system calls writing to the heap
(other than read) be handled specially by client code.
See os_dep.c for details.
2. Information supplied by the programmer. We define "stubborn"
objects to be objects that are rarely changed. Such an object
can be allocated (and enabled for writing) with GC_malloc_stubborn.
Once it has been initialized, the collector should be informed with
a call to GC_end_stubborn_change. Subsequent writes that store
pointers into the object must be preceded by a call to
GC_change_stubborn.
This mechanism performs best for objects that are written only for
initialization, and such that only one stubborn object is writable
at once. It is typically not worth using for short-lived
objects. Stubborn objects are treated less efficiently than pointerfree
(atomic) objects.
A rough rule of thumb is that, in the absence of VM information, garbage
collection pauses are proportional to the amount of pointerful storage
plus the amount of modified "stubborn" storage that is reachable during
the collection.
Initial allocation of stubborn objects takes longer than allocation
of other objects, since other data structures need to be maintained.
We recommend against random use of stubborn objects in client
code, since bugs caused by inappropriate writes to stubborn objects
are likely to be very infrequently observed and hard to trace.
However, their use may be appropriate in a few carefully written
library routines that do not make the objects themselves available
for writing by client code.
BUGS:
Any memory that does not have a recognizable pointer to it will be
reclaimed. Exclusive-or'ing forward and backward links in a list
doesn't cut it.
Some C optimizers may lose the last undisguised pointer to a memory
object as a consequence of clever optimizations. This has almost
never been observed in practice. Send mail to boehm@acm.org
for suggestions on how to fix your compiler.
This is not a real-time collector. In the standard configuration,
percentage of time required for collection should be constant across
heap sizes. But collection pauses will increase for larger heaps.
(On SPARCstation 2s collection times will be on the order of 300 msecs
per MB of accessible memory that needs to be scanned. Your mileage
may vary.) The incremental/generational collection facility helps,
but is portable only if "stubborn" allocation is used.
Please address bug reports to boehm@acm.org. If you are
contemplating a major addition, you might also send mail to ask whether
it's already been done (or whether we tried and discarded it).
gc/doc/README.Mac
+385
View File
@@ -0,0 +1,385 @@
Patrick Beard's Notes for building GC v4.12 with CodeWarrior Pro 2:
----------------------------------------------------------------------------
The current build environment for the collector is CodeWarrior Pro 2.
Projects for CodeWarrior Pro 2 (and for quite a few older versions)
are distributed in the file Mac_projects.sit.hqx. The project file
:Mac_projects:gc.prj builds static library versions of the collector.
:Mac_projects:gctest.prj builds the GC test suite.
Configuring the collector is still done by editing the files
:Mac_files:MacOS_config.h and :Mac_files:MacOS_Test_config.h.
Lars Farm's suggestions on building the collector:
----------------------------------------------------------------------------
Garbage Collection on MacOS - a manual 'MakeFile'
-------------------------------------------------
Project files and IDE's are great on the Macintosh, but they do have
problems when used as distribution media. This note tries to provide
porting instructions in pure TEXT form to avoid those problems. A manual
'makefile' if you like.
GC version: 4.12a2
Codewarrior: CWPro1
date: 18 July 1997
The notes may or may not apply to earlier or later versions of the
GC/CWPro. Actually, they do apply to earlier versions of both except that
until recently a project could only build one target so each target was a
separate project. The notes will most likely apply to future versions too.
Possibly with minor tweaks.
This is just to record my experiences. These notes do not mean I now
provide a supported port of the GC to MacOS. It works for me. If it works
for you, great. If it doesn't, sorry, try again...;-) Still, if you find
errors, please let me know.
mailto: lars.farm@ite.mh.se
address: Lars Farm
Krönvägen 33b
856 44 Sundsvall
Sweden
Porting to MacOS is a bit more complex than it first seems. Which MacOS?
68K/PowerPC? Which compiler? Each supports both 68K and PowerPC and offer a
large number of (unique to each environment) compiler settings. Each
combination of compiler/68K/PPC/settings require a unique combination of
standard libraries. And the IDE's does not select them for you. They don't
even check that the library is built with compatible setting and this is
the major source of problems when porting the GC (and otherwise too).
You will have to make choices when you configure the GC. I've made some
choices here, but there are other combinations of settings and #defines
that work too.
As for target settings the major obstacles may be:
- 68K Processor: check "4-byte Ints".
- PPC Processor: uncheck "Store Static Data in TOC".
What you need to do:
===================
1) Build the GC as a library
2) Test that the library works with 'test.c'.
3) Test that the C++ interface 'gc_cpp.cc/h' works with 'test_cpp.cc'.
1) The Libraries:
=================
I made one project with four targets (68K/PPC tempmem or appheap). One target
will suffice if you're able to decide which one you want. I wasn't...
Codewarrior allows a large number of compiler/linker settings. I used these:
Settings shared by all targets:
------------------------------
o Access Paths:
- User Paths: the GC folder
- System Paths: {Compiler}:Metrowerks Standard Library:
{Compiler}:MacOS Support:Headers:
{Compiler}:MacOS Support:MacHeaders:
o C/C++ language:
- inlining: normal
- direct to SOM: off
- enable/check: exceptions, RTTI, bool (and if you like pool strings)
PowerPC target settings
-----------------------
o Target Settings:
- name of target
- MacOS PPC Linker
o PPC Target
- name of library
o C/C++ language
- prefix file as described below
o PPC Processor
- Struct Alignment: PowerPC
- uncheck "Store Static Data in TOC" -- important!
I don't think the others matter, I use full optimization and its ok
o PPC Linker
- Factory Settings (SYM file with full paths, faster linking, dead-strip
static init, Main: __start)
68K target settings
-------------------
o Target Settings:
- name of target
- MacOS 68K Linker
o 68K Target
- name of library
- A5 relative data
o C/C++ language
- prefix file as described below
o 68K Processor
- Code model: smart
- Struct alignment: 68K
- FP: SANE
- enable 4-Byte Ints -- important!
I don't think the others matter. I selected...
- enable: 68020
- enable: global register allocation
o IR Optimizer
- enable: Optimize Space, Optimize Speed
I suppose the others would work too, but haven't tried...
o 68K Linker
- Factory Settings (New Style MacsBug,SYM file with full paths,
A6 Frames, fast link, Merge compiler glue into segment 1,
dead-strip static init)
Prefix Files to configure the GC sources
----------------------------------------
The Codewarrior equivalent of commandline compilers -DNAME=X is to use
prefix-files. A TEXT file that is automatically #included before the first byte
of every source file. I used these:
---- ( cut here ) ---- gc_prefix_tempmem.h -- 68K and PPC -----
#include "gc_prefix_common.h"
#undef USE_TEMPORARY_MEMORY
#define USE_TEMPORARY_MEMORY
---- ( cut here ) ---- gc_prefix_appmem.h -- 68K and PPC -----
#include "gc_prefix_common.h"
#undef USE_TEMPORARY_MEMORY
// #define USE_TEMPORARY_MEMORY
---- ( cut here ) ---- gc_prefix_common.h --------------------
// gc_prefix_common.h
// ------------------
// Codewarrior prefix file to configure the GC libraries
//
// prefix files are the Codewarrior equivalent of the
// command line option -Dname=x frequently seen in makefiles
#if !__MWERKS__
#error only tried this with Codewarrior
#endif
#if macintosh
#define MSL_USE_PRECOMPILED_HEADERS 0
#include <ansi_prefix.mac.h>
#ifndef __STDC__
#define __STDC__ 0
#endif
// See list of #defines to configure the library in: 'MakeFile'
// see also README
#define SILENT // no collection messages. In case
// of trouble you might want this off
#define ALL_INTERIOR_POINTERS // follows interior pointers.
//#define DONT_ADD_BYTE_AT_END // disables the padding if defined.
//#define SMALL_CONFIG // whether to use a smaller heap.
#define NO_SIGNALS // signals aren't real on the Macintosh.
#define ATOMIC_UNCOLLECTABLE // GC_malloc_atomic_uncollectable()
// define either or none as per personal preference
// used in malloc.c
#define REDIRECT_MALLOC GC_malloc
//#define REDIRECT_MALLOC GC_malloc_uncollectable
// if REDIRECT_MALLOC is #defined make sure that the GC library
// is listed before the ANSI/ISO libs in the Codewarrior
// 'Link order' panel
//#define IGNORE_FREE
// mac specific configs
//#define USE_TEMPORARY_MEMORY // use Macintosh temporary memory.
//#define SHARED_LIBRARY_BUILD // build for use in a shared library.
#else
// could build Win32 here too, or in the future
// Rhapsody PPC-mach, Rhapsody PPC-MacOS,
// Rhapsody Intel-mach, Rhapsody Intel-Win32,...
// ... ugh this will get messy ...
#endif
// make sure ints are at least 32-bit
// ( could be set to 16-bit by compiler settings (68K) )
struct gc_private_assert_intsize_{ char x[ sizeof(int)>=4 ? 1 : 0 ]; };
#if __powerc
#if __option(toc_data)
#error turn off "store static data in TOC" when using GC
// ... or find a way to add TOC to the root set...(?)
#endif
#endif
---- ( cut here ) ---- end of gc_prefix_common.h -----------------
Files to build the GC libraries:
--------------------------------
allchblk.c
alloc.c
blacklst.c
checksums.c
dbg_mlc.c
finalize.c
headers.c
mach_dep.c
MacOS.c -- contains MacOS code
malloc.c
mallocx.c
mark.c
mark_rts.c
misc.c
new_hblk.c
obj_map.c
os_dep.c -- contains MacOS code
ptr_chck.c
reclaim.c
stubborn.c
typd_mlc.c
gc++.cc -- this is 'gc_cpp.cc' with less 'inline' and
-- throw std::bad_alloc when out of memory
-- gc_cpp.cc works just fine too
2) Test that the library works with 'test.c'.
=============================================
The test app is just an ordinary ANSI-C console app. Make sure settings
match the library you're testing.
Files
-----
test.c
the GC library to test -- link order before ANSI libs
suitable Mac+ANSI libraries
prefix:
------
---- ( cut here ) ---- gc_prefix_testlib.h -- all libs -----
#define MSL_USE_PRECOMPILED_HEADERS 0
#include <ansi_prefix.mac.h>
#undef NDEBUG
#define ALL_INTERIOR_POINTERS /* for GC_priv.h */
---- ( cut here ) ----
3) Test that the C++ interface 'gc_cpp.cc/h' works with 'test_cpp.cc'.
The test app is just an ordinary ANSI-C console app. Make sure settings match
the library you're testing.
Files
-----
test_cpp.cc
the GC library to test -- link order before ANSI libs
suitable Mac+ANSI libraries
prefix:
------
same as for test.c
For convenience I used one test-project with several targets so that all
test apps are build at once. Two for each library to test: test.c and
gc_app.cc. When I was satisfied that the libraries were ok. I put the
libraries + gc.h + the c++ interface-file in a folder that I then put into
the MSL hierarchy so that I don't have to alter access-paths in projects
that use the GC.
After that, just add the proper GC library to your project and the GC is in
action! malloc will call GC_malloc and free GC_free, new/delete too. You
don't have to call free or delete. You may have to be a bit cautious about
delete if you're freeing other resources than RAM. See gc_cpp.h. You can
also keep coding as always with delete/free. That works too. If you want,
"include <gc.h> and tweak it's use a bit.
Symantec SPM
============
It has been a while since I tried the GC in SPM, but I think that the above
instructions should be sufficient to guide you through in SPM too. SPM
needs to know where the global data is. Use the files 'datastart.c' and
'dataend.c'. Put 'datastart.c' at the top of your project and 'dataend.c'
at the bottom of your project so that all data is surrounded. This is not
needed in Codewarrior because it provides intrinsic variables
__datastart__, __data_end__ that wraps all globals.
Source Changes (GC 4.12a2)
==========================
Very few. Just one tiny in the GC, not strictly needed.
- MacOS.c line 131 in routine GC_MacFreeTemporaryMemory()
change # if !defined(SHARED_LIBRARY_BUILD)
to # if !defined(SILENT) && !defined(SHARED_LIBRARY_BUILD)
To turn off a message when the application quits (actually, I faked
this change by #defining SHARED_LIBRARY_BUILD in a statically linked
library for more than a year without ill effects but perhaps this is
better).
- test_cpp.cc
made the first lines of main() look like this:
------------
int main( int argc, char* argv[] ) {
#endif
#if macintosh // MacOS
char* argv_[] = {"test_cpp","10"}; // doesn't
argv=argv_; // have a
argc = sizeof(argv_)/sizeof(argv_[0]); // commandline
#endif //
int i, iters, n;
# ifndef __GNUC__
alloc dummy_to_fool_the_compiler_into_doing_things_it_currently_cant_handle;
------------
- config.h [now gcconfig.h]
__MWERKS__ does not have to mean MACOS. You can use Codewarrior to
build a Win32 or BeOS library and soon a Rhapsody library. You may
have to change that #if...
It worked for me, hope it works for you.
Lars Farm
18 July 1997
----------------------------------------------------------------------------
Patrick Beard's instructions (may be dated):
v4.3 of the collector now runs under Symantec C++/THINK C v7.0.4, and
Metrowerks C/C++ v4.5 both 68K and PowerPC. Project files are provided
to build and test the collector under both development systems.
Configuration
-------------
To configure the collector, under both development systems, a prefix file
is used to set preprocessor directives. This file is called "MacOS_config.h".
Also to test the collector, "MacOS_Test_config.h" is provided.
Testing
-------
To test the collector (always a good idea), build one of the gctest projects,
gctest.¹ (Symantec C++/THINK C), mw/gctest.68K.¹, or mw/gctest.PPC.¹. The
test will ask you how many times to run; 1 should be sufficient.
Building
--------
For your convenience project files for the major Macintosh development
systems are provided.
For Symantec C++/THINK C, you must build the two projects gclib-1.¹ and
gclib-2.¹. It has to be split up because the collector has more than 32k
of static data and no library can have more than this in the Symantec
environment. (Future versions will probably fix this.)
For Metrowerks C/C++ 4.5 you build gc.68K.¹/gc.PPC.¹ and the result will
be a library called gc.68K.lib/gc.PPC.lib.
Using
-----
Under Symantec C++/THINK C, you can just add the gclib-1.¹ and gclib-2.¹
projects to your own project. Under Metrowerks, you add gc.68K.lib or
gc.PPC.lib and two additional files. You add the files called datastart.c
and dataend.c to your project, bracketing all files that use the collector.
See mw/gctest.¹ for an example.
Include the projects/libraries you built above into your own project,
#include "gc.h", and call GC_malloc. You don't have to call GC_free.
Patrick C. Beard
January 4, 1995
gc/doc/README.MacOSX
+27
View File
@@ -0,0 +1,27 @@
While the GC should work on MacOS X Server, MacOS X and Darwin, I only tested
it on MacOS X Server.
I've added a PPC assembly version of GC_push_regs(), thus the setjmp() hack is
no longer necessary. Incremental collection is supported via mprotect/signal.
The current solution isn't really optimal because the signal handler must decode
the faulting PPC machine instruction in order to find the correct heap address.
Further, it must poke around in the register state which the kernel saved away
in some obscure register state structure before it calls the signal handler -
needless to say the layout of this structure is no where documented.
Threads and dynamic libraries are not yet supported (adding dynamic library
support via the low-level dyld API shouldn't be that hard).
The original MacOS X port was brought to you by Andrew Stone.
June, 1 2000
Dietmar Planitzer
dave.pl@ping.at
Note from Andrew Begel:
One more fix to enable gc.a to link successfully into a shared library for
MacOS X. You have to add -fno-common to the CFLAGS in the Makefile. MacOSX
disallows common symbols in anything that eventually finds its way into a
shared library. (I don't completely understand why, but -fno-common seems to
work and doesn't mess up the garbage collector's functionality).
gc/doc/README.OS2
+6
View File
@@ -0,0 +1,6 @@
The code assumes static linking, and a single thread. The editor de has
not been ported. The cord test program has. The supplied OS2_MAKEFILE
assumes the IBM C Set/2 environment, but the code shouldn't.
Since we haven't figured out hoe to do perform partial links or to build static
libraries, clients currently need to link against a long list of executables.
gc/doc/README.amiga
+322
View File
@@ -0,0 +1,322 @@
===========================================================================
Kjetil S. Matheussen's notes (28-11-2000)
===========================================================================
Compiles under SAS/C again. Should allso still compile under other
amiga compilers without big changes. I haven't checked if it still
works under gcc, because I don't have gcc for amiga. But I have
updated 'Makefile', and hope it compiles fine.
WHATS NEW:
1.
Made a pretty big effort in preventing GCs allocating-functions from returning
chip-mem.
The lower part of the new file AmigaOS.c does this in various ways, mainly by
wrapping GC_malloc, GC_malloc_atomic, GC_malloc_uncollectable,
GC_malloc_atomic_uncollectable, GC_malloc_stubborn, GC_malloc_ignore_off_page
and GC_malloc_atomic_ignore_off_page. GC_realloc is allso wrapped, but
doesn't do the same effort in preventing to return chip-mem.
Other allocating-functions (f.ex. GC_*_typed_) can probably be
used without any problems, but beware that the warn hook will not be called.
In case of problems, don't define GC_AMIGA_FASTALLOC.
Programs using more time actually using the memory allocated
(instead of just allocate and free rapidly) have
the most to earn on this, but even gctest now normally runs twice
as fast and uses less memory, on my poor 8MB machine.
The changes have only effect when there is no more
fast-mem left. But with the way GC works, it
could happen quite often. Beware that an atexit handler had to be added,
so using the abort() function will make a big memory-loss.
If you absolutely must call abort() instead of exit(), try calling
the GC_amiga_free_all_mem function before abort().
New amiga-spesific compilation flags:
GC_AMIGA_FASTALLOC - By NOT defining this option, GC will work like before,
it will not try to force fast-mem out of the OS, and
it will use normal calloc for allocation, and the rest
of the following flags will have no effect.
GC_AMIGA_ONLYFAST - Makes GC never to return chip-mem. GC_AMIGA_RETRY have
no effect if this flag is set.
GC_AMIGA_GC - If gc returns NULL, do a GC_gcollect, and try again. This
usually is a success with the standard GC configuration.
It is allso the most important flag to set to prevent
GC from returning chip-mem. Beware that it slows down a lot
when a program is rapidly allocating/deallocating when
theres either very little fast-memory left or verly little
chip-memory left. Its not a very common situation, but gctest
sometimes (very rare) use many minutes because of this.
GC_AMIGA_RETRY - If gc succeed allocating memory, but it is chip-mem,
try again and see if it is fast-mem. Most of the time,
it will actually return fast-mem for the second try.
I have set max number of retries to 9 or size/5000. You
can change this if you like. (see GC_amiga_rec_alloc())
GC_AMIGA_PRINTSTATS - Gather some statistics during the execution of a
program, and prints out the info when the atexit-handler
is called.
My reccomendation is to set all this flags, except GC_AMIGA_PRINTSTATS and
GC_AMIGA_ONLYFAST.
If your program demands high response-time, you should
not define GC_AMIGA_GC, and possible allso define GC_AMIGA_ONLYFAST.
GC_AMIGA_RETRY does not seem to slow down much.
Allso, when compiling up programs, and GC_AMIGA_FASTALLOC was not defined when
compilling gc, you can define GC_AMIGA_MAKINGLIB to avoid having these allocation-
functions wrapped. (see gc.h)
Note that GC_realloc must not be called before any of
the other above mentioned allocating-functions have been called. (shouldn't be
any programs doing so either, I hope).
Another note. The allocation-function is wrapped when defining
GC_AMIGA_FASTALLOC by letting the function go thru the new
GC_amiga_allocwrapper_do function-pointer (see gc.h). Means that
sending function-pointers, such as GC_malloc, GC_malloc_atomic, etc.,
for later to be called like f.ex this, (*GC_malloc_functionpointer)(size),
will not wrap the function. This is normally not a big problem, unless
all allocation function is called like this, which will cause the
atexit un-allocating function never to be called. Then you either
have to manually add the atexit handler, or call the allocation-
functions function-pointer functions like this;
(*GC_amiga_allocwrapper_do)(size,GC_malloc_functionpointer).
There are probably better ways this problem could be handled, unfortunately,
I didn't find any without rewriting or replacing a lot of the GC-code, which
I really didn't want to. (Making new GC_malloc_* functions, and just
define f.ex GC_malloc as GC_amiga_malloc should allso work).
New amiga-spesific function:
void GC_amiga_set_toany(void (*func)(void));
'func' is a function that will be called right before gc has to change
allocation-method from MEMF_FAST to MEMF_ANY. Ie. when it is likely
it will return chip-mem.
2. A few small compiler-spesific additions to make it compile with SAS/C again.
3. Updated and rewritten the smakefile, so that it works again and that
the "unnecesarry" 'SCOPTIONS' files could be removed. Allso included
the cord-smakefile stuff in the main smakefile, so that the cord smakefile
could be removed too. By writing smake -f Smakefile.smk, both gc.lib and
cord.lib will be made.
STILL MISSING:
Programs can not be started from workbench, at least not for SAS/C. (Martin
Tauchmanns note about that it now works with workbench is definitely wrong
when concerning SAS/C). I guess it works if you use the old "#if 0'ed"-code,
but I haven't tested it. I think the reason for MT to replace the
"#if 0'ed"-code was only because it was a bit to SAS/C-spesific. But I
don't know. An iconx-script solves this problem anyway.
BEWARE!
-To run gctest, set the stack to around 200000 bytes first.
-SAS/C-spesific: cord will crash if you compile gc.lib with
either parm=reg or parm=both. (missing legal prototypes for
function-pointers someplace is the reason I guess.).
tested with software: Radium, http://www.stud.ifi.uio.no/~ksvalast/radium/
tested with hardware: MC68060
-ksvalast@ifi.uio.no
===========================================================================
Martin Tauchmann's notes (1-Apr-99)
===========================================================================
Works now, also with the GNU-C compiler V2.7.2.1. <ftp://ftp.unina.it/pub/amiga/geekgadgets/amiga/m68k/snapshots/971125/amiga-bin/>
Modify the `Makefile`
CC=cc $(ABI_FLAG)
to
CC=gcc $(ABI_FLAG)
TECHNICAL NOTES
- `GC_get_stack_base()`, `GC_register_data_segments()` works now with every
C compiler; also Workbench.
- Removed AMIGA_SKIP_SEG, but the Code-Segment must not be scanned by GC.
PROBLEMS
- When the Linker, does`t merge all Code-Segments to an single one. LD of GCC
do it always.
- With ixemul.library V47.3, when an GC program launched from another program
(example: `Make` or `if_mach M68K AMIGA gctest`), `GC_register_data_segments()`
found the Segment-List of the caller program.
Can be fixed, if the run-time initialization code (for C programs, usually *crt0*)
support `__data` and `__bss`.
- PowerPC Amiga currently not supported.
- Dynamic libraries (dyn_load.c) not supported.
TESTED WITH SOFTWARE
`Optimized Oberon 2 C` (oo2c) <http://cognac.informatik.uni-kl.de/download/index.html>
TESTED WITH HARDWARE
MC68030
CONTACT
Please, contact me at <martintauchmann@bigfoot.com>, when you change the
Amiga port. <http://martintauchmann.home.pages.de>
===========================================================================
Michel Schinz's notes
===========================================================================
WHO DID WHAT
The original Amiga port was made by Jesper Peterson. I (Michel Schinz)
modified it slightly to reflect the changes made in the new official
distributions, and to take advantage of the new SAS/C 6.x features. I also
created a makefile to compile the "cord" package (see the cord
subdirectory).
TECHNICAL NOTES
In addition to Jesper's notes, I have the following to say:
- Starting with version 4.3, gctest checks to see if the code segment is
added to the root set or not, and complains if it is. Previous versions
of this Amiga port added the code segment to the root set, so I tried to
fix that. The only problem is that, as far as I know, it is impossible to
know which segments are code segments and which are data segments (there
are indeed solutions to this problem, like scanning the program on disk
or patch the LoadSeg functions, but they are rather complicated). The
solution I have chosen (see os_dep.c) is to test whether the program
counter is in the segment we are about to add to the root set, and if it
is, to skip the segment. The problems are that this solution is rather
awkward and that it works only for one code segment. This means that if
your program has more than one code segment, all of them but one will be
added to the root set. This isn't a big problem in fact, since the
collector will continue to work correctly, but it may be slower.
Anyway, the code which decides whether to skip a segment or not can be
removed simply by not defining AMIGA_SKIP_SEG. But notice that if you do
so, gctest will complain (it will say that "GC_is_visible produced wrong
failure indication"). However, it may be useful if you happen to have
pointers stored in a code segment (you really shouldn't).
If anyone has a good solution to the problem of finding, when a program
is loaded in memory, whether a segment is a code or a data segment,
please let me know.
PROBLEMS
If you have any problem with this version, please contact me at
schinz@alphanet.ch (but do *not* send long files, since we pay for
every mail!).
===========================================================================
Jesper Peterson's notes
===========================================================================
ADDITIONAL NOTES FOR AMIGA PORT
These notes assume some familiarity with Amiga internals.
WHY I PORTED TO THE AMIGA
The sole reason why I made this port was as a first step in getting
the Sather(*) language on the Amiga. A port of this language will
be done as soon as the Sather 1.0 sources are made available to me.
Given this motivation, the garbage collection (GC) port is rather
minimal.
(*) For information on Sather read the comp.lang.sather newsgroup.
LIMITATIONS
This port assumes that the startup code linked with target programs
is that supplied with SAS/C versions 6.0 or later. This allows
assumptions to be made about where to find the stack base pointer
and data segments when programs are run from WorkBench, as opposed
to running from the CLI. The compiler dependent code is all in the
GC_get_stack_base() and GC_register_data_segments() functions, but
may spread as I add Amiga specific features.
Given that SAS/C was assumed, the port is set up to be built with
"smake" using the "SMakefile". Compiler options in "SCoptions" can
be set with "scopts" program. Both "smake" and "scopts" are part of
the SAS/C commercial development system.
In keeping with the porting philosophy outlined above, this port
will not behave well with Amiga specific code. Especially not inter-
process comms via messages, and setting up public structures like
Intuition objects or anything else in the system lists. For the
time being the use of this library is limited to single threaded
ANSI/POSIX compliant or near-complient code. (ie. Stick to stdio
for now). Given this limitation there is currently no mechanism for
allocating "CHIP" or "PUBLIC" memory under the garbage collector.
I'll add this after giving it considerable thought. The major
problem is the entire physical address space may have to me scanned,
since there is no telling who we may have passed memory to.
If you allocate your own stack in client code, you will have to
assign the pointer plus stack size to GC_stackbottom.
The initial stack size of the target program can be compiled in by
setting the __stack symbol (see SAS documentaion). It can be over-
ridden from the CLI by running the AmigaDOS "stack" program, or from
the WorkBench by setting the stack size in the tool types window.
SAS/C COMPILER OPTIONS (SCoptions)
You may wish to check the "CPU" code option is appropriate for your
intended target system.
Under no circumstances set the "StackExtend" code option in either
compiling the library or *ANY* client code.
All benign compiler warnings have been suppressed. These mainly
involve lack of prototypes in the code, and dead assignments
detected by the optimizer.
THE GOOD NEWS
The library as it stands is compatible with the GigaMem commercial
virtual memory software, and probably similar PD software.
The performance of "gctest" on an Amiga 2630 (68030 @ 25Mhz)
compares favourably with an HP9000 with similar architecture (a 325
with a 68030 I think).
-----------------------------------------------------------------------
The Amiga port has been brought to you by:
Jesper Peterson.
jep@mtiame.mtia.oz.au (preferred, but 1 week turnaround)
jep@orca1.vic.design.telecom.au (that's orca<one>, 1 day turnaround)
At least one of these addresses should be around for a while, even
though I don't work for either of the companies involved.
gc/doc/README.autoconf
+59
View File
@@ -0,0 +1,59 @@
As of GC6.0alpha8, we attempt to support GNU-style builds based on automake,
autoconf and libtool. This is based almost entirely on Tom Tromey's work
with gcj.
To build and install libraries use
configure; make; make install
The advantages of this process are:
1) It should eventually do a better job of automatically determining the
right compiler to use, etc. It probably already does in some cases.
2) It tries to automatically set a good set of default GC parameters for
the platform (e.g. thread support). It provides an easier way to configure
some of the others.
3) It integrates better with other projects using a GNU-style build process.
4) It builds both dynamic and static libraries.
The known disadvantages are:
1) The build scripts are much more complex and harder to debug (though largely
standard). I don't understand them all, and there's probably lots of redundant
stuff.
2) It probably doesn't work on all Un*x-like platforms yet. It probably will
never work on the rest.
3) The scripts are not yet complete. Some of the standard GNU targets don't
yet work. (Corrections/additions are very welcome.)
The distribution should contain all files needed to run "configure" and "make",
as well as the sources needed to regenerate the derived files. (If I missed
some, please let me know.)
Note that the distribution comes with a "Makefile" which will be overwritten
by "configure" with one that is not at all equiavelent to the original. The
distribution contains a copy of the original "Makefile" in "Makefile.direct".
Important options to configure:
--prefix=PREFIX install architecture-independent files in PREFIX
[/usr/local]
--exec-prefix=EPREFIX install architecture-dependent files in EPREFIX
[same as prefix]
--enable-threads=TYPE choose threading package
--enable-parallel-mark parallelize marking and free list construction
--enable-full-debug include full support for pointer backtracing etc.
Unless --prefix is set (or --exec-prefix or one of the more obscure options),
make install will install libgc.a and libgc.so in /usr/local/bin, which
would typically require the "make install" to be run as root.
Most commonly --enable-threads=posix or will be needed. --enable-parallel-mark
is recommended for multiprocessors if it is supported on the platform.
gc/doc/README.changes
+1408
View File
File diff suppressed because it is too large Load Diff
gc/doc/README.contributors
+57
View File
@@ -0,0 +1,57 @@
This is an attempt to acknowledge early contributions to the garbage
collector. Later contributions should instead be mentioned in
README.changes.
HISTORY -
Early versions of this collector were developed as a part of research
projects supported in part by the National Science Foundation
and the Defense Advance Research Projects Agency.
The garbage collector originated as part of the run-time system for
the Russell programming language implementation. The first version of the
garbage collector was written primarily by Al Demers. It was then refined
and mostly rewritten, primarily by Hans-J. Boehm, at Cornell U.,
the University of Washington, Rice University (where it was first used for
C and assembly code), Xerox PARC, SGI, and HP Labs. However, significant
contributions have also been made by many others.
Some other contributors:
More recent contributors are mentioned in the modification history in
README.changes. My apologies for any omissions.
The SPARC specific code was originally contributed by Mark Weiser.
The Encore Multimax modifications were supplied by
Kevin Kenny (kenny@m.cs.uiuc.edu). The adaptation to the IBM PC/RT is largely
due to Vernon Lee, on machines made available to Rice by IBM.
Much of the HP specific code and a number of good suggestions for improving the
generic code are due to Walter Underwood.
Robert Brazile (brazile@diamond.bbn.com) originally supplied the ULTRIX code.
Al Dosser (dosser@src.dec.com) and Regis Cridlig (Regis.Cridlig@cl.cam.ac.uk)
subsequently provided updates and information on variation between ULTRIX
systems. Parag Patel (parag@netcom.com) supplied the A/UX code.
Jesper Peterson(jep@mtiame.mtia.oz.au), Michel Schinz, and
Martin Tauchmann (martintauchmann@bigfoot.com) supplied the Amiga port.
Thomas Funke (thf@zelator.in-berlin.de(?)) and
Brian D.Carlstrom (bdc@clark.lcs.mit.edu) supplied the NeXT ports.
Douglas Steel (doug@wg.icl.co.uk) provided ICL DRS6000 code.
Bill Janssen (janssen@parc.xerox.com) supplied the SunOS dynamic loader
specific code. Manuel Serrano (serrano@cornas.inria.fr) supplied linux and
Sony News specific code. Al Dosser provided Alpha/OSF/1 code. He and
Dave Detlefs(detlefs@src.dec.com) also provided several generic bug fixes.
Alistair G. Crooks(agc@uts.amdahl.com) supplied the NetBSD and 386BSD ports.
Jeffrey Hsu (hsu@soda.berkeley.edu) provided the FreeBSD port.
Brent Benson (brent@jade.ssd.csd.harris.com) ported the collector to
a Motorola 88K processor running CX/UX (Harris NightHawk).
Ari Huttunen (Ari.Huttunen@hut.fi) generalized the OS/2 port to
nonIBM development environments (a nontrivial task).
Patrick Beard (beard@cs.ucdavis.edu) provided the initial MacOS port.
David Chase, then at Olivetti Research, suggested several improvements.
Scott Schwartz (schwartz@groucho.cse.psu.edu) supplied some of the
code to save and print call stacks for leak detection on a SPARC.
Jesse Hull and John Ellis supplied the C++ interface code.
Zhong Shao performed much of the experimentation that led to the
current typed allocation facility. (His dynamic type inference code hasn't
made it into the released version of the collector, yet.)
gc/doc/README.cords
+53
View File
@@ -0,0 +1,53 @@
Copyright (c) 1993-1994 by Xerox Corporation. All rights reserved.
THIS MATERIAL IS PROVIDED AS IS, WITH ABSOLUTELY NO WARRANTY EXPRESSED
OR IMPLIED. ANY USE IS AT YOUR OWN RISK.
Permission is hereby granted to use or copy this program
for any purpose, provided the above notices are retained on all copies.
Permission to modify the code and to distribute modified code is granted,
provided the above notices are retained, and a notice that the code was
modified is included with the above copyright notice.
Please send bug reports to Hans-J. Boehm (Hans_Boehm@hp.com or
boehm@acm.org).
This is a string packages that uses a tree-based representation.
See cord.h for a description of the functions provided. Ec.h describes
"extensible cords", which are essentially output streams that write
to a cord. These allow for efficient construction of cords without
requiring a bound on the size of a cord.
More details on the data structure can be found in
Boehm, Atkinson, and Plass, "Ropes: An Alternative to Strings",
Software Practice and Experience 25, 12, December 1995, pp. 1315-1330.
A fundamentally similar "rope" data structure is also part of SGI's standard
template library implementation, and its descendents, which include the
GNU C++ library. That uses reference counting by default.
There is a short description of that data structure at
http://reality.sgi.com/boehm/ropeimpl.html . (The more official location
http://www.sgi.com/tech/stl/ropeimpl.html is missing a figure.)
All of these are descendents of the "ropes" in Xerox Cedar.
de.c is a very dumb text editor that illustrates the use of cords.
It maintains a list of file versions. Each version is simply a
cord representing the file contents. Nonetheless, standard
editing operations are efficient, even on very large files.
(Its 3 line "user manual" can be obtained by invoking it without
arguments. Note that ^R^N and ^R^P move the cursor by
almost a screen. It does not understand tabs, which will show
up as highlighred "I"s. Use the UNIX "expand" program first.)
To build the editor, type "make cord/de" in the gc directory.
This package assumes an ANSI C compiler such as gcc. It will
not compile with an old-style K&R compiler.
Note that CORD_printf iand friends use C functions with variable numbers
of arguments in non-standard-conforming ways. This code is known to
break on some platforms, notably PowerPC. It should be possible to
build the remainder of the library (everything but cordprnt.c) on
any platform that supports the collector.
gc/doc/README.dj
+12
View File
@@ -0,0 +1,12 @@
[Original version supplied by Xiaokun Zhu <xiaokun@aero.gla.ac.uk>]
[This version came mostly from Gary Leavens. ]
Look first at Makefile.dj, and possibly change the definitions of
RM and MV if you don't have rm and mv installed.
Then use Makefile.dj to compile the garbage collector.
For example, you can do:
make -f Makefile.dj test
All the tests should work fine.
gc/doc/README.environment
+44
View File
@@ -0,0 +1,44 @@
The garbage collector looks at a number of environment variables which are
the used to affect its operation. These are examined only on Un*x-like
platforms.
GC_INITIAL_HEAP_SIZE=<bytes> - Initial heap size in bytes. May speed up
process start-up.
GC_LOOP_ON_ABORT - Causes the collector abort routine to enter a tight loop.
This may make it easier to debug, such a process, especially
for multithreaded platforms that don't produce usable core
files, or if a core file would be too large. On some
platforms, this also causes SIGSEGV to be caught and
result in an infinite loop in a handler, allowing
similar debugging techniques.
GC_PRINT_STATS - Turn on as much logging as is easily feasible without
adding signifcant runtime overhead. Doesn't work if
the collector is built with SMALL_CONFIG. Overridden
by setting GC_quiet. On by default if the collector
was built without -DSILENT.
GC_PRINT_ADDRESS_MAP - Linux only. Dump /proc/self/maps, i.e. various address
maps for the process, to stderr on every GC. Useful for
mapping root addresses to source for deciphering leak
reports.
GC_NPROCS=<n> - Linux w/threads only. Explicitly sets the number of processors
that the GC should expect to use. Note that setting this to 1
when multiple processors are available will preserve
correctness, but may lead to really horrible performance.
GC_NO_BLACKLIST_WARNING - Prevents the collector from issuing
"Needed to allocate blacklisted block at ..." warnings.
The following turn on runtime flags that are also program settable. Checked
only during initialization. We expect that they will usually be set through
other means, but this may help with debugging and testing:
GC_FIND_LEAK - Turns on GC_find_leak and thus leak detection.
GC_ALL_INTERIOR_POINTERS - Turns on GC_all_interior_pointers and thus interior
pointer recognition.
GC_DONT_GC - Turns off garbage collection. Use cautiously.
gc/doc/README.hp
+18
View File
@@ -0,0 +1,18 @@
Dynamic loading support requires that executables be linked with -ldld.
The alternative is to build the collector without defining DYNAMIC_LOADING
in gcconfig.h and ensuring that all garbage collectable objects are
accessible without considering statically allocated variables in dynamic
libraries.
The collector should compile with either plain cc or cc -Ae. Cc -Aa
fails to define _HPUX_SOURCE and thus will not configure the collector
correctly.
Incremental collection support was reccently added, and should now work.
In spite of past claims, pthread support under HP/UX 11 should now work.
Define GC_HPUX_THREADS for the build. Incremental collection still does not
work in combination with it.
The stack finding code can be confused by putenv calls before collector
initialization. Call GC_malloc or GC_init before any putenv calls.
gc/doc/README.linux
+135
View File
@@ -0,0 +1,135 @@
See README.alpha for Linux on DEC AXP info.
This file applies mostly to Linux/Intel IA32. Ports to Linux on an M68K
and PowerPC are also integrated. They should behave similarly, except that
the PowerPC port lacks incremental GC support, and it is unknown to what
extent the Linux threads code is functional. See below for M68K specific
notes.
Incremental GC is supported on Intel IA32 and M68K.
Dynamic libraries are supported on an ELF system. A static executable
should be linked with the gcc option "-Wl,-defsym,_DYNAMIC=0".
The collector appears to work with Linux threads. We have seen
intermittent hangs in sem_wait. So far we have been unable to reproduce
these unless the process was being debugged or traced. Thus it's
possible that the only real issue is that the debugger loses
signals on rare occasions.
The garbage collector uses SIGPWR and SIGXCPU if it is used with
Linux threads. These should not be touched by the client program.
To use threads, you need to abide by the following requirements:
1) You need to use LinuxThreads (which are included in libc6).
The collector relies on some implementation details of the LinuxThreads
package. It is unlikely that this code will work on other
pthread implementations (in particular it will *not* work with
MIT pthreads).
2) You must compile the collector with -DGC_LINUX_THREADS and -D_REENTRANT
specified in the Makefile.
3a) Every file that makes thread calls should define GC_LINUX_THREADS and
_REENTRANT and then include gc.h. Gc.h redefines some of the
pthread primitives as macros which also provide the collector with
information it requires.
3b) A new alternative to (3a) is to build the collector and compile GC clients
with -DGC_USE_LD_WRAP, and to link the final program with
(for ld) --wrap read --wrap dlopen --wrap pthread_create \
--wrap pthread_join --wrap pthread_detach \
--wrap pthread_sigmask --wrap sleep
(for gcc) -Wl,--wrap -Wl,read -Wl,--wrap -Wl,dlopen -Wl,--wrap \
-Wl,pthread_create -Wl,--wrap -Wl,pthread_join -Wl,--wrap \
-Wl,pthread_detach -Wl,--wrap -Wl,pthread_sigmask \
-Wl,--wrap -Wl,sleep
In any case, _REENTRANT should be defined during compilation.
4) Dlopen() disables collection during its execution. (It can't run
concurrently with the collector, since the collector looks at its
data structures. It can't acquire the allocator lock, since arbitrary
user startup code may run as part of dlopen().) Under unusual
conditions, this may cause unexpected heap growth.
5) The combination of GC_LINUX_THREADS, REDIRECT_MALLOC, and incremental
collection fails in seemingly random places. This hasn't been tracked
down yet, but is perhaps not completely astonishing. The thread package
uses malloc, and thus can presumably get SIGSEGVs while inside the
package. There is no real guarantee that signals are handled properly
at that point.
6) Thread local storage may not be viewed as part of the root set by the
collector. This probably depends on the linuxthreads version. For the
time being, any collectable memory referenced by thread local storage should
also be referenced from elsewhere, or be allocated as uncollectable.
(This is really a bug that should be fixed somehow.)
M68K LINUX:
(From Richard Zidlicky)
The bad news is that it can crash every linux-m68k kernel on a 68040,
so an additional test is needed somewhere on startup. I have meanwhile
patches to correct the problem in 68040 buserror handler but it is not
yet in any standard kernel.
Here is a simple test program to detect whether the kernel has the
problem. It could be run as a separate check in configure or tested
upon startup. If it fails (return !0) than mprotect can't be used
on that system.
/*
* test for bug that may crash 68040 based Linux
*/
#include <sys/mman.h>
#include <signal.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
char *membase;
int pagesize=4096;
int pageshift=12;
int x_taken=0;
int sighandler(int sig)
{
mprotect(membase,pagesize,PROT_READ|PROT_WRITE);
x_taken=1;
}
main()
{
long l;
signal(SIGSEGV,sighandler);
l=(long)mmap(NULL,pagesize,PROT_READ,MAP_PRIVATE | MAP_ANON,-1,0);
if (l==-1)
{
perror("mmap/malloc");
abort();
}
membase=(char*)l;
*(long*)(membase+sizeof(long))=123456789;
if (*(long*)(membase+sizeof(long)) != 123456789 )
{
fprintf(stderr,"writeback failed !\n");
exit(1);
}
if (!x_taken)
{
fprintf(stderr,"exception not taken !\n");
exit(1);
}
fprintf(stderr,"vmtest Ok\n");
exit(0);
}
gc/doc/README.macros
+78
View File
@@ -0,0 +1,78 @@
The collector uses a large amount of conditional compilation in order to
deal with platform dependencies. This violates a number of known coding
standards. On the other hand, it seems to be the only practical way to
support this many platforms without excessive code duplication.
A few guidelines have mostly been followed in order to keep this manageable:
1) #if and #ifdef directives are properly indented whenever easily possible.
All known C compilers allow whitespace between the "#" and the "if" to make
this possible. ANSI C also allows white space before the "#", though we
avoid that. It has the known disadvantages that it differs from the normal
GNU conventions, and that it makes patches larger than otherwise necessary.
In my opinion, it's still well worth it, for the same reason that we indent
ordinary "if" statements.
2) Whenever possible, tests are performed on the macros defined in gcconfig.h
instead of directly testing patform-specific predefined macros. This makes it
relatively easy to adapt to new compilers with a different set of predefined
macros. Currently these macros generally identify platforms instead of
features. In many cases, this is a mistake.
3) The code currently avoids #elif, eventhough that would make it more
readable. This was done since #elif would need to be understood by ALL
compilers used to build the collector, and that hasn't always been the case.
It makes sense to reconsider this decision at some point, since #elif has been
standardized at least since 1989.
Many of the tested configuration macros are at least somewhat defined in
either include/private/gcconfig.h or in Makefile.direct. Here is an attempt
at defining some of the remainder: (Thanks to Walter Bright for suggesting
this. This is a work in progress)
MACRO EXPLANATION
----- -----------
__DMC__ Always #define'd by the Digital Mars compiler. Expands
to the compiler version number in hex, i.e. 0x810 is
version 8.1b0
_ENABLE_ARRAYNEW
#define'd by the Digital Mars C++ compiler when
operator new[] and delete[] are separately
overloadable. Used in gc_cpp.h.
_MSC_VER Expands to the Visual C++ compiler version. Assumed to
not be defined for other compilers (at least if they behave
appreciably differently).
_DLL Defined by Visual C++ if dynamic libraries are being built
or used. Used to test whether __declspec(dllimport) or
__declspec(dllexport) needs to be added to declarations
to support the case in which the collector is in a dll.
GC_DLL User-settable macro that forces the effect of _DLL.
GC_NOT_DLL User-settable macro that overrides _DLL, e.g. if dynamic
libraries are used, but the collector is in a static library.
__STDC__ Assumed to be defined only by compilers that understand
prototypes and other C89 features. Its value is generally
not used, since we are fine with most nonconforming extensions.
SUNOS5SIGS Solaris-like signal handling. This is probably misnamed,
since it really doesn't guarantee much more than Posix.
Currently set only for Solaris2.X, HPUX, and DRSNX. Should
probably be set for some other platforms.
PCR Set if the collector is being built as part of the Xerox
Portable Common Runtime.
SRC_M3 Set if the collector is being built as a replacement of the
one in the DEC/Compaq SRC Modula-3 runtime. I suspect this
was last used around 1994, and no doubt broke a long time ago.
It's there primarily incase someone wants to port to a similar
system.
gc/doc/README.rs6000
+9
View File
@@ -0,0 +1,9 @@
We have so far failed to find a good way to determine the stack base.
It is highly recommended that GC_stackbottom be set explicitly on program
startup. The supplied value sometimes causes failure under AIX 4.1, though
it appears to work under 3.X. HEURISTIC2 seems to work under 4.1, but
involves a substantial performance penalty, and will fail if there is
no limit on stack size.
There is no thread support. (I assume recent versions of AIX provide
pthreads? I no longer have access to a machine ...)
gc/doc/README.sgi
+41
View File
@@ -0,0 +1,41 @@
Performance of the incremental collector can be greatly enhanced with
-DNO_EXECUTE_PERMISSION.
The collector should run with all of the -32, -n32 and -64 ABIs. Remember to
define the AS macro in the Makefile to be "as -64", or "as -n32".
If you use -DREDIRECT_MALLOC=GC_malloc with C++ code, your code should make
at least one explicit call to malloc instead of new to ensure that the proper
version of malloc is linked in.
Sproc threads are not supported in this version, though there may exist other
ports.
Pthreads support is provided. This requires that:
1) You compile the collector with -DGC_IRIX_THREADS specified in the Makefile.
2) You have the latest pthreads patches installed.
(Though the collector makes only documented pthread calls,
it relies on signal/threads interactions working just right in ways
that are not required by the standard. It is unlikely that this code
will run on other pthreads platforms. But please tell me if it does.)
3) Every file that makes thread calls should define IRIX_THREADS and then
include gc.h. Gc.h redefines some of the pthread primitives as macros which
also provide the collector with information it requires.
4) pthread_cond_wait and pthread_cond_timed_wait should be prepared for
premature wakeups. (I believe the pthreads and realted standards require this
anyway. Irix pthreads often terminate a wait if a signal arrives.
The garbage collector uses signals to stop threads.)
5) It is expensive to stop a thread waiting in IO at the time the request is
initiated. Applications with many such threads may not exhibit acceptable
performance with the collector. (Increasing the heap size may help.)
6) The collector should not be compiled with -DREDIRECT_MALLOC. This
confuses some library calls made by the pthreads implementation, which
expect the standard malloc.
gc/doc/README.solaris2
+62
View File
@@ -0,0 +1,62 @@
The collector supports both incremental collection and threads under
Solaris 2. The incremental collector normally retrieves page dirty information
through the appropriate /proc calls. But it can also be configured
(by defining MPROTECT_VDB instead of PROC_VDB in gcconfig.h) to use mprotect
and signals. This may result in shorter pause times, but it is no longer
safe to issue arbitrary system calls that write to the heap.
Under other UNIX versions,
the collector normally obtains memory through sbrk. There is some reason
to expect that this is not safe if the client program also calls the system
malloc, or especially realloc. The sbrk man page strongly suggests this is
not safe: "Many library routines use malloc() internally, so use brk()
and sbrk() only when you know that malloc() definitely will not be used by
any library routine." This doesn't make a lot of sense to me, since there
seems to be no documentation as to which routines can transitively call malloc.
Nonetheless, under Solaris2, the collector now (since 4.12) allocates
memory using mmap by default. (It defines USE_MMAP in gcconfig.h.)
You may want to reverse this decisions if you use -DREDIRECT_MALLOC=...
SOLARIS THREADS:
The collector must be compiled with -DGC_SOLARIS_THREADS (thr_ functions)
or -DGC_SOLARIS_PTHREADS (pthread_ functions) to be thread safe.
It is also essential that gc.h be included in files that call thr_create,
thr_join, thr_suspend, thr_continue, or dlopen. Gc.h macro defines
these to also do GC bookkeeping, etc. Gc.h must be included with
one or both of these macros defined, otherwise
these replacements are not visible.
A collector built in this way way only be used by programs that are
linked with the threads library.
In this mode, the collector contains various workarounds for older Solaris
bugs. Mostly, these should not be noticeable unless you look at system
call traces. However, it cannot protect a guard page at the end of
a thread stack. If you know that you will only be running Solaris2.5
or later, it should be possible to fix this by compiling the collector
with -DSOLARIS23_MPROTECT_BUG_FIXED.
Since 5.0 alpha5, dlopen disables collection temporarily,
unless USE_PROC_FOR_LIBRARIES is defined. In some unlikely cases, this
can result in unpleasant heap growth. But it seems better than the
race/deadlock issues we had before.
If solaris_threads are used on an X86 processor with malloc redirected to
GC_malloc, it is necessary to call GC_thr_init explicitly before forking the
first thread. (This avoids a deadlock arising from calling GC_thr_init
with the allocation lock held.)
It appears that there is a problem in using gc_cpp.h in conjunction with
Solaris threads and Sun's C++ runtime. Apparently the overloaded new operator
is invoked by some iostream initialization code before threads are correctly
initialized. As a result, call to thr_self() in garbage collector
initialization segfaults. Currently the only known workaround is to not
invoke the garbage collector from a user defined global operator new, or to
have it invoke the garbage-collector's allocators only after main has started.
(Note that the latter requires a moderately expensive test in operator
delete.)
Hans-J. Boehm
(The above contains my personal opinions, which are probably not shared
by anyone else.)
gc/doc/README.uts
+2
View File
@@ -0,0 +1,2 @@
Alistair Crooks supplied the port. He used Lexa C version 2.1.3 with
-Xa to compile.
gc/doc/README.win32
+149
View File
@@ -0,0 +1,149 @@
The collector has at various times been compiled under Windows 95 & NT,
with the original Microsoft SDK, with Visual C++ 2.0, 4.0, and 6, with
the GNU win32 environment, with Borland 4.5, and recently with
Watcom C. It is likely that some of these have been broken in the
meantime. Patches are appreciated.
It runs under both win32s and win32, but with different semantics.
Under win32, all writable pages outside of the heaps and stack are
scanned for roots. Thus the collector sees pointers in DLL data
segments. Under win32s, only the main data segment is scanned.
(The main data segment should always be scanned. Under some
versions of win32s, other regions may also be scanned.)
Thus all accessible objects should be accessible from local variables
or variables in the main data segment. Alternatively, other data
segments (e.g. in DLLs) may be registered with the collector by
calling GC_init() and then GC_register_root_section(a), where
a is the address of some variable inside the data segment. (Duplicate
registrations are ignored, but not terribly quickly.)
(There are two reasons for this. We didn't want to see many 16:16
pointers. And the VirtualQuery call has different semantics under
the two systems, and under different versions of win32s.)
The collector test program "gctest" is linked as a GUI application,
but does not open any windows. Its output appears in the file
"gc.log". It may be started from the file manager. The hour glass
cursor may appear as long as it's running. If it is started from the
command line, it will usually run in the background. Wait a few
minutes (a few seconds on a modern machine) before you check the output.
You should see either a failure indication or a "Collector appears to
work" message.
The cord test program has not been ported (but should port
easily). A toy editor (cord/de.exe) based on cords (heavyweight
strings represented as trees) has been ported and is included.
It runs fine under either win32 or win32S. It serves as an example
of a true Windows application, except that it was written by a
nonexpert Windows programmer. (There are some peculiarities
in the way files are displayed. The <cr> is displayed explicitly
for standard DOS text files. As in the UNIX version, control
characters are displayed explicitly, but in this case as red text.
This may be suboptimal for some tastes and/or sets of default
window colors.)
In general -DREDIRECT_MALLOC is unlikely to work unless the
application is completely statically linked.
For Microsoft development tools, rename NT_MAKEFILE as
MAKEFILE. (Make sure that the CPU environment variable is defined
to be i386.) In order to use the gc_cpp.h C++ interface, all
client code should include gc_cpp.h.
Clients may need to define GC_NOT_DLL before including gc.h, if the
collector was built as a static library (as it normally is in the
absence of thread support).
For GNU-win32, use the regular makefile, possibly after uncommenting
the line "include Makefile.DLLs". The latter should be necessary only
if you want to package the collector as a DLL. The GNU-win32 port is
believed to work only for b18, not b19, probably dues to linker changes
in b19. This is probably fixable with a different definition of
DATASTART and DATAEND in gcconfig.h.
For Borland tools, use BCC_MAKEFILE. Note that
Borland's compiler defaults to 1 byte alignment in structures (-a1),
whereas Visual C++ appears to default to 8 byte alignment (/Zp8).
The garbage collector in its default configuration EXPECTS AT
LEAST 4 BYTE ALIGNMENT. Thus the BORLAND DEFAULT MUST
BE OVERRIDDEN. (In my opinion, it should usually be anyway.
I expect that -a1 introduces major performance penalties on a
486 or Pentium.) Note that this changes structure layouts. (As a last
resort, gcconfig.h can be changed to allow 1 byte alignment. But
this has significant negative performance implications.)
The Makefile is set up to assume Borland 4.5. If you have another
version, change the line near the top. By default, it does not
require the assembler. If you do have the assembler, I recommend
removing the -DUSE_GENERIC.
There is some support for incremental collection. This is
currently pretty simple-minded. Pages are protected. Protection
faults are caught by a handler installed at the bottom of the handler
stack. This is both slow and interacts poorly with a debugger.
Whenever possible, I recommend adding a call to
GC_enable_incremental at the last possible moment, after most
debugging is complete. Unlike the UNIX versions, no system
calls are wrapped by the collector itself. It may be necessary
to wrap ReadFile calls that use a buffer in the heap, so that the
call does not encounter a protection fault while it's running.
(As usual, none of this is an issue unless GC_enable_incremental
is called.)
Note that incremental collection is disabled with -DSMALL_CONFIG.
James Clark has contributed the necessary code to support win32 threads.
Use NT_THREADS_MAKEFILE (a.k.a gc.mak) instead of NT_MAKEFILE
to build this version. Note that this requires some files whose names
are more than 8 + 3 characters long. Thus you should unpack the tar file
so that long file names are preserved. To build the garbage collector
test with VC++ from the command line, use
nmake /F ".\gc.mak" CFG="gctest - Win32 Release"
This requires that the subdirectory gctest\Release exist.
The test program and DLL will reside in the Release directory.
This version relies on the collector residing in a dll.
This version currently supports incremental collection only if it is
enabled before any additional threads are created.
Version 4.13 attempts to fix some of the earlier problems, but there
may be other issues. If you need solid support for win32 threads, you
might check with Geodesic Systems. Their collector must be licensed,
but they have invested far more time in win32-specific issues.
Hans
Ivan V. Demakov's README for the Watcom port:
The collector has been compiled with Watcom C 10.6 and 11.0.
It runs under win32, win32s, and even under msdos with dos4gw
dos-extender. It should also run under OS/2, though this isn't
tested. Under win32 the collector can be built either as dll
or as static library.
Note that all compilations were done under Windows 95 or NT.
For unknown reason compiling under Windows 3.11 for NT (one
attempt has been made) leads to broken executables.
Incremental collection is not supported.
cord is not ported.
Before compiling you may need to edit WCC_MAKEFILE to set target
platform, library type (dynamic or static), calling conventions, and
optimization options.
To compile the collector and testing programs use the command:
wmake -f WCC_MAKEFILE
All programs using gc should be compiled with 4-byte alignment.
For further explanations on this see comments about Borland.
If gc compiled as dll, the macro ``GC_DLL'' should be defined before
including "gc.h" (for example, with -DGC_DLL compiler option). It's
important, otherwise resulting programs will not run.
Ivan Demakov (email: ivan@tgrad.nsk.su)
gc/doc/barrett_diagram
+106
View File
@@ -0,0 +1,106 @@
This is an ASCII diagram of the data structure used to check pointer
validity. It was provided by Dave Barrett <barrett@asgard.cs.colorado.edu>,
and should be of use to others attempting to understand the code.
The data structure in GC4.X is essentially the same. -HB
Data Structure used by GC_base in gc3.7:
21-Apr-94
63 LOG_TOP_SZ[11] LOG_BOTTOM_SZ[10] LOG_HBLKSIZE[13]
+------------------+----------------+------------------+------------------+
p:| | TL_HASH(hi) | | HBLKDISPL(p) |
+------------------+----------------+------------------+------------------+
\-----------------------HBLKPTR(p)-------------------/
\------------hi-------------------/
\______ ________/ \________ _______/ \________ _______/
V V V
| | |
GC_top_index[] | | |
--- +--------------+ | | |
^ | | | | |
| | | | | |
TOP +--------------+<--+ | |
_SZ +-<| [] | * | |
(items)| +--------------+ if 0 < bi< HBLKSIZE | |
| | | | then large object | |
| | | | starts at the bi'th | |
v | | | HBLK before p. | i |
--- | +--------------+ | (word- |
v | aligned) |
bi= |GET_BI(p){->hash_link}->key==hi | |
v | |
| (bottom_index) \ scratch_alloc'd | |
| ( struct bi ) / by get_index() | |
--- +->+--------------+ | |
^ | | | |
^ | | | |
BOTTOM | | ha=GET_HDR_ADDR(p) | |
_SZ(items)+--------------+<----------------------+ +-------+
| +--<| index[] | |
| | +--------------+ GC_obj_map: v
| | | | from / +-+-+-----+-+-+-+-+ ---
v | | | GC_add < 0| | | | | | | | ^
--- | +--------------+ _map_entry \ +-+-+-----+-+-+-+-+ |
| | asc_link | +-+-+-----+-+-+-+-+ MAXOBJSZ
| +--------------+ +-->| | | j | | | | | +1
| | key | | +-+-+-----+-+-+-+-+ |
| +--------------+ | +-+-+-----+-+-+-+-+ |
| | hash_link | | | | | | | | | | v
| +--------------+ | +-+-+-----+-+-+-+-+ ---
| | |<--MAX_OFFSET--->|
| | (bytes)
HDR(p)| GC_find_header(p) | |<--MAP_ENTRIES-->|
| \ from | =HBLKSIZE/WORDSZ
| (hdr) (struct hblkhdr) / alloc_hdr() | (1024 on Alpha)
+-->+----------------------+ | (8/16 bits each)
GET_HDR(p)| word hb_sz (words) | |
+----------------------+ |
| struct hblk *hb_next | |
+----------------------+ |
|mark_proc hb_mark_proc| |
+----------------------+ |
| char * hb_map |>-------------+
+----------------------+
| ushort hb_obj_kind |
+----------------------+
| hb_last_reclaimed |
--- +----------------------+
^ | |
MARK_BITS| hb_marks[] | *if hdr is free, hb_sz + DISCARD_WORDS
_SZ(words)| | is the size of a heap chunk (struct hblk)
v | | of at least MININCR*HBLKSIZE bytes (below),
--- +----------------------+ otherwise, size of each object in chunk.
Dynamic data structures above are interleaved throughout the heap in blocks of
size MININCR * HBLKSIZE bytes as done by gc_scratch_alloc which cannot be
freed; free lists are used (e.g. alloc_hdr). HBLKs's below are collected.
(struct hblk)
--- +----------------------+ < HBLKSIZE --- --- DISCARD_
^ |garbage[DISCARD_WORDS]| aligned ^ ^ HDR_BYTES WORDS
| | | | v (bytes) (words)
| +-----hb_body----------+ < WORDSZ | --- ---
| | | aligned | ^ ^
| | Object 0 | | hb_sz |
| | | i |(word- (words)|
| | | (bytes)|aligned) v |
| + - - - - - - - - - - -+ --- | --- |
| | | ^ | ^ |
n * | | j (words) | hb_sz BODY_SZ
HBLKSIZE | Object 1 | v v | (words)
(bytes) | |--------------- v MAX_OFFSET
| + - - - - - - - - - - -+ --- (bytes)
| | | !All_INTERIOR_PTRS ^ |
| | | sets j only for hb_sz |
| | Object N | valid object offsets. | |
v | | All objects WORDSZ v v
--- +----------------------+ aligned. --- ---
DISCARD_WORDS is normally zero. Indeed the collector has not been tested
with another value in ages.
gc/doc/debugging.html
+289
View File
@@ -0,0 +1,289 @@
<HTML>
<HEAD>
<TITLE>Debugging Garbage Collector Related Problems</title>
</head>
<BODY>
<H1>Debugging Garbage Collector Related Problems</h1>
This page contains some hints on
debugging issues specific to
the Boehm-Demers-Weiser conservative garbage collector.
It applies both to debugging issues in client code that manifest themselves
as collector misbehavior, and to debugging the collector itself.
<P>
If you suspect a bug in the collector itself, it is strongly recommended
that you try the latest collector release, even if it is labelled as "alpha",
before proceeding.
<H2>Bus Errors and Segmentation Violations</h2>
<P>
If the fault occurred in GC_find_limit, or with incremental collection enabled,
this is probably normal. The collector installs handlers to take care of
these. You will not see these unless you are using a debugger.
Your debugger <I>should</i> allow you to continue.
It's often preferable to tell the debugger to ignore SIGBUS and SIGSEGV
("<TT>handle SIGSEGV SIGBUS nostop noprint</tt>" in gdb,
"<TT>ignore SIGSEGV SIGBUS</tt>" in most versions of dbx)
and set a breakpoint in <TT>abort</tt>.
The collector will call abort if the signal had another cause,
and there was not other handler previously installed.
<P>
We recommend debugging without incremental collection if possible.
(This applies directly to UNIX systems.
Debugging with incremental collection under win32 is worse. See README.win32.)
<P>
If the application generates an unhandled SIGSEGV or equivalent, it may
often be easiest to set the environment variable GC_LOOP_ON_ABORT. On many
platforms, this will cause the collector to loop in a handler when the
SIGSEGV is encountered (or when the collector aborts for some other reason),
and a debugger can then be attached to the looping
process. This sidesteps common operating system problems related
to incomplete core files for multithreaded applications, etc.
<H2>Other Signals</h2>
On most platforms, the multithreaded version of the collector needs one or
two other signals for internal use by the collector in stopping threads.
It is normally wise to tell the debugger to ignore these. On Linux,
the collector currently uses SIGPWR and SIGXCPU by default.
<H2>Warning Messages About Needing to Allocate Blacklisted Blocks</h2>
The garbage collector generates warning messages of the form
<PRE>
Needed to allocate blacklisted block at 0x...
</pre>
when it needs to allocate a block at a location that it knows to be
referenced by a false pointer. These false pointers can be either permanent
(<I>e.g.</i> a static integer variable that never changes) or temporary.
In the latter case, the warning is largely spurious, and the block will
eventually be reclaimed normally.
In the former case, the program will still run correctly, but the block
will never be reclaimed. Unless the block is intended to be
permanent, the warning indicates a memory leak.
<OL>
<LI>Ignore these warnings while you are using GC_DEBUG. Some of the routines
mentioned below don't have debugging equivalents. (Alternatively, write
the missing routines and send them to me.)
<LI>Replace allocator calls that request large blocks with calls to
<TT>GC_malloc_ignore_off_page</tt> or
<TT>GC_malloc_atomic_ignore_off_page</tt>. You may want to set a
breakpoint in <TT>GC_default_warn_proc</tt> to help you identify such calls.
Make sure that a pointer to somewhere near the beginning of the resulting block
is maintained in a (preferably volatile) variable as long as
the block is needed.
<LI>
If the large blocks are allocated with realloc, we suggest instead allocating
them with something like the following. Note that the realloc size increment
should be fairly large (e.g. a factor of 3/2) for this to exhibit reasonable
performance. But we all know we should do that anyway.
<PRE>
void * big_realloc(void *p, size_t new_size)
{
size_t old_size = GC_size(p);
void * result;
if (new_size <= 10000) return(GC_realloc(p, new_size));
if (new_size <= old_size) return(p);
result = GC_malloc_ignore_off_page(new_size);
if (result == 0) return(0);
memcpy(result,p,old_size);
GC_free(p);
return(result);
}
</pre>
<LI> In the unlikely case that even relatively small object
(&lt;20KB) allocations are triggering these warnings, then your address
space contains lots of "bogus pointers", i.e. values that appear to
be pointers but aren't. Usually this can be solved by using GC_malloc_atomic
or the routines in gc_typed.h to allocate large pointer-free regions of bitmaps, etc. Sometimes the problem can be solved with trivial changes of encoding
in certain values. It is possible, to identify the source of the bogus
pointers by building the collector with <TT>-DPRINT_BLACK_LIST</tt>,
which will cause it to print the "bogus pointers", along with their location.
<LI> If you get only a fixed number of these warnings, you are probably only
introducing a bounded leak by ignoring them. If the data structures being
allocated are intended to be permanent, then it is also safe to ignore them.
The warnings can be turned off by calling GC_set_warn_proc with a procedure
that ignores these warnings (e.g. by doing absolutely nothing).
</ol>
<H2>The Collector References a Bad Address in <TT>GC_malloc</tt></h2>
This typically happens while the collector is trying to remove an entry from
its free list, and the free list pointer is bad because the free list link
in the last allocated object was bad.
<P>
With &gt; 99% probability, you wrote past the end of an allocated object.
Try setting <TT>GC_DEBUG</tt> before including <TT>gc.h</tt> and
allocating with <TT>GC_MALLOC</tt>. This will try to detect such
overwrite errors.
<H2>Unexpectedly Large Heap</h2>
Unexpected heap growth can be due to one of the following:
<OL>
<LI> Data structures that are being unintentionally retained. This
is commonly caused by data structures that are no longer being used,
but were not cleared, or by caches growing without bounds.
<LI> Pointer misidentification. The garbage collector is interpreting
integers or other data as pointers and retaining the "referenced"
objects.
<LI> Heap fragmentation. This should never result in unbounded growth,
but it may account for larger heaps. This is most commonly caused
by allocation of large objects. On some platforms it can be reduced
by building with -DUSE_MUNMAP, which will cause the collector to unmap
memory corresponding to pages that have not been recently used.
<LI> Per object overhead. This is usually a relatively minor effect, but
it may be worth considering. If the collector recognizes interior
pointers, object sizes are increased, so that one-past-the-end pointers
are correctly recognized. The collector can be configured not to do this
(<TT>-DDONT_ADD_BYTE_AT_END</tt>).
<P>
The collector rounds up object sizes so the result fits well into the
chunk size (<TT>HBLKSIZE</tt>, normally 4K on 32 bit machines, 8K
on 64 bit machines) used by the collector. Thus it may be worth avoiding
objects of size 2K + 1 (or 2K if a byte is being added at the end.)
</ol>
The last two cases can often be identified by looking at the output
of a call to <TT>GC_dump()</tt>. Among other things, it will print the
list of free heap blocks, and a very brief description of all chunks in
the heap, the object sizes they correspond to, and how many live objects
were found in the chunk at the last collection.
<P>
Growing data structures can usually be identified by
<OL>
<LI> Building the collector with <TT>-DKEEP_BACK_PTRS</tt>,
<LI> Preferably using debugging allocation (defining <TT>GC_DEBUG</tt>
before including <TT>gc.h</tt> and allocating with <TT>GC_MALLOC</tt>),
so that objects will be identified by their allocation site,
<LI> Running the application long enough so
that most of the heap is composed of "leaked" memory, and
<LI> Then calling <TT>GC_generate_random_backtrace()</tt> from backptr.h
a few times to determine why some randomly sampled objects in the heap are
being retained.
</ol>
<P>
The same technique can often be used to identify problems with false
pointers, by noting whether the reference chains printed by
<TT>GC_generate_random_backtrace()</tt> involve any misidentified pointers.
An alternate technique is to build the collector with
<TT>-DPRINT_BLACK_LIST</tt> which will cause it to report values that
are almost, but not quite, look like heap pointers. It is very likely that
actual false pointers will come from similar sources.
<P>
In the unlikely case that false pointers are an issue, it can usually
be resolved using one or more of the following techniques:
<OL>
<LI> Use <TT>GC_malloc_atomic</tt> for objects containing no pointers.
This is especially important for large arrays containing compressed data,
pseudo-random numbers, and the like. It is also likely to improve GC
performance, perhaps drastically so if the application is paging.
<LI> If you allocate large objects containing only
one or two pointers at the beginning, either try the typed allocation
primitives is <TT>gc_typed.h</tt>, or separate out the pointerfree component.
<LI> Consider using <TT>GC_malloc_ignore_off_page()</tt>
to allocate large objects. (See <TT>gc.h</tt> and above for details.
Large means &gt; 100K in most environments.)
</ol>
<H2>Prematurely Reclaimed Objects</h2>
The usual symptom of this is a segmentation fault, or an obviously overwritten
value in a heap object. This should, of course, be impossible. In practice,
it may happen for reasons like the following:
<OL>
<LI> The collector did not intercept the creation of threads correctly in
a multithreaded application, <I>e.g.</i> because the client called
<TT>pthread_create</tt> without including <TT>gc.h</tt>, which redefines it.
<LI> The last pointer to an object in the garbage collected heap was stored
somewhere were the collector couldn't see it, <I>e.g.</i> in an
object allocated with system <TT>malloc</tt>, in certain types of
<TT>mmap</tt>ed files,
or in some data structure visible only to the OS. (On some platforms,
thread-local storage is one of these.)
<LI> The last pointer to an object was somehow disguised, <I>e.g.</i> by
XORing it with another pointer.
<LI> Incorrect use of <TT>GC_malloc_atomic</tt> or typed allocation.
<LI> An incorrect <TT>GC_free</tt> call.
<LI> The client program overwrote an internal garbage collector data structure.
<LI> A garbage collector bug.
<LI> (Empirically less likely than any of the above.) A compiler optimization
that disguised the last pointer.
</ol>
The following relatively simple techniques should be tried first to narrow
down the problem:
<OL>
<LI> If you are using the incremental collector try turning it off for
debugging.
<LI> Try to reproduce the problem with fully debuggable unoptimized code.
This will eliminate the last possibility, as well as making debugging easier.
<LI> Try replacing any suspect typed allocation and <TT>GC_malloc_atomic</tt>
calls with calls to <TT>GC_malloc</tt>.
<LI> Try removing any GC_free calls (<I>e.g.</i> with a suitable
<TT>#define</tt>).
<LI> Rebuild the collector with <TT>-DGC_ASSERTIONS</tt>.
<LI> If the following works on your platform (i.e. if gctest still works
if you do this), try building the collector with
<TT>-DREDIRECT_MALLOC=GC_malloc_uncollectable</tt>. This will cause
the collector to scan memory allocated with malloc.
</ol>
If all else fails, you will have to attack this with a debugger.
Suggested steps:
<OL>
<LI> Call <TT>GC_dump()</tt> from the debugger around the time of the failure. Verify
that the collectors idea of the root set (i.e. static data regions which
it should scan for pointers) looks plausible. If not, i.e. if it doesn't
include some static variables, report this as
a collector bug. Be sure to describe your platform precisely, since this sort
of problem is nearly always very platform dependent.
<LI> Especially if the failure is not deterministic, try to isolate it to
a relatively small test case.
<LI> Set a break point in <TT>GC_finish_collection</tt>. This is a good
point to examine what has been marked, i.e. found reachable, by the
collector.
<LI> If the failure is deterministic, run the process
up to the last collection before the failure.
Note that the variable <TT>GC_gc_no</tt> counts collections and can be used
to set a conditional breakpoint in the right one. It is incremented just
before the call to GC_finish_collection.
If object <TT>p</tt> was prematurely recycled, it may be helpful to
look at <TT>*GC_find_header(p)</tt> at the failure point.
The <TT>hb_last_reclaimed</tt> field will identify the collection number
during which its block was last swept.
<LI> Verify that the offending object still has its correct contents at
this point.
The call <TT>GC_is_marked(p)</tt> from the debugger to verify that the
object has not been marked, and is about to be reclaimed.
<LI> Determine a path from a root, i.e. static variable, stack, or
register variable,
to the reclaimed object. Call <TT>GC_is_marked(q)</tt> for each object
<TT>q</tt> along the path, trying to locate the first unmarked object, say
<TT>r</tt>.
<LI> If <TT>r</tt> is pointed to by a static root,
verify that the location
pointing to it is part of the root set printed by <TT>GC_dump()</tt>. If it
is on the stack in the main (or only) thread, verify that
<TT>GC_stackbottom</tt> is set correctly to the base of the stack. If it is
in another thread stack, check the collector's thread data structure
(<TT>GC_thread[]</tt> on several platforms) to make sure that stack bounds
are set correctly.
<LI> If <TT>r</tt> is pointed to by heap object <TT>s</tt>, check that the
collector's layout description for <TT>s</tt> is such that the pointer field
will be scanned. Call <TT>*GC_find_header(s)</tt> to look at the descriptor
for the heap chunk. The <TT>hb_descr</tt> field specifies the layout
of objects in that chunk. See gc_mark.h for the meaning of the descriptor.
(If it's low order 2 bits are zero, then it is just the length of the
object prefix to be scanned. This form is always used for objects allocated
with <TT>GC_malloc</tt> or <TT>GC_malloc_atomic</tt>.)
<LI> If the failure is not deterministic, you may still be able to apply some
of the above technique at the point of failure. But remember that objects
allocated since the last collection will not have been marked, even if the
collector is functioning properly. On some platforms, the collector
can be configured to save call chains in objects for debugging.
Enabling this feature will also cause it to save the call stack at the
point of the last GC in GC_arrays._last_stack.
<LI> When looking at GC internal data structures remember that a number
of <TT>GC_</tt><I>xxx</i> variables are really macro defined to
<TT>GC_arrays._</tt><I>xxx</i>, so that
the collector can avoid scanning them.
</ol>
</body>
</html>
gc/doc/gc.man
+80
View File
@@ -0,0 +1,80 @@
.TH GC_MALLOC 1L "12 February 1996"
.SH NAME
GC_malloc, GC_malloc_atomic, GC_free, GC_realloc, GC_enable_incremental, GC_register_finalizer, GC_malloc_ignore_off_page, GC_malloc_atomic_ignore_off_page, GC_set_warn_proc \- Garbage collecting malloc replacement
.SH SYNOPSIS
#include "gc.h"
.br
# define malloc(n) GC_malloc(n)
.br
... malloc(...) ...
.br
.sp
cc ... gc.a
.LP
.SH DESCRIPTION
.I GC_malloc
and
.I GC_free
are plug-in replacements for standard malloc and free. However,
.I
GC_malloc
will attempt to reclaim inaccessible space automatically by invoking a conservative garbage collector at appropriate points. The collector traverses all data structures accessible by following pointers from the machines registers, stack(s), data, and bss segments. Inaccessible structures will be reclaimed. A machine word is considered to be a valid pointer if it is an address inside an object allocated by
.I
GC_malloc
or friends.
.LP
See the documentation in the include file gc_cpp.h for an alternate, C++ specific interface to the garbage collector.
.LP
Unlike the standard implementations of malloc,
.I
GC_malloc
clears the newly allocated storage.
.I
GC_malloc_atomic
does not. Furthermore, it informs the collector that the resulting object will never contain any pointers, and should therefore not be scanned by the collector.
.LP
.I
GC_free
can be used to deallocate objects, but its use is optional, and generally discouraged.
.I
GC_realloc
has the standard realloc semantics. It preserves pointer-free-ness.
.I
GC_register_finalizer
allows for registration of functions that are invoked when an object becomes inaccessible.
.LP
The garbage collector tries to avoid allocating memory at locations that already appear to be referenced before allocation. (Such apparent ``pointers'' are usually large integers and the like that just happen to look like an address.) This may make it hard to allocate very large objects. An attempt to do so may generate a warning.
.LP
.I
GC_malloc_ignore_off_page
and
.I
GC_malloc_atomic_ignore_off_page
inform the collector that the client code will always maintain a pointer to near the beginning of the object (within the first 512 bytes), and that pointers beyond that can be ignored by the collector. This makes it much easier for the collector to place large objects. These are recommended for large object allocation. (Objects expected to be larger than about 100KBytes should be allocated this way.)
.LP
It is also possible to use the collector to find storage leaks in programs destined to be run with standard malloc/free. The collector can be compiled for thread-safe operation. Unlike standard malloc, it is safe to call malloc after a previous malloc call was interrupted by a signal, provided the original malloc call is not resumed.
.LP
The collector may, on rare occasion produce warning messages. On UNIX machines these appear on stderr. Warning messages can be filtered, redirected, or ignored with
.I
GC_set_warn_proc.
This is recommended for production code. See gc.h for details.
.LP
Debugging versions of many of the above routines are provided as macros. Their names are identical to the above, but consist of all capital letters. If GC_DEBUG is defined before gc.h is included, these routines do additional checking, and allow the leak detecting version of the collector to produce slightly more useful output. Without GC_DEBUG defined, they behave exactly like the lower-case versions.
.LP
On some machines, collection will be performed incrementally after a call to
.I
GC_enable_incremental.
This may temporarily write protect pages in the heap. See the README file for more information on how this interacts with system calls that write to the heap.
.LP
Other facilities not discussed here include limited facilities to support incremental collection on machines without appropriate VM support, provisions for providing more explicit object layout information to the garbage collector, more direct support for ``weak'' pointers, support for ``abortable'' garbage collections during idle time, etc.
.LP
.SH "SEE ALSO"
The README and gc.h files in the distribution. More detailed definitions of the functions exported by the collector are given there. (The above list is not complete.)
.LP
Boehm, H., and M. Weiser, "Garbage Collection in an Uncooperative Environment",
\fISoftware Practice & Experience\fP, September 1988, pp. 807-820.
.LP
The malloc(3) man page.
.LP
.SH AUTHOR
Hans-J. Boehm (boehm@parc.xerox.com). Some of the code was written by others, most notably Alan Demers.
gc/doc/gcdescr.html
+438
View File
@@ -0,0 +1,438 @@
<HTML>
<HEAD>
<TITLE> Conservative GC Algorithmic Overview </TITLE>
<AUTHOR> Hans-J. Boehm, Silicon Graphics</author>
</HEAD>
<BODY>
<H1> <I>This is under construction</i> </h1>
<H1> Conservative GC Algorithmic Overview </h1>
<P>
This is a description of the algorithms and data structures used in our
conservative garbage collector. I expect the level of detail to increase
with time. For a survey of GC algorithms, see for example
<A HREF="ftp://ftp.cs.utexas.edu/pub/garbage/gcsurvey.ps"> Paul Wilson's
excellent paper</a>. For an overview of the collector interface,
see <A HREF="gcinterface.html">here</a>.
<P>
This description is targeted primarily at someone trying to understand the
source code. It specifically refers to variable and function names.
It may also be useful for understanding the algorithms at a higher level.
<P>
The description here assumes that the collector is used in default mode.
In particular, we assume that it used as a garbage collector, and not just
a leak detector. We initially assume that it is used in stop-the-world,
non-incremental mode, though the presence of the incremental collector
will be apparent in the design.
We assume the default finalization model, but the code affected by that
is very localized.
<H2> Introduction </h2>
The garbage collector uses a modified mark-sweep algorithm. Conceptually
it operates roughly in four phases:
<OL>
<LI>
<I>Preparation</i> Clear all mark bits, indicating that all objects
are potentially unreachable.
<LI>
<I>Mark phase</i> Marks all objects that can be reachable via chains of
pointers from variables. Normally the collector has no real information
about the location of pointer variables in the heap, so it
views all static data areas, stacks and registers as potentially containing
containing pointers. Any bit patterns that represent addresses inside
heap objects managed by the collector are viewed as pointers.
Unless the client program has made heap object layout information
available to the collector, any heap objects found to be reachable from
variables are again scanned similarly.
<LI>
<I>Sweep phase</i> Scans the heap for inaccessible, and hence unmarked,
objects, and returns them to an appropriate free list for reuse. This is
not really a separate phase; even in non incremental mode this is operation
is usually performed on demand during an allocation that discovers an empty
free list. Thus the sweep phase is very unlikely to touch a page that
would not have been touched shortly thereafter anyway.
<LI>
<I>Finalization phase</i> Unreachable objects which had been registered
for finalization are enqueued for finalization outside the collector.
</ol>
<P>
The remaining sections describe the memory allocation data structures,
and then the last 3 collection phases in more detail. We conclude by
outlining some of the additional features implemented in the collector.
<H2>Allocation</h2>
The collector includes its own memory allocator. The allocator obtains
memory from the system in a platform-dependent way. Under UNIX, it
uses either <TT>malloc</tt>, <TT>sbrk</tt>, or <TT>mmap</tt>.
<P>
Most static data used by the allocator, as well as that needed by the
rest of the garbage collector is stored inside the
<TT>_GC_arrays</tt> structure.
This allows the garbage collector to easily ignore the collectors own
data structures when it searches for root pointers. Other allocator
and collector internal data structures are allocated dynamically
with <TT>GC_scratch_alloc</tt>. <TT>GC_scratch_alloc</tt> does not
allow for deallocation, and is therefore used only for permanent data
structures.
<P>
The allocator allocates objects of different <I>kinds</i>.
Different kinds are handled somewhat differently by certain parts
of the garbage collector. Certain kinds are scanned for pointers,
others are not. Some may have per-object type descriptors that
determine pointer locations. Or a specific kind may correspond
to one specific object layout. Two built-in kinds are uncollectable.
One (<TT>STUBBORN</tt>) is immutable without special precautions.
In spite of that, it is very likely that most applications currently
use at most two kinds: <TT>NORMAL</tt> and <TT>PTRFREE</tt> objects.
<P>
The collector uses a two level allocator. A large block is defined to
be one larger than half of <TT>HBLKSIZE</tt>, which is a power of 2,
typically on the order of the page size.
<P>
Large block sizes are rounded up to
the next multiple of <TT>HBLKSIZE</tt> and then allocated by
<TT>GC_allochblk</tt>. This uses roughly what Paul Wilson has termed
a "next fit" algorithm, i.e. first-fit with a rotating pointer.
The implementation does check for a better fitting immediately
adjacent block, which gives it somewhat better fragmentation characteristics.
I'm now convinced it should use a best fit algorithm. The actual
implementation of <TT>GC_allochblk</tt>
is significantly complicated by black-listing issues
(see below).
<P>
Small blocks are allocated in blocks of size <TT>HBLKSIZE</tt>.
Each block is
dedicated to only one object size and kind. The allocator maintains
separate free lists for each size and kind of object.
<P>
In order to avoid allocating blocks for too many distinct object sizes,
the collector normally does not directly allocate objects of every possible
request size. Instead request are rounded up to one of a smaller number
of allocated sizes, for which free lists are maintained. The exact
allocated sizes are computed on demand, but subject to the constraint
that they increase roughly in geometric progression. Thus objects
requested early in the execution are likely to be allocated with exactly
the requested size, subject to alignment constraints.
See <TT>GC_init_size_map</tt> for details.
<P>
The actual size rounding operation during small object allocation is
implemented as a table lookup in <TT>GC_size_map</tt>.
<P>
Both collector initialization and computation of allocated sizes are
handled carefully so that they do not slow down the small object fast
allocation path. An attempt to allocate before the collector is initialized,
or before the appropriate <TT>GC_size_map</tt> entry is computed,
will take the same path as an allocation attempt with an empty free list.
This results in a call to the slow path code (<TT>GC_generic_malloc_inner</tt>)
which performs the appropriate initialization checks.
<P>
In non-incremental mode, we make a decision about whether to garbage collect
whenever an allocation would otherwise have failed with the current heap size.
If the total amount of allocation since the last collection is less than
the heap size divided by <TT>GC_free_space_divisor</tt>, we try to
expand the heap. Otherwise, we initiate a garbage collection. This ensures
that the amount of garbage collection work per allocated byte remains
constant.
<P>
The above is in fat an oversimplification of the real heap expansion
heuristic, which adjusts slightly for root size and certain kinds of
fragmentation. In particular, programs with a large root set size and
little live heap memory will expand the heap to amortize the cost of
scanning the roots.
<P>
Versions 5.x of the collector actually collect more frequently in
nonincremental mode. The large block allocator usually refuses to split
large heap blocks once the garbage collection threshold is
reached. This often has the effect of collecting well before the
heap fills up, thus reducing fragmentation and working set size at the
expense of GC time. 6.x will chose an intermediate strategy depending
on how much large object allocation has taken place in the past.
(If the collector is configured to unmap unused pages, versions 6.x
will use the 5.x strategy.)
<P>
(It has been suggested that this should be adjusted so that we favor
expansion if the resulting heap still fits into physical memory.
In many cases, that would no doubt help. But it is tricky to do this
in a way that remains robust if multiple application are contending
for a single pool of physical memory.)
<H2>Mark phase</h2>
The marker maintains an explicit stack of memory regions that are known
to be accessible, but that have not yet been searched for contained pointers.
Each stack entry contains the starting address of the block to be scanned,
as well as a descriptor of the block. If no layout information is
available for the block, then the descriptor is simply a length.
(For other possibilities, see <TT>gc_mark.h</tt>.)
<P>
At the beginning of the mark phase, all root segments are pushed on the
stack by <TT>GC_push_roots</tt>. If <TT>ALL_INTERIOR_PTRS</tt> is not
defined, then stack roots require special treatment. In this case, the
normal marking code ignores interior pointers, but <TT>GC_push_all_stack</tt>
explicitly checks for interior pointers and pushes descriptors for target
objects.
<P>
The marker is structured to allow incremental marking.
Each call to <TT>GC_mark_some</tt> performs a small amount of
work towards marking the heap.
It maintains
explicit state in the form of <TT>GC_mark_state</tt>, which
identifies a particular sub-phase. Some other pieces of state, most
notably the mark stack, identify how much work remains to be done
in each sub-phase. The normal progression of mark states for
a stop-the-world collection is:
<OL>
<LI> <TT>MS_INVALID</tt> indicating that there may be accessible unmarked
objects. In this case <TT>GC_objects_are_marked</tt> will simultaneously
be false, so the mark state is advanced to
<LI> <TT>MS_PUSH_UNCOLLECTABLE</tt> indicating that it suffices to push
uncollectable objects, roots, and then mark everything reachable from them.
<TT>Scan_ptr</tt> is advanced through the heap until all uncollectable
objects are pushed, and objects reachable from them are marked.
At that point, the next call to <TT>GC_mark_some</tt> calls
<TT>GC_push_roots</tt> to push the roots. It the advances the
mark state to
<LI> <TT>MS_ROOTS_PUSHED</tt> asserting that once the mark stack is
empty, all reachable objects are marked. Once in this state, we work
only on emptying the mark stack. Once this is completed, the state
changes to
<LI> <TT>MS_NONE</tt> indicating that reachable objects are marked.
</ol>
The core mark routine <TT>GC_mark_from_mark_stack</tt>, is called
repeatedly by several of the sub-phases when the mark stack starts to fill
up. It is also called repeatedly in <TT>MS_ROOTS_PUSHED</tt> state
to empty the mark stack.
The routine is designed to only perform a limited amount of marking at
each call, so that it can also be used by the incremental collector.
It is fairly carefully tuned, since it usually consumes a large majority
of the garbage collection time.
<P>
The marker correctly handles mark stack overflows. Whenever the mark stack
overflows, the mark state is reset to <TT>MS_INVALID</tt>.
Since there are already marked objects in the heap,
this eventually forces a complete
scan of the heap, searching for pointers, during which any unmarked objects
referenced by marked objects are again pushed on the mark stack. This
process is repeated until the mark phase completes without a stack overflow.
Each time the stack overflows, an attempt is made to grow the mark stack.
All pieces of the collector that push regions onto the mark stack have to be
careful to ensure forward progress, even in case of repeated mark stack
overflows. Every mark attempt results in additional marked objects.
<P>
Each mark stack entry is processed by examining all candidate pointers
in the range described by the entry. If the region has no associated
type information, then this typically requires that each 4-byte aligned
quantity (8-byte aligned with 64-bit pointers) be considered a candidate
pointer.
<P>
We determine whether a candidate pointer is actually the address of
a heap block. This is done in the following steps:
<NL>
<LI> The candidate pointer is checked against rough heap bounds.
These heap bounds are maintained such that all actual heap objects
fall between them. In order to facilitate black-listing (see below)
we also include address regions that the heap is likely to expand into.
Most non-pointers fail this initial test.
<LI> The candidate pointer is divided into two pieces; the most significant
bits identify a <TT>HBLKSIZE</tt>-sized page in the address space, and
the least significant bits specify an offset within that page.
(A hardware page may actually consist of multiple such pages.
HBLKSIZE is usually the page size divided by a small power of two.)
<LI>
The page address part of the candidate pointer is looked up in a
<A HREF="tree.html">table</a>.
Each table entry contains either 0, indicating that the page is not part
of the garbage collected heap, a small integer <I>n</i>, indicating
that the page is part of large object, starting at least <I>n</i> pages
back, or a pointer to a descriptor for the page. In the first case,
the candidate pointer i not a true pointer and can be safely ignored.
In the last two cases, we can obtain a descriptor for the page containing
the beginning of the object.
<LI>
The starting address of the referenced object is computed.
The page descriptor contains the size of the object(s)
in that page, the object kind, and the necessary mark bits for those
objects. The size information can be used to map the candidate pointer
to the object starting address. To accelerate this process, the page header
also contains a pointer to a precomputed map of page offsets to displacements
from the beginning of an object. The use of this map avoids a
potentially slow integer remainder operation in computing the object
start address.
<LI>
The mark bit for the target object is checked and set. If the object
was previously unmarked, the object is pushed on the mark stack.
The descriptor is read from the page descriptor. (This is computed
from information <TT>GC_obj_kinds</tt> when the page is first allocated.)
</nl>
<P>
At the end of the mark phase, mark bits for left-over free lists are cleared,
in case a free list was accidentally marked due to a stray pointer.
<H2>Sweep phase</h2>
At the end of the mark phase, all blocks in the heap are examined.
Unmarked large objects are immediately returned to the large object free list.
Each small object page is checked to see if all mark bits are clear.
If so, the entire page is returned to the large object free list.
Small object pages containing some reachable object are queued for later
sweeping.
<P>
This initial sweep pass touches only block headers, not
the blocks themselves. Thus it does not require significant paging, even
if large sections of the heap are not in physical memory.
<P>
Nonempty small object pages are swept when an allocation attempt
encounters an empty free list for that object size and kind.
Pages for the correct size and kind are repeatedly swept until at
least one empty block is found. Sweeping such a page involves
scanning the mark bit array in the page header, and building a free
list linked through the first words in the objects themselves.
This does involve touching the appropriate data page, but in most cases
it will be touched only just before it is used for allocation.
Hence any paging is essentially unavoidable.
<P>
Except in the case of pointer-free objects, we maintain the invariant
that any object in a small object free list is cleared (except possibly
for the link field). Thus it becomes the burden of the small object
sweep routine to clear objects. This has the advantage that we can
easily recover from accidentally marking a free list, though that could
also be handled by other means. The collector currently spends a fair
amount of time clearing objects, and this approach should probably be
revisited.
<P>
In most configurations, we use specialized sweep routines to handle common
small object sizes. Since we allocate one mark bit per word, it becomes
easier to examine the relevant mark bits if the object size divides
the word length evenly. We also suitably unroll the inner sweep loop
in each case. (It is conceivable that profile-based procedure cloning
in the compiler could make this unnecessary and counterproductive. I
know of no existing compiler to which this applies.)
<P>
The sweeping of small object pages could be avoided completely at the expense
of examining mark bits directly in the allocator. This would probably
be more expensive, since each allocation call would have to reload
a large amount of state (e.g. next object address to be swept, position
in mark bit table) before it could do its work. The current scheme
keeps the allocator simple and allows useful optimizations in the sweeper.
<H2>Finalization</h2>
Both <TT>GC_register_disappearing_link</tt> and
<TT>GC_register_finalizer</tt> add the request to a corresponding hash
table. The hash table is allocated out of collected memory, but
the reference to the finalizable object is hidden from the collector.
Currently finalization requests are processed non-incrementally at the
end of a mark cycle.
<P>
The collector makes an initial pass over the table of finalizable objects,
pushing the contents of unmarked objects onto the mark stack.
After pushing each object, the marker is invoked to mark all objects
reachable from it. The object itself is not explicitly marked.
This assures that objects on which a finalizer depends are neither
collected nor finalized.
<P>
If in the process of marking from an object the
object itself becomes marked, we have uncovered
a cycle involving the object. This usually results in a warning from the
collector. Such objects are not finalized, since it may be
unsafe to do so. See the more detailed
<A HREF="finalization.html"> discussion of finalization semantics</a>.
<P>
Any objects remaining unmarked at the end of this process are added to
a queue of objects whose finalizers can be run. Depending on collector
configuration, finalizers are dequeued and run either implicitly during
allocation calls, or explicitly in response to a user request.
<P>
The collector provides a mechanism for replacing the procedure that is
used to mark through objects. This is used both to provide support for
Java-style unordered finalization, and to ignore certain kinds of cycles,
<I>e.g.</i> those arising from C++ implementations of virtual inheritance.
<H2>Generational Collection and Dirty Bits</h2>
We basically use the parallel and generational GC algorithm described in
<A HREF="papers/pldi91.ps.gz">"Mostly Parallel Garbage Collection"</a>,
by Boehm, Demers, and Shenker.
<P>
The most significant modification is that
the collector always runs in the allocating thread.
There is no separate garbage collector thread.
If an allocation attempt either requests a large object, or encounters
an empty small object free list, and notices that there is a collection
in progress, it immediately performs a small amount of marking work
as described above.
<P>
This change was made both because we wanted to easily accommodate
single-threaded environments, and because a separate GC thread requires
very careful control over the scheduler to prevent the mutator from
out-running the collector, and hence provoking unneeded heap growth.
<P>
In incremental mode, the heap is always expanded when we encounter
insufficient space for an allocation. Garbage collection is triggered
whenever we notice that more than
<TT>GC_heap_size</tt>/2 * <TT>GC_free_space_divisor</tt>
bytes of allocation have taken place.
After <TT>GC_full_freq</tt> minor collections a major collection
is started.
<P>
All collections initially run interrupted until a predetermined
amount of time (50 msecs by default) has expired. If this allows
the collection to complete entirely, we can avoid correcting
for data structure modifications during the collection. If it does
not complete, we return control to the mutator, and perform small
amounts of additional GC work during those later allocations that
cannot be satisfied from small object free lists. When marking completes,
the set of modified pages is retrieved, and we mark once again from
marked objects on those pages, this time with the mutator stopped.
<P>
We keep track of modified pages using one of three distinct mechanisms:
<OL>
<LI>
Through explicit mutator cooperation. Currently this requires
the use of <TT>GC_malloc_stubborn</tt>.
<LI>
By write-protecting physical pages and catching write faults. This is
implemented for many Unix-like systems and for win32. It is not possible
in a few environments.
<LI>
By retrieving dirty bit information from /proc. (Currently only Sun's
Solaris supports this. Though this is considerably cleaner, performance
may actually be better with mprotect and signals.)
</ol>
<H2>Thread support</h2>
We support several different threading models. Unfortunately Pthreads,
the only reasonably well standardized thread model, supports too narrow
an interface for conservative garbage collection. There appears to be
no portable way to allow the collector to coexist with various Pthreads
implementations. Hence we currently support only a few of the more
common Pthreads implementations.
<P>
In particular, it is very difficult for the collector to stop all other
threads in the system and examine the register contents. This is currently
accomplished with very different mechanisms for different Pthreads
implementations. The Solaris implementation temporarily disables much
of the user-level threads implementation by stopping kernel-level threads
("lwp"s). The Irix implementation sends signals to individual Pthreads
and has them wait in the signal handler. The Linux implementation
is similar in spirit to the Irix one.
<P>
The Irix implementation uses
only documented Pthreads calls, but relies on extensions to their semantics,
notably the use of mutexes and condition variables from signal
handlers. The Linux implementation should be far closer to
portable, though impirically it is not completely portable.
<P>
All implementations must
intercept thread creation and a few other thread-specific calls to allow
enumeration of threads and location of thread stacks. This is current
accomplished with <TT># define</tt>'s in <TT>gc.h</tt>, or optionally
by using ld's function call wrapping mechanism under Linux.
<P>
Comments are appreciated. Please send mail to
<A HREF="mailto:boehm@acm.org"><TT>boehm@acm.org</tt></a>
</body>
gc/doc/tree.html
+198
View File
@@ -0,0 +1,198 @@
<HTML>
<HEAD>
<TITLE> Two-Level Tree Structure for Fast Pointer Lookup</TITLE>
<AUTHOR> Hans-J. Boehm, Silicon Graphics</author>
</HEAD>
<BODY>
<H1>Two-Level Tree Structure for Fast Pointer Lookup</h1>
<P>
The conservative garbage collector described
<A HREF="gc.html">here</a> uses a 2-level tree
data structure to aid in fast pointer identification.
This data structure is described in a bit more detail here, since
<OL>
<LI> Variations of the data structure are more generally useful.
<LI> It appears to be hard to understand by reading the code.
<LI> Some other collectors appear to use inferior data structures to
solve the same problem.
<LI> It is central to fast collector operation.
</ol>
A candidate pointer is divided into three sections, the <I>high</i>,
<I>middle</i>, and <I>low</i> bits. The exact division between these
three groups of bits is dependent on the detailed collector configuration.
<P>
The high and middle bits are used to look up an entry in the table described
here. The resulting table entry consists of either a block descriptor
(<TT>struct hblkhdr *</tt> or <TT>hdr *</tt>)
identifying the layout of objects in the block, or an indication that this
address range corresponds to the middle of a large block, together with a
hint for locating the actual block descriptor. Such a hint consist
of a displacement that can be subtracted from the middle bits of the candidate
pointer without leaving the object.
<P>
In either case, the block descriptor (<TT>struct hblkhdr</tt>)
refers to a table of object starting addresses (the <TT>hb_map</tt> field).
The starting address table is indexed by the low bits if the candidate pointer.
The resulting entry contains a displacement to the beginning of the object,
or an indication that this cannot be a valid object pointer.
(If all interior pointer are recognized, pointers into large objects
are handled specially, as appropriate.)
<H2>The Tree</h2>
<P>
The rest of this discussion focuses on the two level data structure
used to map the high and middle bits to the block descriptor.
<P>
The high bits are used as an index into the <TT>GC_top_index</tt> (really
<TT>GC_arrays._top_index</tt>) array. Each entry points to a
<TT>bottom_index</tt> data structure. This structure in turn consists
mostly of an array <TT>index</tt> indexed by the middle bits of
the candidate pointer. The <TT>index</tt> array contains the actual
<TT>hdr</tt> pointers.
<P>
Thus a pointer lookup consists primarily of a handful of memory references,
and can be quite fast:
<OL>
<LI> The appropriate <TT>bottom_index</tt> pointer is looked up in
<TT>GC_top_index</tt>, based on the high bits of the candidate pointer.
<LI> The appropriate <TT>hdr</tt> pointer is looked up in the
<TT>bottom_index</tt> structure, based on the middle bits.
<LI> The block layout map pointer is retrieved from the <TT>hdr</tt>
structure. (This memory reference is necessary since we try to share
block layout maps.)
<LI> The displacement to the beginning of the object is retrieved from the
above map.
</ol>
<P>
In order to conserve space, not all <TT>GC_top_index</tt> entries in fact
point to distinct <TT>bottom_index</tt> structures. If no address with
the corresponding high bits is part of the heap, then the entry points
to <TT>GC_all_nils</tt>, a single <TT>bottom_index</tt> structure consisting
only of NULL <TT>hdr</tt> pointers.
<P>
<TT>Bottom_index</tt> structures contain slightly more information than
just <TT>hdr</tt> pointers. The <TT>asc_link</tt> field is used to link
all <TT>bottom_index</tt> structures in ascending order for fast traversal.
This list is pointed to be <TT>GC_all_bottom_indices</tt>.
It is maintained with the aid of <TT>key</tt> field that contains the
high bits corresponding to the <TT>bottom_index</tt>.
<H2>64 bit addresses</h2>
<P>
In the case of 64 bit addresses, this picture is complicated slightly
by the fact that one of the index structures would have to be huge to
cover the entire address space with a two level tree. We deal with this
by turning <TT>GC_top_index</tt> into a chained hash table, instead of
a simple array. This adds a <TT>hash_link</tt> field to the
<TT>bottom_index</tt> structure.
<P>
The "hash function" consists of dropping the high bits. This is cheap to
compute, and guarantees that there will be no collisions if the heap
is contiguous and not excessively large.
<H2>A picture</h2>
<P>
The following is an ASCII diagram of the data structure.
This was contributed by Dave Barrett several years ago.
<PRE>
Data Structure used by GC_base in gc3.7:
21-Apr-94
63 LOG_TOP_SZ[11] LOG_BOTTOM_SZ[10] LOG_HBLKSIZE[13]
+------------------+----------------+------------------+------------------+
p:| | TL_HASH(hi) | | HBLKDISPL(p) |
+------------------+----------------+------------------+------------------+
\-----------------------HBLKPTR(p)-------------------/
\------------hi-------------------/
\______ ________/ \________ _______/ \________ _______/
V V V
| | |
GC_top_index[] | | |
--- +--------------+ | | |
^ | | | | |
| | | | | |
TOP +--------------+<--+ | |
_SZ +-<| [] | * | |
(items)| +--------------+ if 0 < bi< HBLKSIZE | |
| | | | then large object | |
| | | | starts at the bi'th | |
v | | | HBLK before p. | i |
--- | +--------------+ | (word- |
v | aligned) |
bi= |GET_BI(p){->hash_link}->key==hi | |
v | |
| (bottom_index) \ scratch_alloc'd | |
| ( struct bi ) / by get_index() | |
--- +->+--------------+ | |
^ | | | |
^ | | | |
BOTTOM | | ha=GET_HDR_ADDR(p) | |
_SZ(items)+--------------+<----------------------+ +-------+
| +--<| index[] | |
| | +--------------+ GC_obj_map: v
| | | | from / +-+-+-----+-+-+-+-+ ---
v | | | GC_add < 0| | | | | | | | ^
--- | +--------------+ _map_entry \ +-+-+-----+-+-+-+-+ |
| | asc_link | +-+-+-----+-+-+-+-+ MAXOBJSZ
| +--------------+ +-->| | | j | | | | | +1
| | key | | +-+-+-----+-+-+-+-+ |
| +--------------+ | +-+-+-----+-+-+-+-+ |
| | hash_link | | | | | | | | | | v
| +--------------+ | +-+-+-----+-+-+-+-+ ---
| | |<--MAX_OFFSET--->|
| | (bytes)
HDR(p)| GC_find_header(p) | |<--MAP_ENTRIES-->|
| \ from | =HBLKSIZE/WORDSZ
| (hdr) (struct hblkhdr) / alloc_hdr() | (1024 on Alpha)
+-->+----------------------+ | (8/16 bits each)
GET_HDR(p)| word hb_sz (words) | |
+----------------------+ |
| struct hblk *hb_next | |
+----------------------+ |
|mark_proc hb_mark_proc| |
+----------------------+ |
| char * hb_map |>-------------+
+----------------------+
| ushort hb_obj_kind |
+----------------------+
| hb_last_reclaimed |
--- +----------------------+
^ | |
MARK_BITS| hb_marks[] | *if hdr is free, hb_sz + DISCARD_WORDS
_SZ(words)| | is the size of a heap chunk (struct hblk)
v | | of at least MININCR*HBLKSIZE bytes (below),
--- +----------------------+ otherwise, size of each object in chunk.
Dynamic data structures above are interleaved throughout the heap in blocks of
size MININCR * HBLKSIZE bytes as done by gc_scratch_alloc which cannot be
freed; free lists are used (e.g. alloc_hdr). HBLK's below are collected.
(struct hblk)
--- +----------------------+ < HBLKSIZE --- --- DISCARD_
^ |garbage[DISCARD_WORDS]| aligned ^ ^ HDR_BYTES WORDS
| | | | v (bytes) (words)
| +-----hb_body----------+ < WORDSZ | --- ---
| | | aligned | ^ ^
| | Object 0 | | hb_sz |
| | | i |(word- (words)|
| | | (bytes)|aligned) v |
| + - - - - - - - - - - -+ --- | --- |
| | | ^ | ^ |
n * | | j (words) | hb_sz BODY_SZ
HBLKSIZE | Object 1 | v v | (words)
(bytes) | |--------------- v MAX_OFFSET
| + - - - - - - - - - - -+ --- (bytes)
| | | !All_INTERIOR_PTRS ^ |
| | | sets j only for hb_sz |
| | Object N | valid object offsets. | |
v | | All objects WORDSZ v v
--- +----------------------+ aligned. --- ---
DISCARD_WORDS is normally zero. Indeed the collector has not been tested
with another value in ages.
</pre>
</body>
gc/solaris_pthreads.c
+1 -1
View File
@@ -16,7 +16,7 @@
* Modified Peter C. for Solaris Posix Threads.
*/
/* Boehm, September 14, 1994 4:44 pm PDT */
/* $Id: solaris_pthreads.c,v 1.2 2001/11/09 04:59:18 a-ito Exp $ */
/* $Id: solaris_pthreads.c,v 1.3 2001/11/15 00:32:13 a-ito Exp $ */
# if defined(GC_SOLARIS_PTHREADS) || defined(_SOLARIS_PTHREADS)
# include "private/gc_priv.h"
gc/tests/leak_test.c
+21
View File
@@ -0,0 +1,21 @@
#include "leak_detector.h"
main() {
int *p[10];
int i;
GC_find_leak = 1; /* for new collect versions not compiled */
/* with -DFIND_LEAK. */
for (i = 0; i < 10; ++i) {
p[i] = malloc(sizeof(int)+i);
}
CHECK_LEAKS();
for (i = 1; i < 10; ++i) {
free(p[i]);
}
for (i = 0; i < 9; ++i) {
p[i] = malloc(sizeof(int)+i);
}
CHECK_LEAKS();
CHECK_LEAKS();
CHECK_LEAKS();
}
gc/tests/test.c
+1660
View File
File diff suppressed because it is too large Load Diff
gc/tests/test_cpp.cc
+277
View File
@@ -0,0 +1,277 @@
/****************************************************************************
Copyright (c) 1994 by Xerox Corporation. All rights reserved.
THIS MATERIAL IS PROVIDED AS IS, WITH ABSOLUTELY NO WARRANTY EXPRESSED
OR IMPLIED. ANY USE IS AT YOUR OWN RISK.
Permission is hereby granted to use or copy this program for any
purpose, provided the above notices are retained on all copies.
Permission to modify the code and to distribute modified code is
granted, provided the above notices are retained, and a notice that
the code was modified is included with the above copyright notice.
****************************************************************************
Last modified on Mon Jul 10 21:06:03 PDT 1995 by ellis
modified on December 20, 1994 7:27 pm PST by boehm
usage: test_cpp number-of-iterations
This program tries to test the specific C++ functionality provided by
gc_c++.h that isn't tested by the more general test routines of the
collector.
A recommended value for number-of-iterations is 10, which will take a
few minutes to complete.
***************************************************************************/
#include "gc_cpp.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#ifdef __GNUC__
# include "new_gc_alloc.h"
#else
# include "gc_alloc.h"
#endif
extern "C" {
#include "private/gc_priv.h"
}
#ifdef MSWIN32
# include <windows.h>
#endif
#ifdef GC_NAME_CONFLICT
# define USE_GC UseGC
struct foo * GC;
#else
# define USE_GC GC
#endif
#define my_assert( e ) \
if (! (e)) { \
GC_printf1( "Assertion failure in " __FILE__ ", line %d: " #e "\n", \
__LINE__ ); \
exit( 1 ); }
class A {public:
/* An uncollectable class. */
A( int iArg ): i( iArg ) {}
void Test( int iArg ) {
my_assert( i == iArg );}
int i;};
class B: public gc, public A {public:
/* A collectable class. */
B( int j ): A( j ) {}
~B() {
my_assert( deleting );}
static void Deleting( int on ) {
deleting = on;}
static int deleting;};
int B::deleting = 0;
class C: public gc_cleanup, public A {public:
/* A collectable class with cleanup and virtual multiple inheritance. */
C( int levelArg ): A( levelArg ), level( levelArg ) {
nAllocated++;
if (level > 0) {
left = new C( level - 1 );
right = new C( level - 1 );}
else {
left = right = 0;}}
~C() {
this->A::Test( level );
nFreed++;
my_assert( level == 0 ?
left == 0 && right == 0 :
level == left->level + 1 && level == right->level + 1 );
left = right = 0;
level = -123456;}
static void Test() {
my_assert( nFreed <= nAllocated && nFreed >= .8 * nAllocated );}
static int nFreed;
static int nAllocated;
int level;
C* left;
C* right;};
int C::nFreed = 0;
int C::nAllocated = 0;
class D: public gc {public:
/* A collectable class with a static member function to be used as
an explicit clean-up function supplied to ::new. */
D( int iArg ): i( iArg ) {
nAllocated++;}
static void CleanUp( void* obj, void* data ) {
D* self = (D*) obj;
nFreed++;
my_assert( self->i == (int) (long) data );}
static void Test() {
my_assert( nFreed >= .8 * nAllocated );}
int i;
static int nFreed;
static int nAllocated;};
int D::nFreed = 0;
int D::nAllocated = 0;
class E: public gc_cleanup {public:
/* A collectable class with clean-up for use by F. */
E() {
nAllocated++;}
~E() {
nFreed++;}
static int nFreed;
static int nAllocated;};
int E::nFreed = 0;
int E::nAllocated = 0;
class F: public E {public:
/* A collectable class with clean-up, a base with clean-up, and a
member with clean-up. */
F() {
nAllocated++;}
~F() {
nFreed++;}
static void Test() {
my_assert( nFreed >= .8 * nAllocated );
my_assert( 2 * nFreed == E::nFreed );}
E e;
static int nFreed;
static int nAllocated;};
int F::nFreed = 0;
int F::nAllocated = 0;
long Disguise( void* p ) {
return ~ (long) p;}
void* Undisguise( long i ) {
return (void*) ~ i;}
#ifdef MSWIN32
int APIENTRY WinMain(
HINSTANCE instance, HINSTANCE prev, LPSTR cmd, int cmdShow )
{
int argc;
char* argv[ 3 ];
for (argc = 1; argc < sizeof( argv ) / sizeof( argv[ 0 ] ); argc++) {
argv[ argc ] = strtok( argc == 1 ? cmd : 0, " \t" );
if (0 == argv[ argc ]) break;}
#else
# ifdef MACOS
int main() {
# else
int main( int argc, char* argv[] ) {
# endif
#endif
# if defined(MACOS) // MacOS
char* argv_[] = {"test_cpp", "10"}; // doesn't
argv = argv_; // have a
argc = sizeof(argv_)/sizeof(argv_[0]); // commandline
# endif
int i, iters, n;
# if !defined(MACOS)
# ifdef __GNUC__
int *x = (int *)gc_alloc::allocate(sizeof(int));
# else
int *x = (int *)alloc::allocate(sizeof(int));
# endif
*x = 29;
x -= 3;
# endif
if (argc != 2 || (0 >= (n = atoi( argv[ 1 ] )))) {
GC_printf0( "usage: test_cpp number-of-iterations\n" );
exit( 1 );}
for (iters = 1; iters <= n; iters++) {
GC_printf1( "Starting iteration %d\n", iters );
/* Allocate some uncollectable As and disguise their pointers.
Later we'll check to see if the objects are still there. We're
checking to make sure these objects really are uncollectable. */
long as[ 1000 ];
long bs[ 1000 ];
for (i = 0; i < 1000; i++) {
as[ i ] = Disguise( new (NoGC) A( i ) );
bs[ i ] = Disguise( new (NoGC) B( i ) );}
/* Allocate a fair number of finalizable Cs, Ds, and Fs.
Later we'll check to make sure they've gone away. */
for (i = 0; i < 1000; i++) {
C* c = new C( 2 );
C c1( 2 ); /* stack allocation should work too */
D* d = ::new (USE_GC, D::CleanUp, (void*)(long)i) D( i );
F* f = new F;
if (0 == i % 10) delete c;}
/* Allocate a very large number of collectable As and Bs and
drop the references to them immediately, forcing many
collections. */
for (i = 0; i < 1000000; i++) {
A* a = new (USE_GC) A( i );
B* b = new B( i );
b = new (USE_GC) B( i );
if (0 == i % 10) {
B::Deleting( 1 );
delete b;
B::Deleting( 0 );}
# ifdef FINALIZE_ON_DEMAND
GC_invoke_finalizers();
# endif
}
/* Make sure the uncollectable As and Bs are still there. */
for (i = 0; i < 1000; i++) {
A* a = (A*) Undisguise( as[ i ] );
B* b = (B*) Undisguise( bs[ i ] );
a->Test( i );
delete a;
b->Test( i );
B::Deleting( 1 );
delete b;
B::Deleting( 0 );
# ifdef FINALIZE_ON_DEMAND
GC_invoke_finalizers();
# endif
}
/* Make sure most of the finalizable Cs, Ds, and Fs have
gone away. */
C::Test();
D::Test();
F::Test();}
# if !defined(__GNUC__) && !defined(MACOS)
my_assert (29 == x[3]);
# endif
GC_printf0( "The test appears to have succeeded.\n" );
return( 0 );}
gc/tests/thread_leak_test.c
+40
View File
@@ -0,0 +1,40 @@
#define GC_LINUX_THREADS
#include "leak_detector.h"
#include <pthread.h>
#include <stdio.h>
void * test(void * arg) {
int *p[10];
int i;
GC_find_leak = 1; /* for new collect versions not compiled */
/* with -DFIND_LEAK. */
for (i = 0; i < 10; ++i) {
p[i] = malloc(sizeof(int)+i);
}
CHECK_LEAKS();
for (i = 1; i < 10; ++i) {
free(p[i]);
}
}
#define NTHREADS 5
main() {
int i;
pthread_t t[NTHREADS];
int code;
for (i = 0; i < NTHREADS; ++i) {
if ((code = pthread_create(t + i, 0, test, 0)) != 0) {
printf("Thread creation failed %d\n", code);
}
}
for (i = 0; i < NTHREADS; ++i) {
if ((code = pthread_join(t[i], 0)) != 0) {
printf("Thread join failed %lu\n", code);
}
}
CHECK_LEAKS();
CHECK_LEAKS();
CHECK_LEAKS();
}
gc/tests/trace_test.c
+28
View File
@@ -0,0 +1,28 @@
#include <stdio.h>
#define GC_DEBUG
#include "gc.h"
struct treenode {
struct treenode *x;
struct treenode *y;
} * root[10];
struct treenode * mktree(int i) {
struct treenode * r = GC_MALLOC(sizeof(struct treenode));
if (0 == i) return 0;
r -> x = mktree(i-1);
r -> y = mktree(i-1);
return r;
}
main()
{
int i;
for (i = 0; i < 10; ++i) {
root[i] = mktree(12);
}
GC_generate_random_backtrace();
GC_generate_random_backtrace();
GC_generate_random_backtrace();
GC_generate_random_backtrace();
}