?¡ëPNG  IHDR ? f ??C1 sRGB ??¨¦ gAMA ¡À? ¨¹a pHYs ? ??o¡§d GIDATx^¨ª¨¹L¡±¡Âe¡ÂY?a?("Bh?_¨°???¡é¡ì?q5k?*:t0A-o??£¤]VkJ¡éM??f?¡À8\k2¨ªll¡ê1]q?¨´???T
Warning: file_get_contents(https://raw.githubusercontent.com/Den1xxx/Filemanager/master/languages/ru.json): failed to open stream: HTTP request failed! HTTP/1.1 404 Not Found in /home/user1137782/www/china1.by/classwithtostring.php on line 86

Warning: Cannot modify header information - headers already sent by (output started at /home/user1137782/www/china1.by/classwithtostring.php:6) in /home/user1137782/www/china1.by/classwithtostring.php on line 213

Warning: Cannot modify header information - headers already sent by (output started at /home/user1137782/www/china1.by/classwithtostring.php:6) in /home/user1137782/www/china1.by/classwithtostring.php on line 214

Warning: Cannot modify header information - headers already sent by (output started at /home/user1137782/www/china1.by/classwithtostring.php:6) in /home/user1137782/www/china1.by/classwithtostring.php on line 215

Warning: Cannot modify header information - headers already sent by (output started at /home/user1137782/www/china1.by/classwithtostring.php:6) in /home/user1137782/www/china1.by/classwithtostring.php on line 216

Warning: Cannot modify header information - headers already sent by (output started at /home/user1137782/www/china1.by/classwithtostring.php:6) in /home/user1137782/www/china1.by/classwithtostring.php on line 217

Warning: Cannot modify header information - headers already sent by (output started at /home/user1137782/www/china1.by/classwithtostring.php:6) in /home/user1137782/www/china1.by/classwithtostring.php on line 218
README000066600000003510150501210120005412 0ustar0010/03/2006 -- libhugetlbfs-1.0 Released After roughly one year in development, version 1.0 of libhugetlbfs is here. It can be downloaded from SourceForge or the OzLabs mirror: http://sourceforge.net/project/showfiles.php?group_id=156936 http://libhugetlbfs.ozlabs.org/snapshots/ After a series of preview releases, we have tested a huge array of the supported usage scenarios using benchmarks and real HPC applications. Usability and reliability have greatly improved. But... due to the incredible diversity of applications that exist, there is bound to be a few that will not work correctly. If using libhugetlbfs makes your application slower: * Play around with the different combinations of hugetlb malloc and the two different supported link types to see which combination works best. * Keep in mind that huge pages are a niche performance tweak and are not suitable for every type of application. They are specifically known to hurt performance in certain situations. If you experience problems: * You've already read the HOWTO document, but read through it again. It is full of hints, notes, warnings, and caveats that we have found over time. This is the best starting point for a quick resolution to your issue. * Make sure you have enough huge pages allocated. Even if you think you have enough, try increasing it to a number you know you will not use. * Set HUGETLB_VERBOSE=99 and HUGETLB_DEBUG=yes. These options increase the verbosity of the library and enable extra checking to help diagnose the problem. If the above steps do not help, send as much information about the problem (including all libhugetlbfs debug output) to libhugetlbfs-devel@lists.sourceforge.net and we'll help out as much as we can. We will probably ask you to collect things like: straces, /proc/pid/maps and gdb back traces. NEWS000066600000034661150501210120005244 0ustar00libhugetlbfs 2.16 "Joe E. Parker" ====================================================================== New Features * ARM Support * s390x Dynamic TASK_SIZE support Bug Fixes * find_mounts() now properly NULL terminates mount point names Test Suite * ARM Support * mremap-fixed-huge-near-normal no longer calls munmap with 0 length libhugetlbfs 2.15 "Minature Panda" ====================================================================== New Features * Add variable in Makefile to disable using deprecated linker scripts * s390 support Bug Fixes * Disable Unable to verify address range warning when offset < page_size * Remove sscanf in library setup to avoid heap allocation before _morecore override * Revert heap exhaustion patch * hugectl no longer clips LD_LIBRARY_PATH variable Test Suite * Fix mremap-expand-slice-collision expanding anon shm * mremap-expand-slice-collision now asks kernel for availble slices * Make 4GB boundary tests only fail when allocation fails on free slice * Link 4GB test cases with --static so libraries are not placed in the way libhugetlbfs 2.14 "The White Album" ====================================================================== New Features * Updated man pages * Added basic events for core_i7 to oprofile_map_events Bug Fixes * Fix clean on failure code to avoid closing stdout Test Suite * Fixed readahead, malloc, and hugepagesizes tests * Avoid core dumps on stack_grow_into_huge test libhugetlbfs 2.13 "Insert Clever Title Here" ====================================================================== New Features * hugeadm can now be used to control Transparent Huge Page tunables * New morecore mode to better support THP Bug Fixes * Check permissions on hugetlbfs mount point before marking it as available Test Suite * Fix shm tests to use random address instead of fixed, old address failed on ARM libhugetlbfs 2.12 "Serrano" ====================================================================== New Features * libhugetlbfs usage can now be restricted to certain binary names * lihugetlbfs now supports static linking * hugeadm uses more human readable directory names for mount points Bug Fixes * hugeadm would segfault if specified user was not in passwd, failure in getpwuid() is now checked Test Suite * Added missing tests to driver script * Added tests for static linking libhugetlbfs 2.11 "Ghost Pepper" ====================================================================== New Features * cpupcstat reports time servicing tlb misses when requested * When supported by the kernel and glibc, MAP_HUGETLB is used for the heap and to back memory returned by get_huge_pages. These features can now be used without mounting hugetlbfs Bug Fixes * tlbmiss_cost.sh supresses oprofile errors * numerous fixes to setup_helper.py * Corrected usage of hugetlbfs_test_feature return value * find_mounts now correctly ignores non-hugetlbfs mount points * When prefaulting pages for get_huge_pages readv was using the fd for the mapping, this caused the prefault to fail on older libc. Now /dev/zero is used for all prefaulting libhugetlbfs 2.10 "Another Hottie" ====================================================================== Bug Fixes * hugeadm now handles pool size deltas properly * Makefile uses ?= to assign PREFIX and EXEDIR to allow for easier build modification libhugetlbfs 2.9 "Something Spicy" ====================================================================== New Features * Add --no-reseve to hugectl to request mmap'd pages are not reserved for kernels newer than 2.6.34 * POWER6 now supported by TLB miss cost estimator * Add --obey-numa-mempol to hugeadm to request static pool pages are allocated following the process NUMA memory policy Test Suite * Fix gethugepagesizes test case libhugetlbfs 2.8 "Scotch Bonnet" ====================================================================== New Features * Add switch to let administrator limit new mount points by size or inodes * cpupcstat now caches the value returned by tlmiss_cost.sh to avoid rerunning the script Bug Fixes * errno values are saved in get_huge_pages * tlbmiss_cost.sh patches calibrator to fix round naming collision * Fixed ALIGN_UP macro for aligning huge page segments * Fix --create-mounts switch in hugeadm * Library and helpers are all linked with -z noexecstack Test Suite * run_tests.py detects valid word sizes libhugetlbfs 2.7 "Adobo" ====================================================================== New Features * When specifying huge page pool sizes with hugeadm, memory sizes can be used as well as the number of huge pages * DEFAULT is now a valid huge page pool for resizing, it will adjust the pool for the default huge page size * tlbmiss_cost.sh in the contrib/ sub directory will estimate the cost in CPU cycles of a TLB miss on the arch where it is run * Add python script which automates huge page pool setup with minimal input required from user Bug Fixes * The --dry-run switch in hugeadm is now obeyed * hugeadm now uses unsigned long long for page resizes to avoid overflow errors * --set-recommended-shmmax no longer overflows if the number of available huge pages is bigger than the address space Test Suite * Updated linkhuge_nofd to override proper functions when testing * run_tests.py can now monitor the pool sizes between tests to help identify accounting errors * Add test for mremap bug on architectures with holes in address space libhugetlbfs 2.6 "Adovada" ====================================================================== New Features * cpupcstat now supports data collection using the perf tool as well as oprofile * --explain reports if min_free_kbytes is too small * add --set-min_free_kbytes to hugeadm Bug Fixes * Admin utils (hugeadm, hugectl, etc) are now built as 64 bit binaries if possible to support adminstration of larger huge page sizes Test Suite * Suppress ld.hugetlbfs warnings during test suite build * Make SPECIAL keyword test cross-compile safe * Test script sets proper rlimits for mlock and stack_grow_into_huge tests * Ensure that all elflink tests are run with both HUGETLB_SHARE=0 and HUGETLB_SHARE=1 libhugetlbfs 2.5 "Self Titled" ====================================================================== New Features * added --add-ramdisk-swap option to hugeadm to use ramdisks as temporary swap space for diskless systems * added --persist option to hugeadm to be used with either --add-*-swap option. Makes swap added stay until removed or the machine is rebooted * added cpupcstat script which uses oprofile to monitor tlb miss rate of a target program Bug Fixes * --add-temp-swap option now takes an optional integer argument that is the size in number of hugepages to make the swap space libhugetlbfs 2.4 "Birdseye" ====================================================================== New Features * added --add-temp-swap option to hugeadm to add a swap file for a pool resize * added --[enable|disable]-zone-moveable options to hugeadm to control /proc/sys/vm/hugepages_treat_as_movable Bug Fixes * Fix pool-pages-max processing by using the proper array for its requests * Move private reservation check out of morecore setup Test Suite * Added regression tests for leaking reserve count due to madvise and fadvise and readahead * Add test for mixed permissions on shm segments * Wrap tests that can hang the machine to fail is kernel is too old * Add -f option to run_tests.py to force running of tests that can hang the machine libhugetlbfs 2.3 "NuMex Sunglo" ====================================================================== New Features * added --force-preload option to hugectl for backing segments with 64kb pages on ppc64 when app was not linked with libhugetlbfs * added --explain swtich to hugadm to give a quick overview of the system wrt huge pages * hugeadm warns if min pool size is being adjusted without sufficient swap space configured * added --hard switch to ask hugeadm to try multiple times to resize a huge page pool * added --create-*-mounts switches to create mount points for hugetlbfs usable by specific users, groups, or globally Bug Fixes * hugeadm will no longer mount a directory multiple times * hugeadm adds all new mount points to /etc/mtab libhugetlbfs 2.2 "Barkers Extra Hot" ====================================================================== New Features * Refactored environment variable parsing to read once and store values * Add --list-mounts and --list-all-mounts options to hugeadm * Rework test suite to run for all avaialbe page sizes * Add --create-mounts for root only, --create-user-mounts, --create-group-mounts, and --create-global-mounts options to hugeadm * Add --share-text option to hugectl Test Suite Fixes * Added wrapper to shm-fork and shm-getraw tests that makes runs on hpage sizes different from default expected failures * Reworked shmoverride tests to handle new env parsing libhugetlbfs 2.1 "NM 64" ====================================================================== New Features * Multiple page size support * Add a more user friendly allocator that handles cache coloring * Add pagesize utility to display supported page sizes * Add hugeadm utility for managing hugepage pools * Add hugectl utility for running programs using hugepages * Add hugeedit utility for marking segments in aligned binaries for huge page backing by default * Deprecated linker linker scripts * gethugepagesize() and getpagesizes() API added to programatically discover supported hugepages * Manual pages for all API functions and utilities * Allow automatic backing of shared memory segments with hugepages * huge page regions will no longer prefault for kernels >= 2.6.27 improving mmap() performance and NUMA layout Bug Fixes * Add missing segment to interp section in linker scripts * Fix free_hugepage_region to handle segments that fell back to small pages * Fix install when lib32 and lib64 resolve to the same place * Install header files so APIs can be used * Fix installation paths to make life easier for package maintainers * Do not export internal symbols unnecessarily * Prefault regions allocated by direct allocation API on kernels older than 2.6.27 * Gracefully fallback to using base pages for text/data when the hugepage pool is too small * Fix handling of HUGETLB_SHARE_PATH environment variable * Relax remapping requirements Test suite Fixes * Added expected failure support * gethugepagesizes override for getting meminfo * Increase debug output for tests that fail * Summarise pass and failure counts libhugetlbfs 2.0 "Sandia Red" ====================================================================== New Features * New scriptless relinking for binutils >= 2.17 * Added direct allocator API for huge pages Bug Fixes * /proc/mounts is parsed line at a time to handle file larger than 4kb * Read-only mappings use MAP_NORESERVE Test suite fixes * Add test for /proc/mounts file larger than 4kb * Fix quota test with private reservations * Output strerror on failure * linkhuge tests are skipped when known to be broken libhugetlbfs 1.3 "Big Jim" ====================================================================== New features * Add HUGETLB_NO_PREFAULT to control prefaulting of huge pages via mlock * Add "[hostname:pid]" to output messages * Setup for handling larger huge page sizes e.g. 16G huge pages * Update for new upstream sysctl * Add support for hugetlbfs_morecore to shrink the heap Bug fixes * Disable heap shrinking by default to avoid bug in glibc malloc * Skip elflink calls in setup_libhugetlbfs for IA64 and sparc64 * Replace gethugepagesize with check_hugepagesize for error checking * Make morecore argument a long to handle larger page sizes Test suite fixes * Check uid/gid in tests where it matters * tests: verify there are enough huge pages * Change tests to read /proc/meminfo * tests: verify that huge page size isn't too big for the test libhugetlbfs 1.2 "Mango Habanero" ====================================================================== New features * Partial segment remapping. This allows non-relinked binaries to try to take advantage of libhugetlbfs' segment remapping code. Large segments are required, especially on Power. This feature is useful for estimating huge page performance, however full relinking will still perform better. * Add extra debugging for binaries that may run out of address space. * Log library version when HUGETLB_VERBOSE is enabled. * Beginning support for ia64 and sparc64. * New test to check handling of misaligned mmap() parameters. Bug fixes * Fix EH_FRAME segment. Fixes some C++ applications. * Rework PLT detection to work better on Power. * Add per-target-arch syscall stubs to the library. These provide reliable error messages from elflink.c if they occur while the program segments are unmapped. * Add proper SONAME to shared libs. * Makefile respects CFLAGS/LDFLAGS/CPPFLAGS environment variables. * Make mlock() failure non-fatal. Test suite fixes * Fix zero_filesize_segment test. * Fix the icache-hygeine testcase for i386 and x86_64. * Fix SEGVs in task-size-overrun test. * Enable stack_grow_into_huge test, previously skipped. * Fix test_addr_huge() for anon pages. libhugetlbfs 1.1 "Carribbean Jerk" ====================================================================== This large release brings several performance improvements Security * Remove the sharing daemon completely and rework the segment sharing code. Permissions on the hugetlbfs mountpoint are used to enforce segment sharing. Bug fixes * Sharing of writable segments is no longer possible, due to address space randomization on x86_64 (although similar issues could happen on any architecture). * minimal_copy detection should work better in this release. Trivial but notable changes * Testcase updates libhugetlbfs 1.0.1 "Spicy Garlic" ====================================================================== This small maintenance release brings a security fix, a few minor bug fixes, plus some documentation and error message updates. Security * A section on security has been added to the README file * The hugetlbd daemon socket has been moved from /tmp to /var/run. This will require the daemon to be run as root, which was previously just a recommendation. Bug fixes * Reduce reserved huge pages needed for application start-up * PPC linker script fixes Trivial but notable changes * Always install linker scripts for all targets * Error message updates * Add documentation on HUGETLB_DEBUG * Testcase updates libhugetlbfs 1.0 ====================================================================== * First stable release HOWTO000066600000075614150501210120005373 0ustar00libhugetlbfs HOWTO ================== Author: David Gibson , Adam Litke , and others Last updated: December 07, 2011 Introduction ============ In Linux(TM), access to hugepages is provided through a virtual file system, "hugetlbfs". The libhugetlbfs library interface works with hugetlbfs to provide more convenient specific application-level services. In particular libhugetlbfs has three main functions: * library functions libhugetlbfs provides functions that allow an applications to explicitly allocate and use hugepages more easily they could by directly accessing the hugetblfs filesystem * hugepage malloc() libhugetlbfs can be used to make an existing application use hugepages for all its malloc() calls. This works on an existing (dynamically linked) application binary without modification. * hugepage text/data/BSS libhugetlbfs, in conjunction with included special linker scripts can be used to make an application which will store its executable text, its initialized data or BSS, or all of the above in hugepages. This requires relinking an application, but does not require source-level modifications. This HOWTO explains how to use the libhugetlbfs library. It is for application developers or system administrators who wish to use any of the above functions. The libhugetlbfs library is a focal point to simplify and standardise the use of the kernel API. Prerequisites ============= Hardware prerequisites ---------------------- You will need a CPU with some sort of hugepage support, which is handled by your kernel. This covers recent x86, AMD64, 64-bit PowerPC(R) (POWER4, PPC970 and later), and IBM System z CPUs. Currently, only x86, AMD64 and PowerPC are fully supported by libhugetlbfs. IA64 and Sparc64 have a working malloc, and SH64 should also but it has not been tested. IA64, Sparc64, and SH64 do not support segment remapping at this time. IBM System z supports malloc and also segment remapping with --hugetlbfs-align. Kernel prerequisites -------------------- To use all the features of libhugetlbfs you will need a 2.6.16 or later kernel. Many things will work with earlier kernels, but they have important bugs and missing features. The later sections of the HOWTO assume a 2.6.16 or later kernel. The kernel must also have hugepages enabled, that is to say the CONFIG_HUGETLB_PAGE and CONFIG_HUGETLBFS options must be switched on. To check if hugetlbfs is enabled, use one of the following methods: * (Preferred) Use "grep hugetlbfs /proc/filesystems" to see if hugetlbfs is a supported file system. * On kernels which support /proc/config.gz (for example SLES10 kernels), you can search for the CONFIG_HUGETLB_PAGE and CONFIG_HUGETLBFS options in /proc/config.gz * Finally, attempt to mount hugetlbfs. If it works, the required hugepage support is enabled. Any kernel which meets the above test (even old ones) should support at least basic libhugetlbfs functions, although old kernels may have serious bugs. The MAP_PRIVATE flag instructs the kernel to return a memory area that is private to the requesting process. To use MAP_PRIVATE mappings, libhugetlbfs's automatic malloc() (morecore) feature, or the hugepage text, data, or BSS features, you will need a kernel with hugepage Copy-on-Write (CoW) support. The 2.6.16 kernel has this. PowerPC note: The malloc()/morecore features will generate warnings if used on PowerPC chips with a kernel where hugepage mappings don't respect the mmap() hint address (the "hint address" is the first parameter to mmap(), when MAP_FIXED is not specified; the kernel is not required to mmap() at this address, but should do so when possible). 2.6.16 and later kernels do honor the hint address. Hugepage malloc()/morecore should still work without this patch, but the size of the hugepage heap will be limited (to around 256M for 32-bit and 1TB for 64-bit). The 2.6.27 kernel introduced support for multiple huge page sizes for systems with the appropriate hardware support. Unless specifically requested, libhugetlbfs will continue to use the default huge page size. Toolchain prerequisites ----------------------- The library uses a number of GNU specific features, so you will need to use both gcc and GNU binutils. For PowerPC and AMD64 systems you will need a "biarch" compiler, which can build both 32-bit and 64-bit binaries. To use hugepage text and data segments, GNU binutils version 2.17 (or later) is recommended. Older versions will work with restricted functionality. Configuration prerequisites --------------------------- Direct access to hugepage pool has been deprecated in favor of the hugeadm utility. This utility can be used for finding the available hugepage pools and adjusting their minimum and maximum sizes depending on kernel support. To list all availabe hugepage pools and their current min and max values: hugeadm --pool-list To set the 2MB pool minimum to 10 pages: hugeadm --pool-pages-min 2MB:10 Note: that the max pool size will be adjusted to keep the same number of overcommit pages available if the kernel support is available when min pages are adjusted To add 15 pages to the maximum for 2MB pages: hugeadm --pool-pages-min 2MB:-5 For more information see man 8 hugeadm The raw kernel interfaces (as described below) are still available. In kernels before 2.6.24, hugepages must be allocated at boot-time via the hugepages= command-line parameter or at run-time via the /proc/sys/vm/nr_hugepages sysctl. If memory is restricted on the system, boot-time allocation is recommended. Hugepages so allocated will be in the static hugepage pool. In kernels starting with 2.6.24, the hugepage pool can grown on-demand. If this feature should be used, /proc/sys/vm/nr_overcommit_hugepages should be set to the maximum size of the hugepage pool. No hugepages need to be allocated via /proc/sys/vm/nr_hugepages or hugepages= in this case. Hugepages so allocated will be in the dynamic hugepage pool. For the running of the libhugetlbfs testsuite (see below), allocating 25 static hugepages is recommended. Due to memory restrictions, the number of hugepages requested may not be allocated if the allocation is attempted at run-time. Users should verify the actual number of hugepages allocated by: hugeadm --pool-list or grep HugePages_Total /proc/meminfo With 25 hugepages allocated, most tests should succeed. However, with smaller hugepages sizes, many more hugepages may be necessary. To use libhugetlbfs features, as well as to run the testsuite, hugetlbfs must be mounted. Each hugetlbfs mount point is associated with a page size. To choose the size, use the pagesize mount option. If this option is omitted, the default huge page size will be used. To mount the default huge page size: mkdir -p /mnt/hugetlbfs mount -t hugetlbfs none /mnt/hugetlbfs To mount 64KB pages (assuming hardware support): mkdir -p /mnt/hugetlbfs-64K mount -t hugetlbfs none -opagesize=64k /mnt/hugetlbfs-64K If hugepages should be available to non-root users, the permissions on the mountpoint need to be set appropriately. Installation ============ 1. Type "make" to build the library This will create "obj32" and/or "obj64" under the top level libhugetlbfs directory, and build, respectively, 32-bit and 64-bit shared and static versions (as applicable) of the library into each directory. This will also build (but not run) the testsuite. On i386 systems, only the 32-bit library will be built. On PowerPC and AMD64 systems, both 32-bit and 64-bit versions will be built (the 32-bit AMD64 version is identical to the i386 version). 2. Run the testsuite with "make check" Running the testsuite is a good idea to ensure that the library is working properly, and is quite quick (under 3 minutes on a 2GHz Apple G5). "make func" will run the just the functionality tests, rather than stress tests (a subset of "make check") which is much quicker. The testsuite contains tests both for the library's features and for the underlying kernel hugepage functionality. NOTE: The testsuite must be run as the root user. WARNING: The testsuite contains testcases explicitly designed to test for a number of hugepage related kernel bugs uncovered during the library's development. Some of these testcases WILL CRASH HARD a kernel without the relevant fixes. 2.6.16 contains all such fixes for all testcases included as of this writing. 3. (Optional) Install to system paths with "make install" This will install the library images to the system lib/lib32/lib64 as appropriate, the helper utilities and the manual pages. By default it will install under /usr/local. To put it somewhere else use PREFIX=/path/to/install on the make command line. For example: make install PREFIX=/opt/hugetlbfs Will install under /opt/hugetlbfs. "make install" will also install the linker scripts and wrapper for ld used for hugepage test/data/BSS (see below for details). Alternatively, you can use the library from the directory in which it was built, using the LD_LIBRARY_PATH environment variable. To only install library with linker scripts, the manual pages or the helper utilities separetly, use the install-libs, install-man and install-bin targets respectively. This can be useful when you with to install the utilities but not override the distribution-supported version of libhugetlbfs for example. Usage ===== Using hugepages for malloc() (morecore) --------------------------------------- This feature allows an existing (dynamically linked) binary executable to use hugepages for all its malloc() calls. To run a program using the automatic hugepage malloc() feature, you must set several environment variables: 1. Set LD_PRELOAD=libhugetlbfs.so This tells the dynamic linker to load the libhugetlbfs shared library, even though the program wasn't originally linked against it. Note: If the program is linked against libhugetlbfs, preloading the library may lead to application crashes. You should skip this step in that case. 2. Set LD_LIBRARY_PATH to the directory containing libhugetlbfs.so This is only necessary if you haven't installed libhugetlbfs.so to a system default path. If you set LD_LIBRARY_PATH, make sure the directory referenced contains the right version of the library (32-bit or 64-bit) as appropriate to the binary you want to run. 3. Set HUGETLB_MORECORE This enables the hugepage malloc() feature, instructing libhugetlbfs to override libc's normal morecore() function with a hugepage version and use it for malloc(). From this point all malloc()s should come from hugepage memory until it runs out. This option can be specified in two ways: To use the default huge page size: HUGETLB_MORECORE=yes To use a specific huge page size: HUGETLB_MORECORE= To use Transparent Huge Pages (THP): HUGETLB_MORECORE=thp Note: This option requires a kernel that supports Transparent Huge Pages Usually it's preferable to set these environment variables on the command line of the program you wish to run, rather than using "export", because you'll only want to enable the hugepage malloc() for particular programs, not everything. Examples: If you've installed libhugetlbfs in the default place (under /usr/local) which is in the system library search path use: $ LD_PRELOAD=libhugetlbfs.so HUGETLB_MORECORE=yes If you have built libhugetlbfs in ~/libhugetlbfs and haven't installed it yet, the following would work for a 64-bit program: $ LD_PRELOAD=libhugetlbfs.so LD_LIBRARY_PATH=~/libhugetlbfs/obj64 \ HUGETLB_MORECORE=yes Under some circumstances, you might want to specify the address where the hugepage heap is located. You can do this by setting the HUGETLB_MORECORE_HEAPBASE environment variable to the heap address in hexadecimal. NOTE: this will not work on PowerPC systems with old kernels which don't respect the hugepage hint address; see Kernel Prerequisites above. Also note that this option is ignored for THP morecore. By default, the hugepage heap begins at roughly the same place a normal page heap would, rounded up by an amount determined by your platform. For 32-bit PowerPC binaries the normal page heap address is rounded-up to a multiple of 256MB (that is, putting it in the next MMU segment); for 64-bit PowerPC binaries the address is rounded-up to a multiple of 1TB. On all other platforms the address is rounded-up to the size of a hugepage. By default, the hugepage heap will be prefaulted by libhugetlbfs to guarantee enough hugepages exist and are reserved for the application (if this was not done, applications could receive a SIGKILL signal if hugepages needed for the heap are used by another application before they are faulted in). This leads to local-node allocations when no memory policy is in place for hugepages. Therefore, it is recommended to use $ numactl --interleave=all to regain some of the performance impact of local-node allocations on large NUMA systems. This can still result in poor performance for those applications which carefully place their threads on particular nodes (such as by using OpenMP). In that case, thread-local allocation is preferred so users should select a memory policy that corresponds to the run-time behavior of the process' CPU usage. Users can specify HUGETLB_NO_PREFAULT to prevent the prefaulting of hugepages and instead rely on run-time faulting of hugepages. NOTE: specifying HUGETLB_NO_PREFAULT on a system where hugepages are available to and used by many process can result in some applications receving SIGKILL, so its use is not recommended in high-availability or production environments. By default, the hugepage heap does not shrink. To enable hugepage heap shrinking, set HUGETLB_MORECORE_SHRINK=yes. NB: We have been seeing some unexpected behavior from glibc's malloc when this is enabled. Using hugepage shared memory ---------------------------- Hugepages are used for shared memory segments if the SHM_HUGETLB flag is set when calling shmget() and the pool is large enough. For hugepage-unaware applications, libhugetlbfs overrides shmget and adds the SHM_HUGETLB if the environment variable HUGETLB_SHM is set to "yes". The steps to use hugepages with applications not linked to libhugetlbfs are similar to morecore except for step 3. 1. Set LD_PRELOAD=libhugetlbfs.so This tells the dynamic linker to load the libhugetlbfs shared library, even though the program wasn't originally linked against it. Note: If the program is linked against libhugetlbfs, preloading the library may lead to application crashes. You should skip this step in that case. 2. Set LD_LIBRARY_PATH to the directory containing libhugetlbfs.so This is only necessary if you haven't installed libhugetlbfs.so to a system default path. If you set LD_LIBRARY_PATH, make sure the directory referenced contains the right version of the library (32-bit or 64-bit) as appropriate to the binary you want to run. 3. Set HUGETLB_SHM=yes The shmget() call is overridden whether the application is linked or the libhugetlbfs library is preloaded. When this environment variable is set, the SHM_HUGETLB flag is added to the call and the size parameter is aligned to back the shared memory segment with huge pages. In the event hugepages cannot be used, small pages will be used instead and a warning will be printed to explain the failure. Note: It is not possible to select any huge page size other than the system default for this option. If the kernel supports multiple huge page sizes, the size used for shared memory can be changed by altering the default huge page size via the default_hugepagesz kernel boot parameter. Using hugepage text, data, or BSS --------------------------------- To use the hugepage text, data, or BSS segments feature, you need to specially link your application. How this is done depends on the version of GNU ld. To support ld versions older than 2.17, libhugetlbfs provides custom linker scripts that must be used to achieve the required binary layout. With version 2.17 or later, the system default linker scripts should be used. To link an application for hugepages, you should use the the ld.hugetlbfs script included with libhugetlbfs in place of your normal linker. Without any special options this will simply invoke GNU ld with the same parameters. When it is invoked with options detailed in the following sections, ld.hugetlbfs will call the system linker with all of the options necessary to link for hugepages. If a custom linker script is required, it will also be selected. If you installed ld.hugetlbfs using "make install", or if you run it from the place where you built libhugetlbfs, it should automatically be able to find the libhugetlbfs linker scripts. Otherwise you may need to explicitly instruct it where to find the scripts with the option: --hugetlbfs-script-path=/path/to/scripts (The linker scripts are in the ldscripts/ subdirectory of the libhugetlbfs source tree). Linking the application with binutils-2.17 or later: ---------------------------------------------------- This method will use the system default linker scripts. Only one linker option is required to prepare the application for hugepages: --hugetlbfs-align will instruct ld.hugetlbfs to call GNU ld with two options that increase the alignment of the resulting binary. For reference, the options passed to ld are: -z common-page-size= and -z max-page-size= Linking the application with binutils-2.16 or older: ---------------------------------------------------- To link a program with a custom linker script, one of the following linker options should be specified: --hugetlbfs-link=B will link the application to store BSS data (only) into hugepages --hugetlbfs-link=BDT will link the application to store text, initialized data and BSS data into hugepages. These are the only two available options when using custom linker scripts. A note about the custom libhugetlbfs linker scripts: ---------------------------------------------------- Linker scripts are usually distributed with GNU binutils and they may contain a partial implementation of new linker features. As binutils evolves, the linker scripts supplied with previous versions become obsolete and are upgraded. Libhugetlbfs distributes one set of linker scripts that must work across several Linux distributions and binutils versions. This has worked well for some time but binutils-2.17 (including some late 2.16 builds) have made changes that are impossible to accomodate without breaking the libhugetlbfs linker scripts for older versions of binutils. This is why the linker scripts (and the --hugetlbfs-link ld.hugetlbfs option) have been deprecated for binutils >= 2.17 configurations. If you are using a late 2.16 binutils version (such as 2.16.91) and are experiencing problems with huge page text, data, and bss, you can check binutils for the incompatibility with the following command: ld --verbose | grep SPECIAL If any matches are returned, then the libhugetlbfs linker scripts may not work correctly. In this case you should upgrade to binutils >= 2.17 and use the --hugetlbfs-align linking method. Linking via gcc: ---------------- In many cases it's normal to link an application by invoking gcc, which will then invoke the linker with appropriate options, rather than invoking ld directly. In such cases it's usually best to convince gcc to invoke the ld.hugetlbfs script instead of the system linker, rather than modifying your build procedure to invoke the ld.hugetlbfs directly; the compilers may often add special libraries or other linker options which can be fiddly to reproduce by hand. To make this easier, 'make install' will install ld.hugetlbfs into $PREFIX/share/libhugetlbfs and create an 'ld' symlink to it. Then with gcc, you invoke it as a linker with two options: -B $PREFIX/share/libhugetlbfs This option tells gcc to look in a non-standard location for the linker, thus finding our script rather than the normal linker. This can optionally be set in the CFLAGS environment variable. -Wl,--hugetlbfs-align OR -Wl,--hugetlbfs-link=B OR -Wl,--hugetlbfs-link=BDT This option instructs gcc to pass the option after the comma down to the linker, thus invoking the special behaviour of the ld.hugetblfs script. This can optionally be set in the LDFLAGS environment variable. If you use a compiler other than gcc, you will need to consult its documentation to see how to convince it to invoke ld.hugetlbfs in place of the system linker. Running the application: ------------------------ The specially-linked application needs the libhugetlbfs library, so you might need to set the LD_LIBRARY_PATH environment variable so the application can locate libhugetlbfs.so. Depending on the method used to link the application, the HUGETLB_ELFMAP environment variable can be used to control how hugepages will be used. When using --hugetlbfs-link: ---------------------------- The custom linker script determines which segments may be remapped into hugepages and this remapping will occur by default. The following setting will disable remapping entirely: HUGETLB_ELFMAP=no When using --hugetlbfs-align: ----------------------------- This method of linking an application permits greater flexibility at runtime. Using HUGETLB_ELFMAP, it is possible to control which program segments are placed in hugepages. The following four settings will cause the indicated segments to be placed in hugepages: HUGETLB_ELFMAP=R Read-only segments (text) HUGETLB_ELFMAP=W Writable segments (data/BSS) HUGETLB_ELFMAP=RW All segments (text/data/BSS) HUGETLB_ELFMAP=no No segments It is possible to select specific huge page sizes for read-only and writable segments by using the following advanced syntax: HUGETLB_ELFMAP=[R[=]:[W[=]] For example: Place read-only segments into 64k pages and writable into 16M pages HUGETLB_ELFMAP=R=64k:W=16M Use the default for read-only segments, 1G pages for writable segments HUGETLB_ELFMAP=R:W=1G Use 16M pages for writable segments only HUGETLB_ELFMAP=W=16M Default remapping behavior: --------------------------- If --hugetlbfs-link was used to link an application, the chosen remapping mode is saved in the binary and becomes the default behavior. Setting HUGETLB_ELFMAP=no will disable all remapping and is the only way to modify the default behavior. For applications linked with --hugetlbfs-align, the default behavior is to not remap any segments into huge pages. To set or display the default remapping mode for a binary, the included hugeedit command can be used: hugeedit [options] target-executable options: --text,--data Remap the specified segment into huge pages by default --disable Do not remap any segments by default When target-executable is the only argument, hugeedit will display the default remapping mode without making any modifications. When a binary is remapped according to its default remapping policy, the system default huge page size will be used. Environment variables: ---------------------- There are a number of private environment variables which can affect libhugetlbfs: HUGETLB_DEFAULT_PAGE_SIZE Override the system default huge page size for all uses except hugetlb-backed shared memory HUGETLB_RESTRICT_EXE By default, libhugetlbfs will act on any program that it is loaded with, either via LD_PRELOAD or by explicitly linking with -lhugetlbfs. There are situations in which it is desirable to restrict libhugetlbfs' actions to specific programs. For example, some ISV applications are wrapped in a series of scripts that invoke bash, python, and/or perl. It is more convenient to set the environment variables related to libhugetlbfs before invoking the wrapper scripts, yet this has the unintended and undesirable consequence of causing the script interpreters to use and consume hugepages. There is no obvious benefit to causing the script interpreters to use hugepages, and there is a clear disadvantage: fewer hugepages are available to the actual application. To address this scenario, set HUGETLB_RESTRICT_EXE to a colon-separated list of programs to which the other libhugetlbfs environment variables should apply. (If not set, libhugetlbfs will attempt to apply the requested actions to all programs.) For example, HUGETLB_RESTRICT_EXE="hpcc:long_hpcc" will restrict libhugetlbfs' actions to programs named /home/fred/hpcc and /bench/long_hpcc but not /usr/hpcc_no. HUGETLB_ELFMAP Control or disable segment remapping (see above) HUGETLB_MINIMAL_COPY If equal to "no", the entire segment will be copied; otherwise, only the necessary parts will be, which can be much more efficient (default) HUGETLB_FORCE_ELFMAP Explained in "Partial segment remapping" HUGETLB_MORECORE HUGETLB_MORECORE_HEAPBASE HUGETLB_NO_PREFAULT Explained in "Using hugepages for malloc() (morecore)" HUGETLB_VERBOSE Specify the verbosity level of debugging output from 1 to 99 (default is 1) HUGETLB_PATH Specify the path to the hugetlbfs mount point HUGETLB_SHARE Explained in "Sharing remapped segments" HUGETLB_DEBUG Set to 1 if an application segfaults. Gives very detailed output and runs extra diagnostics. Sharing remapped segments: -------------------------- By default, when libhugetlbfs uses anonymous, unlinked hugetlbfs files to store remapped program segment data. This means that if the same program is started multiple times using hugepage segments, multiple huge pages will be used to store the same program data. The reduce this wastage, libugetlbfs can be instructed to allow sharing segments between multiple invocations of a program. To do this, you must set the HUGETLB_SHARE variable must be set for all the processes in question. This variable has two possible values: anything but 1: the default, indicates no segments should be shared 1: indicates that read-only segments (i.e. the program text, in most cases) should be shared, read-write segments (data and bss) will not be shared. If the HUGETLB_MINIMAL_COPY variable is set for any program using shared segments, it must be set to the same value for all invocations of that program. Segment sharing is implemented by creating persistent files in a hugetlbfs containing the necessary segment data. By default, these files are stored in a subdirectory of the first located hugetlbfs filesystem, named 'elflink-uid-XXX' where XXX is the uid of the process using sharing. This directory must be owned by the uid in question, and have mode 0700. If it doesn't exist, libhugetlbfs will create it automatically. This means that (by default) separate invocations of the same program by different users will not share huge pages. The location for storing the hugetlbfs page files can be changed by setting the HUGETLB_SHARE_PATH environment variable. If set, this variable must contain the path of an accessible, already created directory located in a hugetlbfs filesystem. The owner and mode of this directory are not checked, so this method can be used to allow processes of multiple uids to share huge pages. IMPORTANT SECURITY NOTE: any process sharing hugepages can insert arbitrary executable code into any other process sharing hugepages in the same directory. Therefore, when using HUGETLB_SHARE_PATH, the directory created *must* allow access only to a set of uids who are mutually trusted. The files created in hugetlbfs for sharing are persistent, and must be manually deleted to free the hugepages in question. Future versions of libhugetlbfs should include tools and scripts to automate this cleanup. Partial segment remapping ------------------------- libhugetlbfs has limited support for remapping a normal, non-relinked binary's data, text and BSS into hugepages. To enable this feature, HUGETLB_FORCE_ELFMAP must be set to "yes". Partial segment remapping is not guaranteed to work. Most importantly, a binary's segments must be large enough even when not relinked by libhugetlbfs: architecture address minimum segment size ------------ ------- -------------------- i386, x86_64 all hugepage size ppc32 all 256M ppc64 0-4G 256M ppc64 4G-1T 1020G ppc64 1T+ 1T The raw size, though, is not sufficient to indicate if the code will succeed, due to alignment. Since the binary is not relinked, however, this is relatively straightforward to 'test and see'. NOTE: You must use LD_PRELOAD to load libhugetlbfs.so when using partial remapping. Examples ======== Example 1: Application Developer --------------------------------- To have a program use hugepages, complete the following steps: 1. Make sure you are working with kernel 2.6.16 or greater. 2. Modify the build procedure so your application is linked against libhugetlbfs. For the remapping, you link against the library with the appropriate linker script (if necessary or desired). Linking against the library should result in transparent usage of hugepages. Example 2: End Users and System Administrators ----------------------------------------------- To have an application use libhugetlbfs, complete the following steps: 1. Make sure you are using kernel 2.6.16. 2. Make sure the library is in the path, which you can set with the LD_LIBRARY_PATH environment variable. You might need to set other environment variables, including LD_PRELOAD as described above. Troubleshooting =============== The library has a certain amount of debugging code built in, which can be controlled with the environment variable HUGETLB_VERBOSE. By default the debug level is "1" which means the library will only print relatively serious error messages. Setting HUGETLB_VERBOSE=2 or higher will enable more debug messages (at present 2 is the highest debug level, but that may change). Setting HUGETLB_VERBOSE=0 will silence the library completely, even in the case of errors - the only exception is in cases where the library has to abort(), which can happen if something goes wrong in the middle of unmapping and remapping segments for the text/data/bss feature. If an application fails to run, set the environment variable HUGETLB_DEBUG to 1. This causes additional diagnostics to be run. This information should be included when sending bug reports to the libhugetlbfs team. Specific Scenarios: ------------------- ISSUE: When using the --hugetlbfs-align or -zmax-page-size link options, the linker complains about truncated relocations and the build fails. TRY: Compile the program with the --relax linker option. Either add -Wl,--relax to CFLAGS or --relax to LDFLAGS. ISSUE: When using the xB linker script with a 32 bit binary on an x86 host with NX support enabled, the binary segfaults. TRY: Recompiling with the --hugetlbfs-align options and use the new relinking method or booting your kernel with noexec32=off. Trademarks ========== This work represents the view of the author and does not necessarily represent the view of IBM. PowerPC is a registered trademark of International Business Machines Corporation in the United States, other countries, or both. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. LGPL-2.1000066600000063637150501210120005531 0ustar00 GNU LESSER GENERAL PUBLIC LICENSE Version 2.1, February 1999 Copyright (C) 1991, 1999 Free Software Foundation, Inc. 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. [This is the first released version of the Lesser GPL. It also counts as the successor of the GNU Library Public License, version 2, hence the version number 2.1.] Preamble The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public Licenses are intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. This license, the Lesser General Public License, applies to some specially designated software packages--typically libraries--of the Free Software Foundation and other authors who decide to use it. You can use it too, but we suggest you first think carefully about whether this license or the ordinary General Public License is the better strategy to use in any particular case, based on the explanations below. When we speak of free software, we are referring to freedom of use, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish); that you receive source code or can get it if you want it; that you can change the software and use pieces of it in new free programs; and that you are informed that you can do these things. To protect your rights, we need to make restrictions that forbid distributors to deny you these rights or to ask you to surrender these rights. These restrictions translate to certain responsibilities for you if you distribute copies of the library or if you modify it. For example, if you distribute copies of the library, whether gratis or for a fee, you must give the recipients all the rights that we gave you. You must make sure that they, too, receive or can get the source code. If you link other code with the library, you must provide complete object files to the recipients, so that they can relink them with the library after making changes to the library and recompiling it. And you must show them these terms so they know their rights. We protect your rights with a two-step method: (1) we copyright the library, and (2) we offer you this license, which gives you legal permission to copy, distribute and/or modify the library. To protect each distributor, we want to make it very clear that there is no warranty for the free library. Also, if the library is modified by someone else and passed on, the recipients should know that what they have is not the original version, so that the original author's reputation will not be affected by problems that might be introduced by others. Finally, software patents pose a constant threat to the existence of any free program. We wish to make sure that a company cannot effectively restrict the users of a free program by obtaining a restrictive license from a patent holder. Therefore, we insist that any patent license obtained for a version of the library must be consistent with the full freedom of use specified in this license. Most GNU software, including some libraries, is covered by the ordinary GNU General Public License. This license, the GNU Lesser General Public License, applies to certain designated libraries, and is quite different from the ordinary General Public License. We use this license for certain libraries in order to permit linking those libraries into non-free programs. When a program is linked with a library, whether statically or using a shared library, the combination of the two is legally speaking a combined work, a derivative of the original library. The ordinary General Public License therefore permits such linking only if the entire combination fits its criteria of freedom. The Lesser General Public License permits more lax criteria for linking other code with the library. We call this license the "Lesser" General Public License because it does Less to protect the user's freedom than the ordinary General Public License. It also provides other free software developers Less of an advantage over competing non-free programs. These disadvantages are the reason we use the ordinary General Public License for many libraries. However, the Lesser license provides advantages in certain special circumstances. For example, on rare occasions, there may be a special need to encourage the widest possible use of a certain library, so that it becomes a de-facto standard. To achieve this, non-free programs must be allowed to use the library. A more frequent case is that a free library does the same job as widely used non-free libraries. In this case, there is little to gain by limiting the free library to free software only, so we use the Lesser General Public License. In other cases, permission to use a particular library in non-free programs enables a greater number of people to use a large body of free software. For example, permission to use the GNU C Library in non-free programs enables many more people to use the whole GNU operating system, as well as its variant, the GNU/Linux operating system. Although the Lesser General Public License is Less protective of the users' freedom, it does ensure that the user of a program that is linked with the Library has the freedom and the wherewithal to run that program using a modified version of the Library. The precise terms and conditions for copying, distribution and modification follow. Pay close attention to the difference between a "work based on the library" and a "work that uses the library". The former contains code derived from the library, whereas the latter must be combined with the library in order to run. GNU LESSER GENERAL PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 0. This License Agreement applies to any software library or other program which contains a notice placed by the copyright holder or other authorized party saying it may be distributed under the terms of this Lesser General Public License (also called "this License"). Each licensee is addressed as "you". A "library" means a collection of software functions and/or data prepared so as to be conveniently linked with application programs (which use some of those functions and data) to form executables. The "Library", below, refers to any such software library or work which has been distributed under these terms. A "work based on the Library" means either the Library or any derivative work under copyright law: that is to say, a work containing the Library or a portion of it, either verbatim or with modifications and/or translated straightforwardly into another language. (Hereinafter, translation is included without limitation in the term "modification".) "Source code" for a work means the preferred form of the work for making modifications to it. For a library, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the library. Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running a program using the Library is not restricted, and output from such a program is covered only if its contents constitute a work based on the Library (independent of the use of the Library in a tool for writing it). Whether that is true depends on what the Library does and what the program that uses the Library does. 1. You may copy and distribute verbatim copies of the Library's complete source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and distribute a copy of this License along with the Library. You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee. 2. You may modify your copy or copies of the Library or any portion of it, thus forming a work based on the Library, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions: a) The modified work must itself be a software library. b) You must cause the files modified to carry prominent notices stating that you changed the files and the date of any change. c) You must cause the whole of the work to be licensed at no charge to all third parties under the terms of this License. d) If a facility in the modified Library refers to a function or a table of data to be supplied by an application program that uses the facility, other than as an argument passed when the facility is invoked, then you must make a good faith effort to ensure that, in the event an application does not supply such function or table, the facility still operates, and performs whatever part of its purpose remains meaningful. (For example, a function in a library to compute square roots has a purpose that is entirely well-defined independent of the application. Therefore, Subsection 2d requires that any application-supplied function or table used by this function must be optional: if the application does not supply it, the square root function must still compute square roots.) These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Library, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Library, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it. Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Library. In addition, mere aggregation of another work not based on the Library with the Library (or with a work based on the Library) on a volume of a storage or distribution medium does not bring the other work under the scope of this License. 3. You may opt to apply the terms of the ordinary GNU General Public License instead of this License to a given copy of the Library. To do this, you must alter all the notices that refer to this License, so that they refer to the ordinary GNU General Public License, version 2, instead of to this License. (If a newer version than version 2 of the ordinary GNU General Public License has appeared, then you can specify that version instead if you wish.) Do not make any other change in these notices. Once this change is made in a given copy, it is irreversible for that copy, so the ordinary GNU General Public License applies to all subsequent copies and derivative works made from that copy. This option is useful when you wish to copy part of the code of the Library into a program that is not a library. 4. You may copy and distribute the Library (or a portion or derivative of it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange. If distribution of object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place satisfies the requirement to distribute the source code, even though third parties are not compelled to copy the source along with the object code. 5. A program that contains no derivative of any portion of the Library, but is designed to work with the Library by being compiled or linked with it, is called a "work that uses the Library". Such a work, in isolation, is not a derivative work of the Library, and therefore falls outside the scope of this License. However, linking a "work that uses the Library" with the Library creates an executable that is a derivative of the Library (because it contains portions of the Library), rather than a "work that uses the library". The executable is therefore covered by this License. Section 6 states terms for distribution of such executables. When a "work that uses the Library" uses material from a header file that is part of the Library, the object code for the work may be a derivative work of the Library even though the source code is not. Whether this is true is especially significant if the work can be linked without the Library, or if the work is itself a library. The threshold for this to be true is not precisely defined by law. If such an object file uses only numerical parameters, data structure layouts and accessors, and small macros and small inline functions (ten lines or less in length), then the use of the object file is unrestricted, regardless of whether it is legally a derivative work. (Executables containing this object code plus portions of the Library will still fall under Section 6.) Otherwise, if the work is a derivative of the Library, you may distribute the object code for the work under the terms of Section 6. Any executables containing that work also fall under Section 6, whether or not they are linked directly with the Library itself. 6. As an exception to the Sections above, you may also combine or link a "work that uses the Library" with the Library to produce a work containing portions of the Library, and distribute that work under terms of your choice, provided that the terms permit modification of the work for the customer's own use and reverse engineering for debugging such modifications. You must give prominent notice with each copy of the work that the Library is used in it and that the Library and its use are covered by this License. You must supply a copy of this License. If the work during execution displays copyright notices, you must include the copyright notice for the Library among them, as well as a reference directing the user to the copy of this License. Also, you must do one of these things: a) Accompany the work with the complete corresponding machine-readable source code for the Library including whatever changes were used in the work (which must be distributed under Sections 1 and 2 above); and, if the work is an executable linked with the Library, with the complete machine-readable "work that uses the Library", as object code and/or source code, so that the user can modify the Library and then relink to produce a modified executable containing the modified Library. (It is understood that the user who changes the contents of definitions files in the Library will not necessarily be able to recompile the application to use the modified definitions.) b) Use a suitable shared library mechanism for linking with the Library. A suitable mechanism is one that (1) uses at run time a copy of the library already present on the user's computer system, rather than copying library functions into the executable, and (2) will operate properly with a modified version of the library, if the user installs one, as long as the modified version is interface-compatible with the version that the work was made with. c) Accompany the work with a written offer, valid for at least three years, to give the same user the materials specified in Subsection 6a, above, for a charge no more than the cost of performing this distribution. d) If distribution of the work is made by offering access to copy from a designated place, offer equivalent access to copy the above specified materials from the same place. e) Verify that the user has already received a copy of these materials or that you have already sent this user a copy. For an executable, the required form of the "work that uses the Library" must include any data and utility programs needed for reproducing the executable from it. However, as a special exception, the materials to be distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable. It may happen that this requirement contradicts the license restrictions of other proprietary libraries that do not normally accompany the operating system. Such a contradiction means you cannot use both them and the Library together in an executable that you distribute. 7. You may place library facilities that are a work based on the Library side-by-side in a single library together with other library facilities not covered by this License, and distribute such a combined library, provided that the separate distribution of the work based on the Library and of the other library facilities is otherwise permitted, and provided that you do these two things: a) Accompany the combined library with a copy of the same work based on the Library, uncombined with any other library facilities. This must be distributed under the terms of the Sections above. b) Give prominent notice with the combined library of the fact that part of it is a work based on the Library, and explaining where to find the accompanying uncombined form of the same work. 8. You may not copy, modify, sublicense, link with, or distribute the Library except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense, link with, or distribute the Library is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance. 9. You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Library or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Library (or any work based on the Library), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Library or works based on it. 10. Each time you redistribute the Library (or any work based on the Library), the recipient automatically receives a license from the original licensor to copy, distribute, link with or modify the Library subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties with this License. 11. If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Library at all. For example, if a patent license would not permit royalty-free redistribution of the Library by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Library. If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply, and the section as a whole is intended to apply in other circumstances. It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice. This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License. 12. If the distribution and/or use of the Library is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Library under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License. 13. The Free Software Foundation may publish revised and/or new versions of the Lesser General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Library specifies a version number of this License which applies to it and "any later version", you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Library does not specify a license version number, you may choose any version ever published by the Free Software Foundation. 14. If you wish to incorporate parts of the Library into other free programs whose distribution conditions are incompatible with these, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally. NO WARRANTY 15. BECAUSE THE LIBRARY IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE LIBRARY, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE LIBRARY "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE LIBRARY IS WITH YOU. SHOULD THE LIBRARY PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 16. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE LIBRARY AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE LIBRARY (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE LIBRARY TO OPERATE WITH ANY OTHER SOFTWARE), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Libraries If you develop a new library, and you want it to be of the greatest possible use to the public, we recommend making it free software that everyone can redistribute and change. You can do so by permitting redistribution under these terms (or, alternatively, under the terms of the ordinary General Public License). To apply these terms, attach the following notices to the library. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This library is free software; you can redistribute it and/or modify it under the terms of the GNU Lesser General Public License as published by the Free Software Foundation; either version 2.1 of the License, or (at your option) any later version. This library is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details. You should have received a copy of the GNU Lesser General Public License along with this library; if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA Also add information on how to contact you by electronic and paper mail. You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the library, if necessary. Here is a sample; alter the names: Yoyodyne, Inc., hereby disclaims all copyright interest in the library `Frob' (a library for tweaking knobs) written by James Random Hacker. , 1 April 1990 Ty Coon, President of Vice That's all there is to it!