resource – System resource management¶
|Purpose:||Manage the system resource limits for a Unix program.|
The functions in resource probe the current system resources consumed by a process, and place limits on them to control how much load a program can impose on a system.
Use getrusage() to probe the resources used by the current process and/or its children. The return value is a data structure containing several resource metrics based on the current state of the system.
Not all of the resource values gathered are displayed here. Refer to the stdlib docs for a more complete list.
import resource import time usage = resource.getrusage(resource.RUSAGE_SELF) for name, desc in [ ('ru_utime', 'User time'), ('ru_stime', 'System time'), ('ru_maxrss', 'Max. Resident Set Size'), ('ru_ixrss', 'Shared Memory Size'), ('ru_idrss', 'Unshared Memory Size'), ('ru_isrss', 'Stack Size'), ('ru_inblock', 'Block inputs'), ('ru_oublock', 'Block outputs'), ]: print '%-25s (%-10s) = %s' % (desc, name, getattr(usage, name))
Because the test program is extremely simple, it does not use very many resources:
$ python resource_getrusage.py User time (ru_utime ) = 0.010192 System time (ru_stime ) = 0.005743 Max. Resident Set Size (ru_maxrss ) = 3891200 Shared Memory Size (ru_ixrss ) = 0 Unshared Memory Size (ru_idrss ) = 0 Stack Size (ru_isrss ) = 0 Block inputs (ru_inblock) = 0 Block outputs (ru_oublock) = 0
Separate from the current actual usage, it is possible to check the limits imposed on the application, and then change them.
import resource for name, desc in [ ('RLIMIT_CORE', 'core file size'), ('RLIMIT_CPU', 'CPU time'), ('RLIMIT_FSIZE', 'file size'), ('RLIMIT_DATA', 'heap size'), ('RLIMIT_STACK', 'stack size'), ('RLIMIT_RSS', 'resident set size'), ('RLIMIT_NPROC', 'number of processes'), ('RLIMIT_NOFILE', 'number of open files'), ('RLIMIT_MEMLOCK', 'lockable memory address'), ]: limit_num = getattr(resource, name) soft, hard = resource.getrlimit(limit_num) print 'Maximum %-25s (%-15s) : %20s %20s' % (desc, name, soft, hard)
The return value for each limit is a tuple containing the soft limit imposed by the current configuration and the hard limit imposed by the operating system.
$ python resource_getrlimit.py Maximum core file size (RLIMIT_CORE ) : 0 9223372036854775807 Maximum CPU time (RLIMIT_CPU ) : 9223372036854775807 9223372036854775807 Maximum file size (RLIMIT_FSIZE ) : 9223372036854775807 9223372036854775807 Maximum heap size (RLIMIT_DATA ) : 9223372036854775807 9223372036854775807 Maximum stack size (RLIMIT_STACK ) : 8388608 67104768 Maximum resident set size (RLIMIT_RSS ) : 9223372036854775807 9223372036854775807 Maximum number of processes (RLIMIT_NPROC ) : 709 1064 Maximum number of open files (RLIMIT_NOFILE ) : 2560 9223372036854775807 Maximum lockable memory address (RLIMIT_MEMLOCK ) : 9223372036854775807 9223372036854775807
The limits can be changed with setrlimit(). For example, to control the number of files a process can open the RLIMIT_NOFILE value can be set to use a smaller soft limit value.
import resource import os soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE) print 'Soft limit starts as :', soft resource.setrlimit(resource.RLIMIT_NOFILE, (4, hard)) soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE) print 'Soft limit changed to :', soft random = open('/dev/random', 'r') print 'random has fd =', random.fileno() try: null = open('/dev/null', 'w') except IOError, err: print err else: print 'null has fd =', null.fileno()
$ python resource_setrlimit_nofile.py Soft limit starts as : 2560 Soft limit changed to : 4 random has fd = 3 [Errno 24] Too many open files: '/dev/null'
It can also be useful to limit the amount of CPU time a process should consume, to avoid eating up too much time. When the process runs past the allotted amount of time, it it sent a SIGXCPU signal.
import resource import sys import signal import time # Set up a signal handler to notify us # when we run out of time. def time_expired(n, stack): print 'EXPIRED :', time.ctime() raise SystemExit('(time ran out)') signal.signal(signal.SIGXCPU, time_expired) # Adjust the CPU time limit soft, hard = resource.getrlimit(resource.RLIMIT_CPU) print 'Soft limit starts as :', soft resource.setrlimit(resource.RLIMIT_CPU, (1, hard)) soft, hard = resource.getrlimit(resource.RLIMIT_CPU) print 'Soft limit changed to :', soft print # Consume some CPU time in a pointless exercise print 'Starting:', time.ctime() for i in range(200000): for i in range(200000): v = i * i # We should never make it this far print 'Exiting :', time.ctime()
Normally the signal handler should flush all open files and close them, but in this case it just prints a message and exits.
$ python resource_setrlimit_cpu.py Soft limit starts as : 9223372036854775807 Soft limit changed to : 1 Starting: Thu Feb 21 06:36:32 2013 EXPIRED : Thu Feb 21 06:36:33 2013 (time ran out)