resource – System resource management¶
| Purpose: | Manage the system resource limits for a Unix program. |
|---|---|
| Available In: | 1.5.2 |
The functions in resource probe the current system resources consumed by a process, and place limits on them to control how much load a program can impose on a system.
Current Usage¶
Use getrusage() to probe the resources used by the current process and/or its children. The return value is a data structure containing several resource metrics based on the current state of the system.
Note
Not all of the resource values gathered are displayed here. Refer to the stdlib docs for a more complete list.
import resource
import time
usage = resource.getrusage(resource.RUSAGE_SELF)
for name, desc in [
('ru_utime', 'User time'),
('ru_stime', 'System time'),
('ru_maxrss', 'Max. Resident Set Size'),
('ru_ixrss', 'Shared Memory Size'),
('ru_idrss', 'Unshared Memory Size'),
('ru_isrss', 'Stack Size'),
('ru_inblock', 'Block inputs'),
('ru_oublock', 'Block outputs'),
]:
print '%-25s (%-10s) = %s' % (desc, name, getattr(usage, name))
Because the test program is extremely simple, it does not use very many resources:
$ python resource_getrusage.py
User time (ru_utime ) = 0.010192
System time (ru_stime ) = 0.005743
Max. Resident Set Size (ru_maxrss ) = 3891200
Shared Memory Size (ru_ixrss ) = 0
Unshared Memory Size (ru_idrss ) = 0
Stack Size (ru_isrss ) = 0
Block inputs (ru_inblock) = 0
Block outputs (ru_oublock) = 0
Resource Limits¶
Separate from the current actual usage, it is possible to check the limits imposed on the application, and then change them.
import resource
for name, desc in [
('RLIMIT_CORE', 'core file size'),
('RLIMIT_CPU', 'CPU time'),
('RLIMIT_FSIZE', 'file size'),
('RLIMIT_DATA', 'heap size'),
('RLIMIT_STACK', 'stack size'),
('RLIMIT_RSS', 'resident set size'),
('RLIMIT_NPROC', 'number of processes'),
('RLIMIT_NOFILE', 'number of open files'),
('RLIMIT_MEMLOCK', 'lockable memory address'),
]:
limit_num = getattr(resource, name)
soft, hard = resource.getrlimit(limit_num)
print 'Maximum %-25s (%-15s) : %20s %20s' % (desc, name, soft, hard)
The return value for each limit is a tuple containing the soft limit imposed by the current configuration and the hard limit imposed by the operating system.
$ python resource_getrlimit.py
Maximum core file size (RLIMIT_CORE ) : 0 9223372036854775807
Maximum CPU time (RLIMIT_CPU ) : 9223372036854775807 9223372036854775807
Maximum file size (RLIMIT_FSIZE ) : 9223372036854775807 9223372036854775807
Maximum heap size (RLIMIT_DATA ) : 9223372036854775807 9223372036854775807
Maximum stack size (RLIMIT_STACK ) : 8388608 67104768
Maximum resident set size (RLIMIT_RSS ) : 9223372036854775807 9223372036854775807
Maximum number of processes (RLIMIT_NPROC ) : 709 1064
Maximum number of open files (RLIMIT_NOFILE ) : 2560 9223372036854775807
Maximum lockable memory address (RLIMIT_MEMLOCK ) : 9223372036854775807 9223372036854775807
The limits can be changed with setrlimit(). For example, to control the number of files a process can open the RLIMIT_NOFILE value can be set to use a smaller soft limit value.
import resource
import os
soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE)
print 'Soft limit starts as :', soft
resource.setrlimit(resource.RLIMIT_NOFILE, (4, hard))
soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE)
print 'Soft limit changed to :', soft
random = open('/dev/random', 'r')
print 'random has fd =', random.fileno()
try:
null = open('/dev/null', 'w')
except IOError, err:
print err
else:
print 'null has fd =', null.fileno()
$ python resource_setrlimit_nofile.py
Soft limit starts as : 2560
Soft limit changed to : 4
random has fd = 3
[Errno 24] Too many open files: '/dev/null'
It can also be useful to limit the amount of CPU time a process should consume, to avoid eating up too much time. When the process runs past the allotted amount of time, it it sent a SIGXCPU signal.
import resource
import sys
import signal
import time
# Set up a signal handler to notify us
# when we run out of time.
def time_expired(n, stack):
print 'EXPIRED :', time.ctime()
raise SystemExit('(time ran out)')
signal.signal(signal.SIGXCPU, time_expired)
# Adjust the CPU time limit
soft, hard = resource.getrlimit(resource.RLIMIT_CPU)
print 'Soft limit starts as :', soft
resource.setrlimit(resource.RLIMIT_CPU, (1, hard))
soft, hard = resource.getrlimit(resource.RLIMIT_CPU)
print 'Soft limit changed to :', soft
print
# Consume some CPU time in a pointless exercise
print 'Starting:', time.ctime()
for i in range(200000):
for i in range(200000):
v = i * i
# We should never make it this far
print 'Exiting :', time.ctime()
Normally the signal handler should flush all open files and close them, but in this case it just prints a message and exits.
$ python resource_setrlimit_cpu.py
Soft limit starts as : 9223372036854775807
Soft limit changed to : 1
Starting: Thu Feb 21 06:36:32 2013
EXPIRED : Thu Feb 21 06:36:33 2013
(time ran out)
