resource – System resource management

Purpose:Manage the system resource limits for a Unix program.
Available In:1.5.2

The functions in resource probe the current system resources consumed by a process, and place limits on them to control how much load a program can impose on a system.

Current Usage

Use getrusage() to probe the resources used by the current process and/or its children. The return value is a data structure containing several resource metrics based on the current state of the system.


Not all of the resource values gathered are displayed here. Refer to the stdlib docs for a more complete list.

import resource
import time

usage = resource.getrusage(resource.RUSAGE_SELF)

for name, desc in [
    ('ru_utime', 'User time'),
    ('ru_stime', 'System time'),
    ('ru_maxrss', 'Max. Resident Set Size'),
    ('ru_ixrss', 'Shared Memory Size'),
    ('ru_idrss', 'Unshared Memory Size'),
    ('ru_isrss', 'Stack Size'),
    ('ru_inblock', 'Block inputs'),
    ('ru_oublock', 'Block outputs'),
    print '%-25s (%-10s) = %s' % (desc, name, getattr(usage, name))

Because the test program is extremely simple, it does not use very many resources:

$ python

User time                 (ru_utime  ) = 0.010192
System time               (ru_stime  ) = 0.005743
Max. Resident Set Size    (ru_maxrss ) = 3891200
Shared Memory Size        (ru_ixrss  ) = 0
Unshared Memory Size      (ru_idrss  ) = 0
Stack Size                (ru_isrss  ) = 0
Block inputs              (ru_inblock) = 0
Block outputs             (ru_oublock) = 0

Resource Limits

Separate from the current actual usage, it is possible to check the limits imposed on the application, and then change them.

import resource

for name, desc in [
    ('RLIMIT_CORE', 'core file size'),
    ('RLIMIT_CPU',  'CPU time'),
    ('RLIMIT_FSIZE', 'file size'),
    ('RLIMIT_DATA', 'heap size'),
    ('RLIMIT_STACK', 'stack size'),
    ('RLIMIT_RSS', 'resident set size'),
    ('RLIMIT_NPROC', 'number of processes'),
    ('RLIMIT_NOFILE', 'number of open files'),
    ('RLIMIT_MEMLOCK', 'lockable memory address'),
    limit_num = getattr(resource, name)
    soft, hard = resource.getrlimit(limit_num)
    print 'Maximum %-25s (%-15s) : %20s %20s' % (desc, name, soft, hard)

The return value for each limit is a tuple containing the soft limit imposed by the current configuration and the hard limit imposed by the operating system.

$ python

Maximum core file size            (RLIMIT_CORE    ) :                    0  9223372036854775807
Maximum CPU time                  (RLIMIT_CPU     ) :  9223372036854775807  9223372036854775807
Maximum file size                 (RLIMIT_FSIZE   ) :  9223372036854775807  9223372036854775807
Maximum heap size                 (RLIMIT_DATA    ) :  9223372036854775807  9223372036854775807
Maximum stack size                (RLIMIT_STACK   ) :              8388608             67104768
Maximum resident set size         (RLIMIT_RSS     ) :  9223372036854775807  9223372036854775807
Maximum number of processes       (RLIMIT_NPROC   ) :                  709                 1064
Maximum number of open files      (RLIMIT_NOFILE  ) :                 2560  9223372036854775807
Maximum lockable memory address   (RLIMIT_MEMLOCK ) :  9223372036854775807  9223372036854775807

The limits can be changed with setrlimit(). For example, to control the number of files a process can open the RLIMIT_NOFILE value can be set to use a smaller soft limit value.

import resource
import os

soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE)
print 'Soft limit starts as  :', soft

resource.setrlimit(resource.RLIMIT_NOFILE, (4, hard))

soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE)
print 'Soft limit changed to :', soft

random = open('/dev/random', 'r')
print 'random has fd =', random.fileno()
    null = open('/dev/null', 'w')
except IOError, err:
    print err
    print 'null has fd =', null.fileno()
$ python

Soft limit starts as  : 2560
Soft limit changed to : 4
random has fd = 3
[Errno 24] Too many open files: '/dev/null'

It can also be useful to limit the amount of CPU time a process should consume, to avoid eating up too much time. When the process runs past the allotted amount of time, it it sent a SIGXCPU signal.

import resource
import sys
import signal
import time

# Set up a signal handler to notify us
# when we run out of time.
def time_expired(n, stack):
    print 'EXPIRED :', time.ctime()
    raise SystemExit('(time ran out)')

signal.signal(signal.SIGXCPU, time_expired)

# Adjust the CPU time limit
soft, hard = resource.getrlimit(resource.RLIMIT_CPU)
print 'Soft limit starts as  :', soft

resource.setrlimit(resource.RLIMIT_CPU, (1, hard))

soft, hard = resource.getrlimit(resource.RLIMIT_CPU)
print 'Soft limit changed to :', soft

# Consume some CPU time in a pointless exercise
print 'Starting:', time.ctime()
for i in range(200000):
    for i in range(200000):
        v = i * i

# We should never make it this far
print 'Exiting :', time.ctime()

Normally the signal handler should flush all open files and close them, but in this case it just prints a message and exits.

$ python

Soft limit starts as  : 9223372036854775807
Soft limit changed to : 1

Starting: Thu Feb 21 06:36:32 2013
EXPIRED : Thu Feb 21 06:36:33 2013
(time ran out)

See also

The standard library documentation for this module.
For details on registering signal handlers.