stxxl_tool - Online in the Cloud

PROGRAM:

NAME

stxxl_tool - STXXL scratch file administration, and performance testing tool

SYNOPSIS

stxxl_tool <subtool> ...

DESCRIPTION

The stxxl_tool is an umbrella program that houses several sub-tools for STXXL relating to
(e.g.) scratch file manipulation and benchmarking. The behaviour of each sub-tool is given
below.

Size arguments will accept a string like "343KB" or "44 GiB" as their input.

Subtools
info

Print out information about the build system and which optional modules where
compiled into STXXL.

create_files <filesize> <filename ...>

Pre-create large files to keep file system allocation time out to measurements.

benchmark_disks [options] <size> [r|w] [alloc]

Benchmark the disks configured by the standard .stxxl disk configuration files
mechanism. Blocks of 8 MiB are written and/or read in sequence using the block
manager. The batch size describes how many blocks are written/read in one batch.
The are taken from block_manager using given the specified allocation strategy. If
size == 0, then writing/reading operation are done until an error occurs.

Parameters:
size Amount of data to write/read from disks (e.g. 10GiB)
r|w Only read or write blocks (default: both write and read)
alloc Block allocation strategy: RC, SR, FR, striping. (default: RC) Options:
-b, --batch Number of blocks written/read in one batch (default: D * 8MiB)

benchmark_files [options] <length> <filename ...>
Open a file using one of STXXL's file abstractions and perform write/read/verify
tests on the file. Block sizes and batch size can be adjusted via command line. If
length == 0 , then operation will continue till end of space (please ignore the
write error). Memory consumption: block_size * batch_size * num_files

Parameters:
length Length to write in file.
filename File path to run benchmark on. Options:
-o, --offset Starting offset to write in file.
--no-direct open files without O_DIRECT
--sync open files with O_SYNC|O_DSYNC|O_RSYNC
--resize resize the file size after opening, needed e.g. for
creating mmap files
--block_size block size for operations (default 8 MiB)
--batch_size increase (default 1) to submit several I/Os at once and
report average rate
-f, --file-type Method to open file (syscall|mmap|wincall|boostfd|...)
default: syscall
-p, --operations [w]rite pattern, [r]ead without verification, read and
[v]erify pattern (default: 'wv')
--pattern 32-bit pattern to write (default: block index)

Benchmark different file access methods, e.g. syscall or mmap_files.

benchmark_sort [options] <size>

Run benchmark tests of different sorting methods in STXXL. size is the amount of
data to sort, in GiB

Valid options are:
-M, --ram : amount of ram to use when sorting, default 256 MiB

benchmark_disks_random <span> [block_size] [size] [i|r|w] [alloc]

Benchmark _random_ block access on the disks configured by the standard .stxxl disk
configuration files mechanism. Available block sizes are power of two from 4 KiB to
128 MiB. A set of three operations can be performed: sequential initialization,
random reading and random writing.

Parameters:
span Span of external memory to write/read to (e.g. 10GiB).
block_size Size of blocks to randomly write/read (default: 8MiB).
size Amount of data to operate on (e.g. 2GiB), default: whole span.
i|r|w Operations: [i]nitialize, [r]ead, and/or [w]rite (default: all).
alloc Block allocation strategy: RC, SR, FR, striping (default: RC).

benchmark_pqueue [options] [size]

Benchmark the priority queue implementation using a sequence of operations. The PQ
contains pairs of 32- or 64-bit integers, or a 24 byte struct. The operation
sequence is either a simple fill/delete cycle or fill/intermixed inserts/deletes.
Because the memory parameters of the PQ must be set a compile-time, the benchmark
provides only three PQ sizes: for 256 MiB, 1 GiB and 8 GiB of RAM, with the maximum
number of items set accordingly.

Parameters:
size Amount of data to insert (e.g. 1GiB) Options:
-t, --type Value type of tested priority queue:
1 = pair of uint32,
2 = pair of uint64 (default),
3 = 24 byte struct
0 = all of the above
-p, --pq Priority queue configuration to test:
1 = small (256 MiB RAM, 4 GiB elements)
2 = medium (1 GiB RAM, 16 GiB elements) (default)
3 = big (8 GiB RAM, 64 GiB elements)
0 = all of the above
-o, --opseq Operation sequence to perform:
1 = insert all, delete all (default)
2 = insert all, intermixed insert/delete
0 = all of the above

mlock <size>

Allocate some memory and mlock() it to consume physical memory. Needs to run as
root to block more than 64 KiB in default settings.

mallinfo

Show mallinfo statistics.

Use stxxl_tool online using onworks.net services