This is the command sh5util that can be run in the OnWorks free hosting provider using one of our multiple free online workstations such as Ubuntu Online, Fedora Online, Windows online emulator or MAC OS online emulator
PROGRAM:
NAME
sh5util - Tool for merging HDF5 files from the acct_gather_profile plugin that gathers
detailed data for jobs running under Slurm
SYNOPSIS
sh5util
DESCRIPTION
sh5util merges HDF5 files produced on each node for each step of a job into one HDF5 file
for the job. The resulting file can be viewed and manipulated by common HDF5 tools such as
HDF5View, h5dump, h5edit, or h5ls.
sh5util also has two extract modes. The first, writes a limited set of data for specific
nodes, steps, and data series in "comma separated value" form to a file which can be
imported into other analysis tools such as spreadsheets.
The second, (Item-Extract) extracts one data time from one time series for all the samples
on all the nodes from a jobs HDF5 profile.
- Finds sample with maximum value of the item.
- Write CSV file with min, ave, max, and item totals for each node for each
sample
OPTIONS
-E, --extract
Extract data series from a merged job file.
Extract mode options
-i, --input=path
merged file to extract from (default ./job_$jobid.h5)
-N, --node=nodename
Node name to extract (default is all)
-l, --level=[Node:Totals|Node:TimeSeries]
Level to which series is attached. (default Node:Totals)
-s, --series=[Energy | Lustre | Network | Tasks | Task_#]
Tasks is all tasks, Task_# (# is a task id) (default is everything)
-I, --item-extract
Extract one data item from all samples of one data series from all nodes in a
merged job file.
Item-Extract mode options
-s, --series=[Energy | Lustre | Network | Task]
-d, --data
Name of data item in series (See note below).
-j, --jobs=<job(.step)>
Format is <job(.step)>. Merge this job/step (or a comma-separated list of job
steps). This option is required. Not specifying a step will result in all steps
found to be processed.
-h, --help
Print this description of use.
-o, --output=path
Path to a file into which to write.
Default for merge is ./job_$jobid.h5
Default for extract is ./extract_$jobid.csv
-p, --profiledir=dir
Directory location where node-step files exist default is set in acct_gather.conf.
-S, --savefiles
Instead of removing node-step files after merging them into the job file, keep them
around.
--user=user
User who profiled job. (Handy for root user, defaults to user running this
command.)
--usage
Display brief usage message.
Data Items per Series
Energy
Power
CPU_Frequency
Lustre
Reads
Megabytes_Read
Writes
Megabytes_Write
Network
Packets_In
Megabytes_In
Packets_Out
Megabytes_Out
Task
CPU_Frequency
CPU_Time
CPU_Utilization
RSS
VM_Size
Pages
Read_Megabytes
Write_Megabytes
Examples
Merge node-step files (as part of a sbatch script)
sbatch -n1 -d$SLURM_JOB_ID --wrap="sh5util --savefiles -j $SLURM_JOB_ID"
Extract all task data from a node
sh5util -j 42 -N snowflake01 --level=Node:TimeSeries --series=Tasks
Extract all energy data
sh5util -j 42 --series=Energy --data=power
COPYING
Copyright (C) 2013 Bull.
Copyright (C) 2013 SchedMD LLC. Slurm is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2 of the License, or (at your option) any later
version.
Slurm is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without
even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
Use sh5util online using onworks.net services