Etp & localgroupdisk monitoring

From Etp
Jump to: navigation, search

This page explains how to monitor disk space available / occupied on our local file system /project/etp and the LRZ localgroupdisk.

For users

ETP storage

A daily summary of the ETP storage usage (mounted as /project/etpXX on your computer; XX=, 1, 2, 3, 4, 5, 6) is provided here. You can also find daily summary files in /project/etpXX/usage directly. The files are called e.g. usage_2015-07-27.log. Each /project/etpXX/usage/usage_YYYY-MM-DD.log shows the usage for the respective file system.

LOCALGROUPDISK

  • A table and pie chart with disk usage by user on the LOCALGROUPDISK can be found at the Dataset Accounting page maintained by the Uni Wuppertal (grid certificate required).
  • A nice tabular overview of the disk usage by user on the LOCALGROUPDISK which also allows to filter, sort, etc. can be found here. (Note that these tables are only updated on request, i.e. you need to ping me if you want them to be updated. The user / password is the same as for the laptops. Access from local computers should be possible without password.)
  • To check the disk space available on our LOCALGROUPDISK, you can also use rucio like this:
rucio list-rse-usage LRZ-LMU_LOCALGROUPDISK

Output is e.g.

 used: 294.8225 TB
 rse: LRZ-LMU_LOCALGROUPDISK
 updated_at: 2015-08-12 08:10:47
 free: 25.2157 TB
 source: srm
 total: 320.0382 TB


For administrators

Setup and scripts

  • /project/etp usage: RBG provides a daily summary text file for /project/etpXX usage at /project/etpXX/usage/usage_20*.log , e.g.

/project/etp/usage/usage_2015-08-12.log

  • dCache LRZ localgroupdisk usage: Daily cron-job on etpopt01 using ATLAS DDM API to get datasets & replica owners:
 2 2 * * * test -x /project/etp/gduckeck/atlas/ddm/update_lgd_usage.sh && \
 /project/etp/gduckeck/atlas/ddm/update_lgd_usage.sh 2>&1 > /tmp/update_lgd_usage.log
  • transfer to ETP webserver: Daily cron-job on gar-ws-etp23 copies etp & LGC data via ftp to

http://www.etp.physik.uni-muenchen.de/repo

 3 3 * * * /home/g/GDuckeck/bin/transfer_etp_usage_log.sh 2>&1 > /tmp/transfer_etp_usage.log
 5 3 * * *       root    test -x /var/lib/dcache-mon/etp_usage/transfer_logs.sh && \
  /var/lib/dcache-mon/etp_usage/transfer_logs.sh 2>&1 >> /var/log/etp_usage.log