Etp & localgroupdisk monitoring
This page explains how to monitor disk space available / occupied on our local file system /project/etp and the LRZ localgroupdisk.
A daily summary of the ETP storage usage (mounted as /project/etpXX on your computer; XX=, 1, 2, 3, 4, 5, 6) is provided here. You can also find daily summary files in /project/etpXX/usage directly. The files are called e.g. usage_2015-07-27.log. Each /project/etpXX/usage/usage_YYYY-MM-DD.log shows the usage for the respective file system.
- A table and pie chart with disk usage by user on the LOCALGROUPDISK can be found at the Dataset Accounting page maintained by the Uni Wuppertal (grid certificate required).
- A nice tabular overview of the disk usage by user on the LOCALGROUPDISK which also allows to filter, sort, etc. can be found here. (Note that these tables are only updated on request, i.e. you need to ping me if you want them to be updated. The user / password is the same as for the laptops. Access from local computers should be possible without password.)
- To check the disk space available on our LOCALGROUPDISK, you can also use rucio like this:
rucio list-rse-usage LRZ-LMU_LOCALGROUPDISK
Output is e.g.
used: 294.8225 TB rse: LRZ-LMU_LOCALGROUPDISK updated_at: 2015-08-12 08:10:47 free: 25.2157 TB source: srm total: 320.0382 TB
Setup and scripts
- /project/etp usage: RBG provides a daily summary text file for /project/etpXX usage at /project/etpXX/usage/usage_20*.log , e.g.
- dCache LRZ localgroupdisk usage: Daily cron-job on etpopt01 using ATLAS DDM API to get datasets & replica owners:
2 2 * * * test -x /project/etp/gduckeck/atlas/ddm/update_lgd_usage.sh && \ /project/etp/gduckeck/atlas/ddm/update_lgd_usage.sh 2>&1 > /tmp/update_lgd_usage.log
- transfer to ETP webserver: Daily cron-job on gar-ws-etp23 copies etp & LGC data via ftp to
3 3 * * * /home/g/GDuckeck/bin/transfer_etp_usage_log.sh 2>&1 > /tmp/transfer_etp_usage.log
- Monitoring service on lcg-lrz-site-mon.grid.lrz.de:
- fetches data from http://www.etp.physik.uni-muenchen.de/repo to /var/lib/dcache-mon/etp_usage/usagelogs
- daily cron ( /etc/cron.d/etp_usage ) :
5 3 * * * root test -x /var/lib/dcache-mon/etp_usage/transfer_logs.sh && \ /var/lib/dcache-mon/etp_usage/transfer_logs.sh 2>&1 >> /var/log/etp_usage.log