Archive/backup datasets to tape
DE users can request dataset copy to a tape endpoint at GridKa, named FZK-LCG2_LOCALGROUPTAPE. This is intended for cases such as:
- critical datasets and critical phase of analysis, you want to make sure a 2nd copy of these datasets is available
- old but important datasets (reference for paper, thesis, etc). Put copy on tape to free space on disk endpoint
For transfer/access of the datasets on FZK-LCG2_LOCALGROUPTAPE you can use in principle the the standard Rucio tools. For an existing dataset already stored somewhere simply add a rule, e.g.
rucio add-rule user.myname:MyDataSetName 1 FZK-LCG2_LOCALGROUPTAPE
However, there are these additional requirements when you want to replicate your own data files onto tape endpoints.
1. Check whether you have quota to write to FZK-LCG2_LOCALGROUPTAPE
rucio list-account-limits --rse FZK-LCG2_LOCALGROUPTAPE myaccount
If you don't have quota ask Guenter to set/increase.
2. When creating a Rucio dataset from your own files you must specify explicitly a dataset name, i.e.
# this works rucio -v upload --rse LRZ-LMU_SCRATCHDISK user.myname:MyDataSetName file1 file2 file3 # # this does not work rucio -v upload --rse LRZ-LMU_SCRATCHDISK file1 file2 file3 # upload to disk endpoint works, but it cannot be replicated to tape endpoint
3. You must first create and upload the dataset to a disk endpoint and then replicate it to FZK-LCG2_LOCALGROUPTAPE.
# upload to disk first rucio -v upload --rse LRZ-LMU_SCRATCHDISK user.myname:MyDataSetName file1 file2 file3 # replicate to Tape rucio add-rule user.myname:MyDataSetName 1 FZK-LCG2_LOCALGROUPTAPE # # check status (typically takes some 10 mins for a few GB) rucio list-rules user.myname:MyDataSetName