Enable Globus Online transfers with multi hop through NERSC

[ericvaandering]

Check with Radu about configuration. Likely changed a lot since we last looked at this.

• Upgrade to 1.28
• Recheck if we need to add patches to image build scripts
• Add to rucio-flux Int globus rucio-flux#103
• Update secrets to include ones of globus https://github.com/ericvaandering/rucio-flux/blob/int_globus/scripts/create_flux_secrets.sh

[ericvaandering]

Discussion with Radu

For the production submitter configuration, You only have to set

[conveyor]
transfertool = fts3,globus  # if you don't need multi-hop between globus and fts3, you can set it to only 'fts3'

the production poller will have

[conveyor]
transfertool = fts3

It's important to not set filter_transfertool for this instances. If you set it to fts3, it will filter out RSEs which don't contain fts3 in their rse attribute transfertool . Alternatively, you can update all your RSEs to have this attribute set correctly. This will be required in a later (major) release of rucio anyway. We should have done t for 1.29, but we forgot about this database migration. (edited)

For the special "globus-conveyor" helm release, you'll set submitter to

[conveyor]
transfertool = globus,fts3  # if you plan to activate multi-hopping later; otherwise set it to 'globus'
filter_transfertool = globus                                                                             
globus_auth_app = CMSRucioProduction

and poller to

[conveyor]
transfertool = globus
filter_transfertool = globus                                                                             
globus_auth_app = CMSRucioProduction

you cannot get arround without having two separate helm releases for now. Because conveyor-poller can only work on one transfertool at a time. So you need two pollers. One with transfertool = fts3 ; the second with transfertool = globus : https://github.com/rucio/rucio/blob/27a2442e3687deace896fc35865eecd6b0e30200/lib/rucio/daemons/conveyor/poller.py#L94

[ericvaandering]

Working with a patch to Rucio in integration. Patch is still being evaluated by ATLAS. Closing this part of it.