Skip to content
This repository has been archived by the owner on Feb 23, 2023. It is now read-only.

FAQs

Jump to bottom
Víctor Méndez edited this page Jul 21, 2015 · 45 revisions

go to Users FAQs

go to IaaS/DIRAC providers FAQs

Users FAQs

In order to install my software stack, I have to do wget first, and then run a python script that install it on the machine, and finally launch the command to run my simulations. Is this possible with VMDIRAC Cloud scheme?

The interactions with the VM in Cloud Computing are automatic, no user loggin, no manual handling, DIRAC do the magic for you.

As a first approach you can create an ad-hoc image with your software stack, and later, when this image is running in a VM we contextualize with DIRAC's stuff using ssh.

If this is test is successfull, we can automatize the software stack contextualization, which for image maintenance resons is better. For this purpose we could use a cvmfs repository or an automatic installation by HEPiX or ssh contextualization.

Here it is a possible recipe to prepare an ad-hoc image with your software stack.

  1. The faster way is to use specific images for Cloud Computing.
  2. Upload your image base in your cloud provider by URL referencing
  3. Crate a new instance, associate a public key to the instance and an public IP
  4. ssh the instance using default user of your image base (ubuntu, root...)
  5. Install and configure your software stack
  6. Create a snap shoot image, which will be referenced in your DIRAC running pod
  7. Ask dirac administrators to set-up your cloud infrastructure.

Then you will use DIRAC portal to send JDLs and a executable or script.sh containing parameters and whatever run you want. DIRAC will created new Linux-mysw-vm as needed, boot them, contextualize by ssh to be used in Cloud with DIRAC, then mathching with your DIRAC jobs.

When finish some job, you can dowload the outputsanbox of your run

What is the DIRAC .JDL and executable or script.sh ?

It is the way DIRAC allows you to define jobs to be run in distributed resources, you can have a look for an example of .JDL for a job with output sandbox

How can I submitte large scale runs ?

The simplest option is to do a single definition of a parametric job with many runs as parameters. Have a look to specify a sequence of parameters to serialize a simple job.

DIRAC has more complex workflows, ussualy for e-science community requirements, send us a mail with your particular specifications.

I am a single user and I have strong oportunistic vocation, I would like to use cloud resources of others... is anybody out there ?

Yes... you could have a chance, ask your NGI or how to get access to EGI Fedcloud

I am a user registered at VO fedcloud.egi.eu, could you give me a recipe for the impatients, please ?

Here it is, but believes that good work takes time

You can use the DIRAC 4 EGI portal, which it is prepared to accept fecloud.egi.eu using EGI Fedcloud IaaS resources

  1. Delegation of user proxy to the portal in order for it to act on your behalf
    1.1. Using DIRAC portal: Click in certificate login (bottom right). Menu Tools / Proxy Upload
    1.2. Using DIRAC client: follow the instructions of Ibergrid docu web to install a DIRAC client on you machine, then: dirac-proxy-init -g fedcloud_user
  2. Submitting jobs to the DIRAC broker: Select fedcloud_user (bottom right). Menu Tools / Job Launchpad
  3. Monitoring your jobs: Menu Jobs / Job Monitoring
  4. Monitoring VM management
    4.1. Menu Virtual Machines / Overview → Statistical plots
    4.2. Menu Virtual Machines / Browse Instances → VM list and context box with specific VMs history log and VM monitoring plots

... and take care, If you are stalled you could go to docu of DIRAC portal for Ibergrid

IaaS/DIRAC Providers FAQs

I'm lately working on connecting our grid and cloud into DIRAC portal. The first thing we have to figure out is that, how much DIRAC system associates with grid BDII service ?

There is only a very "light" link between DIRAC configuration and the BDII. All DIRAC components take the necessary information for their execution from the DIRAC Configuration. There exists a tool, "Configuration/CE3CSAgent", that can be used to automate information update about the configured Grid CEs from the BDII, the tool can be configured to inform about new CE's supporting your VO, but the DIRAC admin has to take an action to assign this CE to an existing Site or to create a new Site (whatever is more appropriated on each particular case).

There are some efforts to have some cloud BDII, but we are not relying on that at the moment. In order to make VMDIRAC (the DIRAC extension working with clouds) aware of the available endpoint, the access mechanism, or the images that are supported on each case, a configuration section in the /Resources folder is used and has to be populated by the Admin.

we are interested in building a VM that could be a dirac agent (having only outbound connectivity).

In this way we can customize this image with the application software and ask for an execution of those machines on a OpenStack/Amazon based IaaS. The interesting case could be using several virtual machines on different cloud infrastructure geographically distributed. Do you have any installation/configuration guide for the “DIRAC machinery (agents to be run on the VMs)” ?

R: Then you can go for ami library config with userdata. There is a way to porvide in userdata a script to be run in the VMs, there is an additional constrain with OpenStack about userdata size, so some times we had to just download an external scprit and run. An easy option is you just use an user data wich donwload something like: Orchestator configuration scprit and DIRAC client configuration script this general-DIRAC-context is not valid for you, since: python dirac-install -V "VMDIRAC" is goint to install a pre-configure VM for our developing machine, so the simplest for you is to clone the general-DIRAC-context.sh and modified at least python dirac-install -V "Your-Cloud" and ask us to add at global defaults central server your CS url, for auto conf of the VMs. This is not only for OpenStack, it is the same for Amazon EC2, have a look to Amazon how to launch instances with userdata and other Amazon related docu

I only have outbounding connectivity and OpenNebula, which contextualization method could I use

R: Then use prolog/epilog scheme: prolog example epilog example But these are just templates, for latest updates take the corresponding functinalities from latest ssh context scprits Orchestator configuration scprit and DIRAC client configuration script and again, this general-DIRAC-context is not valid for you, since: python dirac-install -V "VMDIRAC" is goint to install a pre-configure VM for our developing machine, so the simplest for you is to clone the general-DIRAC-context.sh and modified at least python dirac-install -V "Your-Cloud" and ask us to add at global defaults central server your CS url, for auto conf of the VMs.

We have a DIRAC server installed, so I guess I should change the reference to the DIRAC server where the VM agents should connect.

Is there a configuration variable to change in the script?

R: Actually, you can do on your own but it is not only the CS, it is about more conf options that you can put in a file in some of your web servers, and the we point to such generic .cfg to be apply in every configuration, something like this example Just tell us when ready, then we could add to the central globals defaults to use your generic .cfg in all your VMs, I think is the simplest option to you.

does the VM need to have a server certificate to interact with the DIRAC server or the list of the supported Certification Authorities (/etc/grid-security/certificates)?

In order to seucure connect VMs with DIRAC server, you need a host certificate, a single one for all your VMs can be ok, default path in VM is /opt/dirac/etc/grid-security. Then you need to set appropiate properties to the VM host in the Configuration: Registry/Hosts/

 vmdirac
 {
   DN = < your DN >
   Properties = GenericPilot
   Properties += LimitedDelegation
   Properties += VmRpcOperation
 }

The issuer of such certificate (the CA) should be at the server side in /opt/dirac/etc/grid-security/certificates

Clone this wiki locally