Skip to content
This repository has been archived by the owner on Feb 23, 2023. It is now read-only.

FAQs

Jump to bottom
Víctor Méndez edited this page Feb 5, 2014 · 45 revisions

go to Users FAQs

go to IaaS providers FAQs

Users FAQs

In order to install my software stack, I have to do wget first, and then run a python script that install it on the machine, and finally launch the command to run my simulations. Is this possible with VMDIRAC Cloud scheme?

The interactions with the VM in Cloud Computing are automatic, no user loggin, no manual handling, DIRAC do the magic for you.

As a first approach you can create an ad-hoc image with your software stack, and later, when this image is running in a VM we contextualize with DIRAC's stuff using ssh.

If this is test is successfull, we can automatize the software stack contextualization, which for image maintenance resons is better. For this purpose we could use a cvmfs repository or an automatic installation by HEPiX or ssh contextualization.

Here it is a possible recipe to prepare an ad-hoc image with your software stack.

  1. Install VirtualBox in your computer.
  2. Download your favorite Linux install image
  3. Run VirtualBox
  4. "new" VM: Name something like "Linux-mysw-vm", OS: Linux, Version (Your like, f.e. Ubuntu), mem (depending on your mysw memory footprint), new virtual disk (check .hdd format), virtual disk size as needed to run your mysw and temporaly store outputs in the VM
  5. Start your Linux-mysw-vm, the first start you are requested to select the .iso with your favorite linux install, to be installed in the VM
  6. Follow your favorite linux installation instructions, (lan, key, install full virtual disk, login, password...)
  7. Finish installation in the VM and restart VM
  8. Login the VM with your user/pass (better using the session xterm at bottom right of ubuntu login, so you directly go to a terminal)
  9. Install dependencies of your mysw and test everything works as expected
  10. Delete output files of the test and rename xdm deamon (for example /etc/init.d/kdm or gnome...) to avoid Desktop Manager to be running (which in Cloud Computing has no sense and it is highly memory consumer)
  11. Your ad-hoc image is ready to be upload to the cloud manager, you will find it in ~/VirtualBox VMs/Linux-mysw-vm/Linux-mysw-vm.hdd image.

This image will have a pubkey injection to allows ssh cloud contextualization.

Then you will use DIRAC portal to send JDLs and a executable or script.sh containing parameters and whatever run you want. DIRAC will created new Linux-mysw-vm as needed, boot them, contextualize by ssh to be used in Cloud with DIRAC, then mathching with your DIRAC jobs.

When finish some job, you can dowload the outputsanbox of your run

What is the DIRAC .JDL and executable or script.sh ?

It is the way DIRAC allows you to define jobs to be run in distributed resources, you can have a look for an example of .JDL for a job with output sandbox

How can I submitte large scale runs ?

The simplest option is to do a single definition of a parametric job with many runs as parameters. Have a look to specify a sequence of parameters to serialize a simple job.

DIRAC has more complex workflows, ussualy for e-science community requirements, send us a mail with your particular specifications.

I am a single user and I have strong oportunistic vocation, I would like to use cloud resources of others... is anybody out there ?

Yes... you could have a chance, ask your NGI or how to get access to EGI Fedcloud

I am a user registered at VO fedcloud.egi.eu, could you give me a recipe for the impatients, please ?

Here it is, but believes that good work takes time

You can use the DIRAC portal for Ibergrid, which it is prepared to accept fecloud.egi.eu using EGI Fedcloud IaaS resources

  1. Delegation of user proxy to the portal in order for it to act on your behalf
    1.1. Using DIRAC portal: Click in certificate login (bottom right). Menu Tools / Proxy Upload
    1.2. Using DIRAC client: follow the instructions of Ibergrid docu web to install a DIRAC client on you machine, then: dirac-proxy-init -g fedcloud_user
  2. Submitting jobs to the DIRAC broker: Select fedcloud_user (bottom right). Menu Tools / Job Launchpad
  3. Monitoring your jobs: Menu Jobs / Job Monitoring
  4. Monitoring VM management
    4.1. Menu Virtual Machines / Overview → Statistical plots
    4.2. Menu Virtual Machines / Browse Instances → VM list and context box with specific VMs history log and VM monitoring plots

... and take care, If you are stalled you could go to docu of DIRAC portal for Ibergrid

IaaS providers FAQs

I'm lately working on connecting our grid and cloud into DIRAC portal. The first thing we have to figure out is that, how much DIRAC system associates with grid BDII service ?

There is only a very "light" link between DIRAC configuration and the BDII. All DIRAC components take the necessary information for their execution from the DIRAC Configuration. There exists a tool, "Configuration/CE3CSAgent", that can be used to automate information update about the configured Grid CEs from the BDII, the tool can be configured to inform about new CE's supporting your VO, but the DIRAC admin has to take an action to assign this CE to an existing Site or to create a new Site (whatever is more appropriated on each particular case).

There are some efforts to have some cloud BDII, but we are not relying on that at the moment. In order to make VMDIRAC (the DIRAC extension working with clouds) aware of the available endpoint, the access mechanism, or the images that are supported on each case, a configuration section in the /Resources folder is used and has to be populated by the Admin.

we are interested in building a VM that could be a dirac agent (having only outbound connectivity).

In this way we can customize this image with the application software and ask for an execution of those machines on a OpenStack based IaaS. The interesting case could be using several virtual machines on different cloud infrastructure geographically distributed. Do you have any installation/configuration guide for the “DIRAC machinery (agents to be run on the VMs)” ?

R: then you can go for ami library config with userdata. There is a way to porvide in userdata a script to be run in the VMs, there is an additional constrain with OpenStack about userdata size, so some times we had to just download an external scprit and run. An easy option is you just use an user data wich donwload something like: Orchestator configuration scprit and DIRAC client configuration script this general-DIRAC-context is not valid for you, since: python dirac-install -V "VMDIRAC" is goint to install a pre-configure VM for our developing machine, so the simplest for you is to clone the general-DIRAC-context.sh and modified at least python dirac-install -V "Your-Cloud" and ask us to add at global defaults central server your CS url, for auto conf of the VMs.

Clone this wiki locally