OCCI library
From Gcube Wiki
Scope
This library has been developed in order to allow the integration of gcube with EGI FedCloud infrastructure. In particular, the usage of the library will facilitate the interaction with the sites exposing OCCI just by manipulating APIs.
Usage
Maven coordinates
The maven artifact coordinates are:
<dependency>
<groupId>org.gcube.resources.federation</groupId>
<artifactId>fhn-occi-connector</artifactId>
<version>1.0.0-SNAPSHOT</version>
</dependency>
Check on Nexus for the latest version.
APIs
In the following the functionalities proposed by the library are shown.
Site Connection
VOMS proxy generation
In order to access the EGI FedCloud infrastructure it is necessary to create a VOMS proxy certificate. Open a shell and run the following command:
$voms-proxy-init --voms fedcloud.egi.eu --rfc --dont-verify-ac
Later it is necessary to digit the own EGI identity grid password.
Enter GRID pass phrase for this identity: Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=ENGINEERING RDLAB/CN=Nunzio Andrea Galante Creating temporary proxy ........................ Done Contacting voms1.egee.cesnet.cz:15002 [/DC=org/DC=terena/DC=tcs/OU=Domain Control Validated/CN=voms1.egee.cesnet.cz] "fedcloud.egi.eu" Done Creating proxy ...................................... Done Your proxy is valid until Thu Nov 12 01:13:17 2015
Authentication
The library provides both VOMS and X509 access authentication methods. A list of sites exposing OCCI and endorsed by FedCloud VO is available at Appdb repository.
String a = X509CredentialManager.createProxy("/tmp/your_proxy.pem", ""); //2nd level proxy FedcloudOCCI jocci = new FedcloudOCCI("endpoint"); //FedcloudOCCI jocci = new FedcloudOCCI("https://carach5.ics.muni.cz:11443"); jocci.setCertificate(a); jocci.setTrustStore("CA Path certificates");
Endpoint Connection
jocci.connect();
Sample output:
Connected to endpoint: https://carach5.ics.muni.cz:11443/
Resources listing
Own compute resources
jocci.getComputeList();
Sample output:
Computes Location: https://carach5.ics.muni.cz:11443/compute/64586
Storage resources
jocci.getNetList();
Sample output:
Storage Locations: https://carach5.ics.muni.cz:11443/storage/774; https://carach5.ics.muni.cz:11443/storage/775; https://carach5.ics.muni.cz:11443/storage/776; https://carach5.ics.muni.cz:11443/storage/777; https://carach5.ics.muni.cz:11443/storage/779;
Network resources
jocci.getStorageList();
Sample output:
Network Locations: https://carach5.ics.muni.cz:11443/network/6 https://carach5.ics.muni.cz:11443/network/8 https://carach5.ics.muni.cz:11443/network/17 https://carach5.ics.muni.cz:11443/network/24
Getting OS templates
jocci.getOsTemplateList();
Sample output:
17 AVAILABLE OS TEMPLATES MIXINS
Category{term=uuid_umd_sl6_fedcloud_warg_138, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=UMD-SL6@fedcloud-warg, location=/mixin/os_tpl/uuid_umd_sl6_fedcloud_warg_138/, attributes=SetCover{[]}}
Category{term=uuid_egi_centos_6_fedcloud_warg_130, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=EGI-CentOS-6@fedcloud-warg, location=/mixin/os_tpl/uuid_egi_centos_6_fedcloud_warg_130/, attributes=SetCover{[]}}
Category{term=uuid_egi_fedcloud_clients_fedcloud_warg_128, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=EGI-FedCloud-Clients@fedcloud-warg, location=/mixin/os_tpl/uuid_egi_fedcloud_clients_fedcloud_warg_128/, attributes=SetCover{[]}}
Category{term=uuid_egi_ubuntu_server_14_04_lts_fedcloud_warg_131, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=EGI-Ubuntu-Server-14.04-LTS@fedcloud-warg, location=/mixin/os_tpl/uuid_egi_ubuntu_server_14_04_lts_fedcloud_warg_131/, attributes=SetCover{[]}}
Category{term=uuid_chipster_ubuntu_server_12_04_lts_fedcloud_dukan_96, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=Chipster-Ubuntu-Server-12.04-LTS@fedcloud-dukan, location=/mixin/os_tpl/uuid_chipster_ubuntu_server_12_04_lts_fedcloud_dukan_96/, attributes=SetCover{[]}}
Category{term=uuid_egi_compss_lofar_debian_7_0_x86_64_0001_fedcloud_dukan_113, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=EGI-COMPSs-Lofar-Debian-7.0-x86_64-0001@fedcloud-dukan, location=/mixin/os_tpl/uuid_egi_compss_lofar_debian_7_0_x86_64_0001_fedcloud_dukan_113/, attributes=SetCover{[]}}
Category{term=uuid_centos_6_5_minimal_fedcloud_dukan_83, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=CentOS-6.5-minimal@fedcloud-dukan, location=/mixin/os_tpl/uuid_centos_6_5_minimal_fedcloud_dukan_83/, attributes=SetCover{[]}}
Category{term=uuid_transplant_base_fedcloud_warg_132, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=transPLANT-Base@fedcloud-warg, location=/mixin/os_tpl/uuid_transplant_base_fedcloud_warg_132/, attributes=SetCover{[]}}
Category{term=uuid_ubuntu_server_14_04_lts_fedcloud_dukan_84, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=Ubuntu-Server-14.04-LTS@fedcloud-dukan, location=/mixin/os_tpl/uuid_ubuntu_server_14_04_lts_fedcloud_dukan_84/, attributes=SetCover{[]}}
Category{term=uuid_egi_pmes_14_ubuntu_15_04_x86_64_0001_fedcloud_warg_108, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=EGI-PMES-14-Ubuntu-15.04-x86_64-0001@fedcloud-warg, location=/mixin/os_tpl/uuid_egi_pmes_14_ubuntu_15_04_x86_64_0001_fedcloud_warg_108/, attributes=SetCover{[]}}
Category{term=uuid_dci_bridge_fedcloud_dukan_109, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=DCI-Bridge@fedcloud-dukan, location=/mixin/os_tpl/uuid_dci_bridge_fedcloud_dukan_109/, attributes=SetCover{[]}}
Category{term=uuid_egi_ubuntu_server_12_04_lts_fedcloud_warg_129, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=EGI-Ubuntu-Server-12.04-LTS@fedcloud-warg, location=/mixin/os_tpl/uuid_egi_ubuntu_server_12_04_lts_fedcloud_warg_129/, attributes=SetCover{[]}}
Category{term=uuid_scientific_linux_6_5_fedcloud_dukan_82, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=Scientific-Linux-6.5@fedcloud-dukan, location=/mixin/os_tpl/uuid_scientific_linux_6_5_fedcloud_dukan_82/, attributes=SetCover{[]}}
Category{term=uuid_cernvm_3_3_0_40gb_fedcloud_dukan_80, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=CERNVM-3.3.0-40GB@fedcloud-dukan, location=/mixin/os_tpl/uuid_cernvm_3_3_0_40gb_fedcloud_dukan_80/, attributes=SetCover{[]}}
Category{term=uuid_moinmoinwiki_fedcloud_warg_124, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=MoinMoinWiki@fedcloud-warg, location=/mixin/os_tpl/uuid_moinmoinwiki_fedcloud_warg_124/, attributes=SetCover{[]}}
Category{term=uuid_gcubesmartexecutor_fedcloud_warg_139, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=GCubeSmartExecutor@fedcloud-warg, location=/mixin/os_tpl/uuid_gcubesmartexecutor_fedcloud_warg_139/, attributes=SetCover{[]}}
Category{term=uuid_ubuntu_server_12_04_lts_fedcloud_dukan_81, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=Ubuntu-Server-12.04-LTS@fedcloud-dukan, location=/mixin/os_tpl/uuid_ubuntu_server_12_04_lts_fedcloud_dukan_81/, attributes=SetCover{[]}}Getting Resource templates
jocci.getResourcesTemplateList();
Sample output:
7 AVAILABLE RESOURCES TEMPLATES MIXINS
Category{term=mammoth, scheme=http://schema.fedcloud.egi.eu/occi/infrastructure/resource_tpl#, title=Mammoth Instance - 8 cores and 32 GB RAM, location=/mixin/resource_tpl/mammoth/, attributes=SetCover{[Attribute{name=occi.compute.cores, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.architecture, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.speed, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.memory, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}]}}
Category{term=large, scheme=http://schema.fedcloud.egi.eu/occi/infrastructure/resource_tpl#, title=Large Instance - 4 cores and 8 GB RAM, location=/mixin/resource_tpl/large/, attributes=SetCover{[Attribute{name=occi.compute.cores, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.architecture, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.speed, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.memory, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}]}}
Category{term=small, scheme=http://schema.fedcloud.egi.eu/occi/infrastructure/resource_tpl#, title=Small Instance - 1 core and 2 GB RAM, location=/mixin/resource_tpl/small/, attributes=SetCover{[Attribute{name=occi.compute.cores, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.architecture, required=false, immutable=false, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.speed, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.memory, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}]}}
Category{term=medium, scheme=http://schema.fedcloud.egi.eu/occi/infrastructure/resource_tpl#, title=Medium Instance - 2 cores and 4 GB RAM, location=/mixin/resource_tpl/medium/, attributes=SetCover{[Attribute{name=occi.compute.cores, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.architecture, required=false, immutable=false, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.speed, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.memory, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}]}}
Category{term=goliath, scheme=http://schema.fedcloud.egi.eu/occi/infrastructure/resource_tpl#, title=Goliath Instance - 16 cores and 64 GB RAM, location=/mixin/resource_tpl/goliath/, attributes=SetCover{[Attribute{name=occi.compute.cores, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.architecture, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.speed, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.memory, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}]}}
Category{term=extra_large, scheme=http://schema.fedcloud.egi.eu/occi/infrastructure/resource_tpl#, title=Extra Large Instance - 4 cores and 16 GB RAM, location=/mixin/resource_tpl/extra_large/, attributes=SetCover{[Attribute{name=occi.compute.cores, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.architecture, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.speed, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.memory, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}]}}
Category{term=sym_mammoth, scheme=http://schema.fedcloud.egi.eu/occi/infrastructure/resource_tpl#, title=Symmetric Mammoth Instance - 32 cores and 32 GB RAM, location=/mixin/resource_tpl/sym_mammoth/, attributes=SetCover{[Attribute{name=occi.compute.cores, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.architecture, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.speed, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.memory, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}]}}Resources Management
Setting Contextualization script
Three different options of providing a contextualization script to a resource when it has to be create are available.
1. By using the content of a public URL
String c = "https://appdb.egi.eu/storage/cs/vapp/15819120-7ee4-4b85-818a-d9bd755a61f0/devsec-init"; URL context = new URL(c);
2. By passing the content of a script via local file
String path = "/my/local/path/script.sh"; File f = new File(path);
3. Providing directly the content of the script in the code (deprecated option)
String gcube_script = "#!/bin/bash \n" + "export SCOPE=devsec \n"+ "export INFRA_NAME=gcube \n"+ "export LOCATION=pisa \n"+ "export COUNTRY=IT \n"+ "/usr/local/bin/smartgears-setup";
Creating and describing resource
To create a compute it is necessary to specify the OS and Resources templates for the image.
Mixin os = jocci.getOsTemplateList().get(0); //first os template avilable Mixin resource = jocci.getResourcesTemplateList().get(0); //first resource template available in the list
Alternatively we can specify directly the Virtual Appliance we want to refer to :
Mixin os = null; for (int i=0; i < jocci.getOsTemplateList().size(); i++) { os = jocci.getOsTemplateList().get(i);//gcube OS template available in the list if (os.getTerm().equals("uuid_gcubesmartexecutorv4_fedcloud_warg_142")) break; }
Depending on the typology of contextualization script to upload, it is possible to use four different approacches to create the resource:
jocci.createComputeAndDescribe(os,resource,"Resource name"); //no context script jocci.createComputeAndDescribe(os,resource,"Resource name",context); //Script from URI jocci.createComputeAndDescribe(os,resource,"Resource name",f); //Script from file jocci.createComputeAndDescribe(os,resource,"Resource name",gcube_script); //String option
An example of output is the following:
Creating compute resource...
Mixin:
Category: uuid_ubuntu_server_12_04_lts_fedcloud_dukan_81;scheme="http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#";class="mixin";title="Ubuntu-Server-12.04-LTS@fedcloud-dukan";rel="http://schemas.ogf.org/occi/infrastructure#os_tpl";location="/mixin/os_tpl/uuid_ubuntu_server_12_04_lts_fedcloud_dukan_81/"
Category: mammoth;scheme="http://schema.fedcloud.egi.eu/occi/infrastructure/resource_tpl#";class="mixin";title="Mammoth Instance - 8 cores and 32 GB RAM";rel="http://schemas.ogf.org/occi/infrastructure#resource_tpl";location="/mixin/resource_tpl/mammoth/";attributes="occi.compute.architecture{immutable} occi.compute.cores{immutable} occi.compute.memory{immutable} occi.compute.speed{immutable}"
#!/bin/bash
# Variables that are used to configure the smartexecutor instance
# The HOST variable is not mandatory. The machine FQDN will be used instead
#export HOST=<instance hostname>
export SCOPE=devsec
export INFRA_NAME=gcube
export LOCATION=pisa
export COUNTRY=IT
# This script is installed into the VM instance
/usr/local/bin/smartgears-setup
Created compute instance at location: 'https://carach5.ics.muni.cz:11443/compute/64586'.
Description:
Category: compute;scheme="http://schemas.ogf.org/occi/infrastructure#";class="kind";location="/compute/"
Category: uuid_ubuntu_server_12_04_lts_fedcloud_dukan_81;scheme="http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#";class="mixin";location="/mixin/os_tpl/uuid_ubuntu_server_12_04_lts_fedcloud_dukan_81/"
Category: compute;scheme="http://opennebula.org/occi/infrastructure#";class="mixin";location="/mixin/compute/"
Category: mammoth;scheme="http://schema.fedcloud.egi.eu/occi/infrastructure/resource_tpl#";class="mixin";location="/mixin/resource_tpl/mammoth/"
Category: user_data;scheme="http://schemas.openstack.org/compute/instance#";class="mixin";location="/mixin/user_data/"
Category: public_key;scheme="http://schemas.openstack.org/instance/credentials#";class="mixin";location="/mixin/public_key/"
X-OCCI-Attribute: occi.compute.cores=8
X-OCCI-Attribute: occi.compute.memory=32.0
X-OCCI-Attribute: occi.compute.state="waiting"
X-OCCI-Attribute: occi.core.id=64586
X-OCCI-Attribute: occi.core.summary="Instantiated with rOCCI-server on Wed, 11 Nov 2015 14:47:37 +0100."
X-OCCI-Attribute: occi.core.title="VM name"
X-OCCI-Attribute: org.opennebula.compute.cpu=8.0
X-OCCI-Attribute: org.opennebula.compute.id=64586
X-OCCI-Attribute: org.openstack.compute.user_data="IyEvYmluL2Jhc2gKIyBWYXJpYWJsZXMgdGhhdCBhcmUgdXNlZCB0byBjb25maWd1cmUgdGhlIHNtYXJ0ZXhlY3V0b3IgaW5zdGFuY2UKIyBUaGUgSE9TVCB2YXJpYWJsZSBpcyBub3QgbWFuZGF0b3J5LiBUaGUgbWFjaGluZSBGUUROIHdpbGwgYmUgdXNlZCBpbnN0ZWFkCiNleHBvcnQgSE9TVD08aW5zdGFuY2UgaG9zdG5hbWU+CmV4cG9ydCBTQ09QRT1kZXZzZWMKZXhwb3J0IElORlJBX05BTUU9Z2N1YmUKZXhwb3J0IExPQ0FUSU9OPXBpc2EKZXhwb3J0IENPVU5UUlk9SVQKCiMgVGhpcyBzY3JpcHQgaXMgaW5zdGFsbGVkIGludG8gdGhlIFZNIGluc3RhbmNlCi91c3IvbG9jYWwvYmluL3NtYXJ0Z2VhcnMtc2V0dXAK"
X-OCCI-Attribute: org.openstack.credentials.publickey.data="ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrbZDYeDVxudawo6FVEPOmBVcRhf/DU1z9v82bvPWMxu/wMqJ0HriUCFYj5yfsfT3eKqlzY1K3HVaS8jQW2JYrR9F5HvDlg/dR4LoLfmzKpB9vuAbpwizEg+OkPAtApJPITasj7xQrCFHtqhVTzkcqbuU4gXa5hX7rVpBWzwCP25wMHima8Y8WcxkSGGBDTxhE0okjUhc07ExTlxsgb/2PX/ziqCjSrjkNaxMIAU4dN0alb7JKjTiS9hYLK3yz2rN91KLMlalO5SgQ8xTRml8PXZAoK86MG2KLBH6VKFsu6mvc16Hp9KNlx2pw9Q9XepVG/Vgm9evEwRjSvqqR8w9p nunzioandreagalante@nunzioandreagalante-Spring-Peak"
Link: </network/24>;rel="http://schemas.ogf.org/occi/infrastructure#network";self="/link/networkinterface/compute_64586_nic_0";category="http://schemas.ogf.org/occi/infrastructure#networkinterface http://opennebula.org/occi/infrastructure#networkinterface http://schemas.ogf.org/occi/infrastructure/networkinterface#ipnetworkinterface";occi.core.id="compute_64586_nic_0";occi.core.source="/compute/64586";occi.core.target="/network/24";occi.core.title="public";occi.networkinterface.address="147.228.242.244";occi.networkinterface.interface="eth0";occi.networkinterface.mac="02:00:93:e4:f2:f4";occi.networkinterface.state="inactive";org.opennebula.networkinterface.bridge="onebr0";
Link: </storage/779>;rel="http://schemas.ogf.org/occi/infrastructure#storage";self="/link/storagelink/compute_64586_disk_0";category="http://schemas.ogf.org/occi/infrastructure#storagelink http://opennebula.org/occi/infrastructure#storagelink";occi.core.id="compute_64586_disk_0";occi.core.source="/compute/64586";occi.core.target="/storage/779";occi.core.title="800f345f-5278-5523-a1dc-8a98476006f8";occi.storagelink.deviceid="/dev/vda";occi.storagelink.state="inactive";
Waiting for compute to become active...
.
.
.
.
Compute active!Even if you choose to use the gcube_smart_executor VA as OS Mixin template, once that the GHN compute has been created, it becomes registered to d4Science infrastructure and can be monitored on https://dev.d4science.org/web/guest/monitor
Starting resource
//do not use after createResourceAndDescribe jocci.startResource(new URI("Resource Location URI")); //jocci.startResource(new URI("https://carach5.ics.muni.cz:11443/compute/64586"));
Sample output:
Starting created compute...OK!
Stopping resource
jocci.stopResource(new URI("Resource Location URI")); //jocci.stopResource(new URI("https://carach5.ics.muni.cz:11443/compute/64586"));
Sample output:
Stopping created compute...OK!
Deleting resource
jocci.deleteResource(new URI("Resource Location URI")); //jocci.deleteResource(new URI("https://carach5.ics.muni.cz:11443/compute/64586"));
Sample output:
Deleting created compute...OK!
