Difference between revisions of "Talk:OCCI library"

From Gcube Wiki
Jump to: navigation, search
(→‎Creating and describing resource)
(→‎Endpoint Connection)
Line 48: Line 48:
 
</source>
 
</source>
  
 +
Sample output:
 
<source lang="html4strict">
 
<source lang="html4strict">
 
Connected to endpoint: https://carach5.ics.muni.cz:11443/
 
Connected to endpoint: https://carach5.ics.muni.cz:11443/

Revision as of 12:42, 12 November 2015

Scope

This library has been developed in order to allow the integration of gcube with EGI FedCloud infrastructure. In particular, the usage of the library will facilitate the interaction with the sites exposing OCCI just by manipulating APIs.

Usage

Maven coordinates

The maven artifact coordinates are: 

<dependency>
  <groupId>org.gcube.dvos</groupId>
  <artifactId>occi-library</artifactId>
  <version>1.0.0-SNAPSHOT</version> 
</dependency>

Check on Nexus for the latest version.

APIs

In the following the functionalities proposed by the library are shown.

VOMS proxy generation

In order to access the EGI FedCloud infrastructure it is necessary to create a VOMS proxy certificate. Open a shell and run the following command: 

$voms-proxy-init --voms fedcloud.egi.eu --rfc --dont-verify-ac

Later it is necessary to digit the own EGI identity grid password. 

Enter GRID pass phrase for this identity:
Your identity: /C=IT/O=INFN/OU=Personal Certificate/L=ENGINEERING RDLAB/CN=Nunzio Andrea Galante
Creating temporary proxy ........................ Done
Contacting  voms1.egee.cesnet.cz:15002 [/DC=org/DC=terena/DC=tcs/OU=Domain Control Validated/CN=voms1.egee.cesnet.cz] "fedcloud.egi.eu" Done
Creating proxy ...................................... Done
 
Your proxy is valid until Thu Nov 12 01:13:17 2015

Authentication

The library provides both VOMS and X509 access authentication methods. A list of sites exposing OCCI and endorsed by FedCloud VO is available at Appdb repository. 

String a = X509CredentialManager.createProxy("/tmp/your_proxy.pem", ""); //2nd level proxy
FedcloudOCCI jocci = new FedcloudOCCI("endpoint");
//FedcloudOCCI jocci = new FedcloudOCCI("https://carach5.ics.muni.cz:11443");
jocci.setCertificate(a);
jocci.setTrustStore("CA Path certificates");

Endpoint Connection

jocci.connect();

Sample output: 

Connected to endpoint: https://carach5.ics.muni.cz:11443/

Listing own compute resources

jocci.getComputeList();
Computes Location: 
https://carach5.ics.muni.cz:11443/compute/64586

Listing storage resources

jocci.getNetList();
Storage Locations: 
https://carach5.ics.muni.cz:11443/storage/774;
https://carach5.ics.muni.cz:11443/storage/775;
https://carach5.ics.muni.cz:11443/storage/776;
https://carach5.ics.muni.cz:11443/storage/777;
https://carach5.ics.muni.cz:11443/storage/779;

Listing network resources

jocci.getStorageList();
Network Locations: 
https://carach5.ics.muni.cz:11443/network/6
https://carach5.ics.muni.cz:11443/network/8
https://carach5.ics.muni.cz:11443/network/17
https://carach5.ics.muni.cz:11443/network/24

Getting OS templates

jocci.getOsTemplateList();
17 AVAILABLE OS TEMPLATES MIXINS
Category{term=uuid_umd_sl6_fedcloud_warg_138, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=UMD-SL6@fedcloud-warg, location=/mixin/os_tpl/uuid_umd_sl6_fedcloud_warg_138/, attributes=SetCover{[]}}
Category{term=uuid_egi_centos_6_fedcloud_warg_130, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=EGI-CentOS-6@fedcloud-warg, location=/mixin/os_tpl/uuid_egi_centos_6_fedcloud_warg_130/, attributes=SetCover{[]}}
Category{term=uuid_egi_fedcloud_clients_fedcloud_warg_128, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=EGI-FedCloud-Clients@fedcloud-warg, location=/mixin/os_tpl/uuid_egi_fedcloud_clients_fedcloud_warg_128/, attributes=SetCover{[]}}
Category{term=uuid_egi_ubuntu_server_14_04_lts_fedcloud_warg_131, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=EGI-Ubuntu-Server-14.04-LTS@fedcloud-warg, location=/mixin/os_tpl/uuid_egi_ubuntu_server_14_04_lts_fedcloud_warg_131/, attributes=SetCover{[]}}
Category{term=uuid_chipster_ubuntu_server_12_04_lts_fedcloud_dukan_96, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=Chipster-Ubuntu-Server-12.04-LTS@fedcloud-dukan, location=/mixin/os_tpl/uuid_chipster_ubuntu_server_12_04_lts_fedcloud_dukan_96/, attributes=SetCover{[]}}
Category{term=uuid_egi_compss_lofar_debian_7_0_x86_64_0001_fedcloud_dukan_113, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=EGI-COMPSs-Lofar-Debian-7.0-x86_64-0001@fedcloud-dukan, location=/mixin/os_tpl/uuid_egi_compss_lofar_debian_7_0_x86_64_0001_fedcloud_dukan_113/, attributes=SetCover{[]}}
Category{term=uuid_centos_6_5_minimal_fedcloud_dukan_83, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=CentOS-6.5-minimal@fedcloud-dukan, location=/mixin/os_tpl/uuid_centos_6_5_minimal_fedcloud_dukan_83/, attributes=SetCover{[]}}
Category{term=uuid_transplant_base_fedcloud_warg_132, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=transPLANT-Base@fedcloud-warg, location=/mixin/os_tpl/uuid_transplant_base_fedcloud_warg_132/, attributes=SetCover{[]}}
Category{term=uuid_ubuntu_server_14_04_lts_fedcloud_dukan_84, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=Ubuntu-Server-14.04-LTS@fedcloud-dukan, location=/mixin/os_tpl/uuid_ubuntu_server_14_04_lts_fedcloud_dukan_84/, attributes=SetCover{[]}}
Category{term=uuid_egi_pmes_14_ubuntu_15_04_x86_64_0001_fedcloud_warg_108, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=EGI-PMES-14-Ubuntu-15.04-x86_64-0001@fedcloud-warg, location=/mixin/os_tpl/uuid_egi_pmes_14_ubuntu_15_04_x86_64_0001_fedcloud_warg_108/, attributes=SetCover{[]}}
Category{term=uuid_dci_bridge_fedcloud_dukan_109, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=DCI-Bridge@fedcloud-dukan, location=/mixin/os_tpl/uuid_dci_bridge_fedcloud_dukan_109/, attributes=SetCover{[]}}
Category{term=uuid_egi_ubuntu_server_12_04_lts_fedcloud_warg_129, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=EGI-Ubuntu-Server-12.04-LTS@fedcloud-warg, location=/mixin/os_tpl/uuid_egi_ubuntu_server_12_04_lts_fedcloud_warg_129/, attributes=SetCover{[]}}
Category{term=uuid_scientific_linux_6_5_fedcloud_dukan_82, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=Scientific-Linux-6.5@fedcloud-dukan, location=/mixin/os_tpl/uuid_scientific_linux_6_5_fedcloud_dukan_82/, attributes=SetCover{[]}}
Category{term=uuid_cernvm_3_3_0_40gb_fedcloud_dukan_80, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=CERNVM-3.3.0-40GB@fedcloud-dukan, location=/mixin/os_tpl/uuid_cernvm_3_3_0_40gb_fedcloud_dukan_80/, attributes=SetCover{[]}}
Category{term=uuid_moinmoinwiki_fedcloud_warg_124, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=MoinMoinWiki@fedcloud-warg, location=/mixin/os_tpl/uuid_moinmoinwiki_fedcloud_warg_124/, attributes=SetCover{[]}}
Category{term=uuid_gcubesmartexecutor_fedcloud_warg_139, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=GCubeSmartExecutor@fedcloud-warg, location=/mixin/os_tpl/uuid_gcubesmartexecutor_fedcloud_warg_139/, attributes=SetCover{[]}}
Category{term=uuid_ubuntu_server_12_04_lts_fedcloud_dukan_81, scheme=http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#, title=Ubuntu-Server-12.04-LTS@fedcloud-dukan, location=/mixin/os_tpl/uuid_ubuntu_server_12_04_lts_fedcloud_dukan_81/, attributes=SetCover{[]}}

Getting Resource templates

jocci.getResourcesTemplateList();
7 AVAILABLE RESOURCES TEMPLATES MIXINS
Category{term=mammoth, scheme=http://schema.fedcloud.egi.eu/occi/infrastructure/resource_tpl#, title=Mammoth Instance - 8 cores and 32 GB RAM, location=/mixin/resource_tpl/mammoth/, attributes=SetCover{[Attribute{name=occi.compute.cores, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.architecture, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.speed, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.memory, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}]}}
Category{term=large, scheme=http://schema.fedcloud.egi.eu/occi/infrastructure/resource_tpl#, title=Large Instance - 4 cores and 8 GB RAM, location=/mixin/resource_tpl/large/, attributes=SetCover{[Attribute{name=occi.compute.cores, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.architecture, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.speed, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.memory, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}]}}
Category{term=small, scheme=http://schema.fedcloud.egi.eu/occi/infrastructure/resource_tpl#, title=Small Instance - 1 core and 2 GB RAM, location=/mixin/resource_tpl/small/, attributes=SetCover{[Attribute{name=occi.compute.cores, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.architecture, required=false, immutable=false, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.speed, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.memory, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}]}}
Category{term=medium, scheme=http://schema.fedcloud.egi.eu/occi/infrastructure/resource_tpl#, title=Medium Instance - 2 cores and 4 GB RAM, location=/mixin/resource_tpl/medium/, attributes=SetCover{[Attribute{name=occi.compute.cores, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.architecture, required=false, immutable=false, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.speed, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.memory, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}]}}
Category{term=goliath, scheme=http://schema.fedcloud.egi.eu/occi/infrastructure/resource_tpl#, title=Goliath Instance - 16 cores and 64 GB RAM, location=/mixin/resource_tpl/goliath/, attributes=SetCover{[Attribute{name=occi.compute.cores, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.architecture, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.speed, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.memory, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}]}}
Category{term=extra_large, scheme=http://schema.fedcloud.egi.eu/occi/infrastructure/resource_tpl#, title=Extra Large Instance - 4 cores and 16 GB RAM, location=/mixin/resource_tpl/extra_large/, attributes=SetCover{[Attribute{name=occi.compute.cores, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.architecture, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.speed, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.memory, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}]}}
Category{term=sym_mammoth, scheme=http://schema.fedcloud.egi.eu/occi/infrastructure/resource_tpl#, title=Symmetric Mammoth Instance - 32 cores and 32 GB RAM, location=/mixin/resource_tpl/sym_mammoth/, attributes=SetCover{[Attribute{name=occi.compute.cores, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.architecture, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.speed, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}, Attribute{name=occi.compute.memory, required=false, immutable=true, type=null, pattern=.*, defaultValue=null, description=null}]}}

Setting Contextualization script

Three different options of providing a contextualization script to a resource when it has to be create are available.

1. By using the content of a public URL 

String c = "https://appdb.egi.eu/storage/cs/vapp/15819120-7ee4-4b85-818a-d9bd755a61f0/devsec-init";
URL context = new URL(c);

2. By passing the content of a script via local file 

String path = "/my/local/path/script.sh";
File f = new File(path);

3. Providing directly the content of the script in the code (deprecated option) 

String gcube_script = "#!/bin/bash \n"  + 
	              "export SCOPE=devsec \n"+
	    	      "export INFRA_NAME=gcube \n"+
	              "export LOCATION=pisa \n"+
	    	      "export COUNTRY=IT \n"+
	    	      "/usr/local/bin/smartgears-setup";

Creating and describing resource

To create a compute it is necessary to specify the OS and Resources templates for the image. 

Mixin os = jocci.getOsTemplateList().get(0); //first os template avilable
Mixin resource = jocci.getResourcesTemplateList().get(0); //first resource template available in the list

Alternatively we can specify directly the Virtual Appliance we want to refer to : 

	      Mixin os = null;
	      for (int i=0; i < jocci.getOsTemplateList().size(); i++) {
			os = jocci.getOsTemplateList().get(i);//gcube OS template available in the list
			if (os.getTerm().equals("uuid_gcubesmartexecutorv4_fedcloud_warg_142"))
				break;
		}

Depending on the typology of contextualization script to upload, it is possible to use four different approacches to create the resource: 

jocci.createComputeAndDescribe(os,resource,"Resource name"); //no context script
jocci.createComputeAndDescribe(os,resource,"Resource name",context); //Script from URI
jocci.createComputeAndDescribe(os,resource,"Resource name",f); //Script from file
jocci.createComputeAndDescribe(os,resource,"Resource name",gcube_script); //String option

An example of output is the following: 

Creating compute resource...
Mixin:
Category: uuid_ubuntu_server_12_04_lts_fedcloud_dukan_81;scheme="http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#";class="mixin";title="Ubuntu-Server-12.04-LTS@fedcloud-dukan";rel="http://schemas.ogf.org/occi/infrastructure#os_tpl";location="/mixin/os_tpl/uuid_ubuntu_server_12_04_lts_fedcloud_dukan_81/"
Category: mammoth;scheme="http://schema.fedcloud.egi.eu/occi/infrastructure/resource_tpl#";class="mixin";title="Mammoth Instance - 8 cores and 32 GB RAM";rel="http://schemas.ogf.org/occi/infrastructure#resource_tpl";location="/mixin/resource_tpl/mammoth/";attributes="occi.compute.architecture{immutable} occi.compute.cores{immutable} occi.compute.memory{immutable} occi.compute.speed{immutable}"
 
#!/bin/bash
# Variables that are used to configure the smartexecutor instance
# The HOST variable is not mandatory. The machine FQDN will be used instead
#export HOST=<instance hostname>
export SCOPE=devsec
export INFRA_NAME=gcube
export LOCATION=pisa
export COUNTRY=IT
 
# This script is installed into the VM instance
/usr/local/bin/smartgears-setup
 
Created compute instance at location: 'https://carach5.ics.muni.cz:11443/compute/64586'.
Description:
Category: compute;scheme="http://schemas.ogf.org/occi/infrastructure#";class="kind";location="/compute/"
Category: uuid_ubuntu_server_12_04_lts_fedcloud_dukan_81;scheme="http://occi.carach5.ics.muni.cz/occi/infrastructure/os_tpl#";class="mixin";location="/mixin/os_tpl/uuid_ubuntu_server_12_04_lts_fedcloud_dukan_81/"
Category: compute;scheme="http://opennebula.org/occi/infrastructure#";class="mixin";location="/mixin/compute/"
Category: mammoth;scheme="http://schema.fedcloud.egi.eu/occi/infrastructure/resource_tpl#";class="mixin";location="/mixin/resource_tpl/mammoth/"
Category: user_data;scheme="http://schemas.openstack.org/compute/instance#";class="mixin";location="/mixin/user_data/"
Category: public_key;scheme="http://schemas.openstack.org/instance/credentials#";class="mixin";location="/mixin/public_key/"
X-OCCI-Attribute: occi.compute.cores=8
X-OCCI-Attribute: occi.compute.memory=32.0
X-OCCI-Attribute: occi.compute.state="waiting"
X-OCCI-Attribute: occi.core.id=64586
X-OCCI-Attribute: occi.core.summary="Instantiated with rOCCI-server on Wed, 11 Nov 2015 14:47:37 +0100."
X-OCCI-Attribute: occi.core.title="VM name"
X-OCCI-Attribute: org.opennebula.compute.cpu=8.0
X-OCCI-Attribute: org.opennebula.compute.id=64586
X-OCCI-Attribute: org.openstack.compute.user_data="IyEvYmluL2Jhc2gKIyBWYXJpYWJsZXMgdGhhdCBhcmUgdXNlZCB0byBjb25maWd1cmUgdGhlIHNtYXJ0ZXhlY3V0b3IgaW5zdGFuY2UKIyBUaGUgSE9TVCB2YXJpYWJsZSBpcyBub3QgbWFuZGF0b3J5LiBUaGUgbWFjaGluZSBGUUROIHdpbGwgYmUgdXNlZCBpbnN0ZWFkCiNleHBvcnQgSE9TVD08aW5zdGFuY2UgaG9zdG5hbWU+CmV4cG9ydCBTQ09QRT1kZXZzZWMKZXhwb3J0IElORlJBX05BTUU9Z2N1YmUKZXhwb3J0IExPQ0FUSU9OPXBpc2EKZXhwb3J0IENPVU5UUlk9SVQKCiMgVGhpcyBzY3JpcHQgaXMgaW5zdGFsbGVkIGludG8gdGhlIFZNIGluc3RhbmNlCi91c3IvbG9jYWwvYmluL3NtYXJ0Z2VhcnMtc2V0dXAK"
X-OCCI-Attribute: org.openstack.credentials.publickey.data="ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrbZDYeDVxudawo6FVEPOmBVcRhf/DU1z9v82bvPWMxu/wMqJ0HriUCFYj5yfsfT3eKqlzY1K3HVaS8jQW2JYrR9F5HvDlg/dR4LoLfmzKpB9vuAbpwizEg+OkPAtApJPITasj7xQrCFHtqhVTzkcqbuU4gXa5hX7rVpBWzwCP25wMHima8Y8WcxkSGGBDTxhE0okjUhc07ExTlxsgb/2PX/ziqCjSrjkNaxMIAU4dN0alb7JKjTiS9hYLK3yz2rN91KLMlalO5SgQ8xTRml8PXZAoK86MG2KLBH6VKFsu6mvc16Hp9KNlx2pw9Q9XepVG/Vgm9evEwRjSvqqR8w9p nunzioandreagalante@nunzioandreagalante-Spring-Peak"
Link: </network/24>;rel="http://schemas.ogf.org/occi/infrastructure#network";self="/link/networkinterface/compute_64586_nic_0";category="http://schemas.ogf.org/occi/infrastructure#networkinterface http://opennebula.org/occi/infrastructure#networkinterface http://schemas.ogf.org/occi/infrastructure/networkinterface#ipnetworkinterface";occi.core.id="compute_64586_nic_0";occi.core.source="/compute/64586";occi.core.target="/network/24";occi.core.title="public";occi.networkinterface.address="147.228.242.244";occi.networkinterface.interface="eth0";occi.networkinterface.mac="02:00:93:e4:f2:f4";occi.networkinterface.state="inactive";org.opennebula.networkinterface.bridge="onebr0";
Link: </storage/779>;rel="http://schemas.ogf.org/occi/infrastructure#storage";self="/link/storagelink/compute_64586_disk_0";category="http://schemas.ogf.org/occi/infrastructure#storagelink http://opennebula.org/occi/infrastructure#storagelink";occi.core.id="compute_64586_disk_0";occi.core.source="/compute/64586";occi.core.target="/storage/779";occi.core.title="800f345f-5278-5523-a1dc-8a98476006f8";occi.storagelink.deviceid="/dev/vda";occi.storagelink.state="inactive";
Waiting for compute to become active...
.
.
.
.
Compute active!

Even if you choose to use the gcube_smart_executor VA as OS Mixin template, once that the GHN compute has been created, it becomes registered to d4Science infrastructure and can be monitored on https://dev.d4science.org/web/guest/monitor

Starting resource

//do not use after createResourceAndDescribe
jocci.startResource(new URI("Resource Location URI"));
//jocci.startResource(new URI("https://carach5.ics.muni.cz:11443/compute/64586"));
Starting created compute...OK!

Stopping resource

jocci.stopResource(new URI("Resource Location URI"));
//jocci.stopResource(new URI("https://carach5.ics.muni.cz:11443/compute/64586"));
Stopping created compute...OK!

Deleting resource

jocci.deleteResource(new URI("Resource Location URI"));
//jocci.deleteResource(new URI("https://carach5.ics.muni.cz:11443/compute/64586"));
Deleting created compute...OK!
Retrieved from "https://wiki.gcube-system.org/index.php?title=Talk:OCCI_library&oldid=23447"