Difference between revisions of "Statistical Algorithms Importer: Python Project FAQ"

From Gcube Wiki
Jump to: navigation, search
(→‎How to use the input file parameters)
(→‎How to use the enumerated parameters)
 
(22 intermediate revisions by 2 users not shown)
Line 3: Line 3:
 
|}
 
|}
  
−
F.A.Q. of Statistical Algorithms Importer (SAI), here are common mistakes we have found in Python Project.
+
F.A.Q. of [[Statistical_Algorithms_Importer|Statistical Algorithms Importer (SAI)]], here are common mistakes we have found in Python Project.
  
 +
== How to use the enumerated parameters ==
 +
:For example we consider this parameter:
 +
[[Image:StatisticalAlgorithmsImporter_InputBasicParamEnumertated0.png|thumb|center|700px|Enumerated Param, SAI]]
  
−
== Matplotlib ==
+
:As you can see, it is a Enumerated parameter and it is specified using a pipe-separated list of strings. The parameter will be transformed into a controlled vocabulary and will be displayed via a drop-down list in DataMiner.
−
Matplotlib require the $DISPLAY environment variable which means a running X server. DataMiner service do not allow a running X server session. So your code must be modified:
+
[[Image:StatisticalAlgorithmsImporter_InputBasicParamEnumertated1.png|thumb|center|700px|Enumerated Param in DataMiner, SAI]]
  
−
<source lang='python'>
+
== How to use the input and output file parameters ==
−
#
+
−
# author Giancarlo Panichi
+
−
#
+
−
# Write file image.png
+
−
#
+
−
import matplotlib as mpl
+
−
mpl.use('Agg')
+
−
import matplotlib.pyplot as plt
+
−
import numpy as np
+
−
....
+
−
plt.imshow(image)
+
−
plt.savefig('image.png')
+
−
</source>
+
−
 
+
−
 
+
−
== How to use the input file parameters ==
+
 
:For example we consider Phonebook algorithm:
 
:For example we consider Phonebook algorithm:
 
[[Image:StatisticalAlgorithmsImporter_Phonebook0.png|thumb|center|700px|Phonebook, SAI]]
 
[[Image:StatisticalAlgorithmsImporter_Phonebook0.png|thumb|center|700px|Phonebook, SAI]]
Line 62: Line 49:
 
def load_numbers(numbers,filename):
 
def load_numbers(numbers,filename):
 
     with open(filename,"r") as f:
 
     with open(filename,"r") as f:
−
        lines = f.readlines()
+
         for in_line in f:             
−
        lines
+
−
         for in_line in lines:             
+
 
             [name,number] = string.split(in_line,",")
 
             [name,number] = string.split(in_line,",")
 
             numbers[name] = number
 
             numbers[name] = number
Line 104: Line 89:
 
[[File:Phonebook.zip|Phonebook.zip]]
 
[[File:Phonebook.zip|Phonebook.zip]]
  
 +
== Matplotlib ==
 +
Matplotlib require the $DISPLAY environment variable which means a running X server. DataMiner service do not allow a running X server session. So your code must be modified:
 +
 +
<source lang='python'>
 +
#
 +
# author Giancarlo Panichi
 +
#
 +
# Write file image.png
 +
#
 +
import matplotlib as mpl
 +
mpl.use('Agg')
 +
import matplotlib.pyplot as plt
 +
import numpy as np
 +
....
 +
plt.imshow(image)
 +
plt.savefig('image.png')
 +
</source>
 +
 +
== Urllib ==
 +
On DataMiner services is installed Urllib3, see:
 +
 +
* http://urllib3.readthedocs.io/en/latest/user-guide.html
 +
 +
<source lang='python'>
 +
#
 +
# author Giancarlo Panichi
 +
#
 +
# Request Url
 +
#
 +
import urllib3
 +
http = urllib3.PoolManager()
 +
r = http.request('GET', 'http://httpbin.org/robots.txt')
 +
 +
print(r.status)
 +
print(r.data)
 +
 +
 +
</source>
 +
 +
 +
== StorageHub ==
 +
:[[StorageHub REST API|StorageHub]] is the service for accessing to the user's D4Science Workspace. Below we show the StorageHubFacilityPython algorithm, it exhibits the interactions with StorageHub through its Rest API:
 +
[[Image:StorageHubFacilityPython0.png|thumb|center|750px|StorageHub Facility Python, SAI]]
 +
 +
:Indicates the I/O parameters:
 +
[[Image:StorageHubFacilityPython1.png|thumb|center|750px|StorageHub Facility Python I/O parameters, SAI]]
 +
 +
:Indicates the python version:
 +
[[Image:StorageHubFacilityPython2.png|thumb|center|750px|StorageHub Facility Python version, SAI]]
 +
 +
:Indicates the main code python:
 +
[[Image:StorageHubFacilityPython3.png|thumb|center|750px|StorageHub Facility Python Code, SAI]]
 +
 +
 +
:DataMiner result:
 +
[[Image:StorageHubFacilityPython4.png|thumb|center|750px|StorageHub Facility Python in DataMiner, SAI]]
 +
 +
:This algorithm shows 5 types of interactions with StorageHub:
 +
:* Get Root Info
 +
:* Get Item Info (requires an itemId as argument1)
 +
:* Get Root Children
 +
:* Get Item Children (requires an itemId as argument1)
 +
:* Item Download (requires an itemId as argument1)
 +
 +
:Python source code of StorageHubFacilityPython:
 +
:[[File:StorageHubFacilityPython.zip|StorageHubFacilityPython.zip]]
 +
 +
:View the demo in DataMiner:
 +
:[https://services.d4science.org/group/rprototypinglab/data-miner?OperatorId=org.gcube.dataanalysis.wps.statisticalmanager.synchserver.mappedclasses.transducerers.STORAGEHUBFACILITYPYTHON StorageHub Facility Python]
  
  
 
[[Category:Statistical Algorithms Importer]]
 
[[Category:Statistical Algorithms Importer]]

Latest revision as of 12:47, 12 February 2021

F.A.Q. of Statistical Algorithms Importer (SAI), here are common mistakes we have found in Python Project.

How to use the enumerated parameters

For example we consider this parameter:
Enumerated Param, SAI
As you can see, it is a Enumerated parameter and it is specified using a pipe-separated list of strings. The parameter will be transformed into a controlled vocabulary and will be displayed via a drop-down list in DataMiner.
Enumerated Param in DataMiner, SAI

How to use the input and output file parameters

For example we consider Phonebook algorithm:
Phonebook, SAI
Phonebook, SAI
Phonebook, SAI


DataMiner show the Phonebook algorithm in this way:
Phonebook in DataMiner interface, SAI


Python script code in sample:
#
# author Giancarlo Panichi
#
# Phonebook
# 
import sys
import string
 
def print_numbers(numbers):
    print "Telephone Numbers:"
    for x in numbers.keys():
        value="Name: "+x+" Number: "+numbers[x]
        print value,
 
def lookup_number(numbers,name):
    if numbers.has_key(name):
        return "For "+name+" the number is "+numbers[name]
    else:
        return "For "+name+" was not found"
 
def load_numbers(numbers,filename):
    with open(filename,"r") as f:
        for in_line in f:            
            [name,number] = string.split(in_line,",")
            numbers[name] = number
 
def save_result(result,filename):
    with open(filename,"w") as f:
        f.write(result)
 
phonebookFile=sys.argv[1] 
name=sys.argv[2]
phone_list = {}
 
load_numbers(phone_list,phonebookFile)
print_numbers(phone_list)
number=lookup_number(phone_list,name)
print number
save_result(number,"result.txt")


phonebook.csv:
Giancarlo,0123456789
Gianpaolo,9876541234
Paolo,1231410414


phonebook result with Giancarlo:
For Giancarlo the number is 0123456789


Phonebook code:

File:Phonebook.zip

Matplotlib

Matplotlib require the $DISPLAY environment variable which means a running X server. DataMiner service do not allow a running X server session. So your code must be modified: 

#
# author Giancarlo Panichi
#
# Write file image.png
# 
import matplotlib as mpl
mpl.use('Agg')
import matplotlib.pyplot as plt
import numpy as np
....
plt.imshow(image)
plt.savefig('image.png')

Urllib

On DataMiner services is installed Urllib3, see: 

#
# author Giancarlo Panichi
#
# Request Url
# 
import urllib3
http = urllib3.PoolManager()
r = http.request('GET', 'http://httpbin.org/robots.txt')
 
print(r.status)
print(r.data)


StorageHub

StorageHub is the service for accessing to the user's D4Science Workspace. Below we show the StorageHubFacilityPython algorithm, it exhibits the interactions with StorageHub through its Rest API:
StorageHub Facility Python, SAI
Indicates the I/O parameters:
StorageHub Facility Python I/O parameters, SAI
Indicates the python version:
StorageHub Facility Python version, SAI
Indicates the main code python:
StorageHub Facility Python Code, SAI


DataMiner result:
StorageHub Facility Python in DataMiner, SAI
This algorithm shows 5 types of interactions with StorageHub:
  • Get Root Info
  • Get Item Info (requires an itemId as argument1)
  • Get Root Children
  • Get Item Children (requires an itemId as argument1)
  • Item Download (requires an itemId as argument1)
Python source code of StorageHubFacilityPython:
File:StorageHubFacilityPython.zip
View the demo in DataMiner:
StorageHub Facility Python
Retrieved from "https://wiki.gcube-system.org/index.php?title=Statistical_Algorithms_Importer:_Python_Project_FAQ&oldid=33839"