Difference between revisions of "Statistical Algorithms Importer: Python Project FAQ"

From Gcube Wiki
Jump to: navigation, search
(How to use the input file parameters)
(StorageHub)
(16 intermediate revisions by one other user not shown)
Line 3: Line 3:
 
|}
 
|}
  
F.A.Q. of Statistical Algorithms Importer (SAI), here are common mistakes we have found in Python Project.
+
F.A.Q. of [[Statistical_Algorithms_Importer|Statistical Algorithms Importer (SAI)]], here are common mistakes we have found in Python Project.
  
  
== Matplotlib ==
+
== How to use the input and output file parameters ==
Matplotlib require the $DISPLAY environment variable which means a running X server. DataMiner service do not allow a running X server session. So your code must be modified:
+
 
+
<source lang='python'>
+
#
+
# author Giancarlo Panichi
+
#
+
# Write file image.png
+
#
+
import matplotlib as mpl
+
mpl.use('Agg')
+
import matplotlib.pyplot as plt
+
import numpy as np
+
....
+
plt.imshow(image)
+
plt.savefig('image.png')
+
</source>
+
 
+
 
+
== How to use the input file parameters ==
+
 
:For example we consider Phonebook algorithm:
 
:For example we consider Phonebook algorithm:
 
[[Image:StatisticalAlgorithmsImporter_Phonebook0.png|thumb|center|700px|Phonebook, SAI]]
 
[[Image:StatisticalAlgorithmsImporter_Phonebook0.png|thumb|center|700px|Phonebook, SAI]]
Line 102: Line 83:
 
[[File:Phonebook.zip|Phonebook.zip]]
 
[[File:Phonebook.zip|Phonebook.zip]]
  
 +
== Matplotlib ==
 +
Matplotlib require the $DISPLAY environment variable which means a running X server. DataMiner service do not allow a running X server session. So your code must be modified:
 +
 +
<source lang='python'>
 +
#
 +
# author Giancarlo Panichi
 +
#
 +
# Write file image.png
 +
#
 +
import matplotlib as mpl
 +
mpl.use('Agg')
 +
import matplotlib.pyplot as plt
 +
import numpy as np
 +
....
 +
plt.imshow(image)
 +
plt.savefig('image.png')
 +
</source>
 +
 +
== Urllib ==
 +
On DataMiner services is installed Urllib3, see:
 +
 +
* http://urllib3.readthedocs.io/en/latest/user-guide.html
 +
 +
<source lang='python'>
 +
#
 +
# author Giancarlo Panichi
 +
#
 +
# Request Url
 +
#
 +
import urllib3
 +
http = urllib3.PoolManager()
 +
r = http.request('GET', 'http://httpbin.org/robots.txt')
 +
 +
print(r.status)
 +
print(r.data)
 +
 +
 +
</source>
 +
 +
 +
== StorageHub ==
 +
:[[StorageHub REST API|StorageHub]] is the service for accessing to the user's D4Science Workspace. Below we show the StorageHubFacilityPython algorithm, it exhibits the interactions with StorageHub through its Rest API:
 +
[[Image:StorageHubFacilityPython0.png|thumb|center|750px|StorageHub Facility Python, SAI]]
 +
 +
:Indicates the I/O parameters:
 +
[[Image:StorageHubFacilityPython1.png|thumb|center|750px|StorageHub Facility Python I/O parameters, SAI]]
 +
 +
:Indicates the python version:
 +
[[Image:StorageHubFacilityPython2.png|thumb|center|750px|StorageHub Facility Python version, SAI]]
 +
 +
:Indicates the main code python:
 +
[[Image:StorageHubFacilityPython3.png|thumb|center|750px|StorageHub Facility Python Code, SAI]]
 +
 +
 +
:DataMiner result:
 +
[[Image:StorageHubFacilityPython4.png|thumb|center|750px|StorageHub Facility Python in DataMiner, SAI]]
 +
 +
:This algorithm shows 5 types of interactions with StorageHub:
 +
:* Get Root Info
 +
:* Get Item Info (requires an itemId as argument1)
 +
:* Get Root Children
 +
:* Get Item Children (requires an itemId as argument1)
 +
:* Item Download (requires an itemId as argument1)
 +
 +
:Python source code of StorageHubFacilityPython:
 +
:[[File:StorageHubFacilityPython.zip|StorageHubFacilityPython.zip]]
 +
 +
:View the demo in DataMiner:
 +
:[https://services.d4science.org/group/rprototypinglab/data-miner?OperatorId=org.gcube.dataanalysis.wps.statisticalmanager.synchserver.mappedclasses.transducerers.STORAGEHUBFACILITYPYTHON StorageHub Facility Python]
  
  
 
[[Category:Statistical Algorithms Importer]]
 
[[Category:Statistical Algorithms Importer]]

Revision as of 16:00, 23 November 2018

F.A.Q. of Statistical Algorithms Importer (SAI), here are common mistakes we have found in Python Project.


How to use the input and output file parameters

For example we consider Phonebook algorithm:
Phonebook, SAI
Phonebook, SAI
Phonebook, SAI


DataMiner show the Phonebook algorithm in this way:
Phonebook in DataMiner interface, SAI


Python script code in sample:
#
# author Giancarlo Panichi
#
# Phonebook
# 
import sys
import string
 
def print_numbers(numbers):
    print "Telephone Numbers:"
    for x in numbers.keys():
        value="Name: "+x+" Number: "+numbers[x]
        print value,
 
def lookup_number(numbers,name):
    if numbers.has_key(name):
        return "For "+name+" the number is "+numbers[name]
    else:
        return "For "+name+" was not found"
 
def load_numbers(numbers,filename):
    with open(filename,"r") as f:
        for in_line in f:            
            [name,number] = string.split(in_line,",")
            numbers[name] = number
 
def save_result(result,filename):
    with open(filename,"w") as f:
        f.write(result)
 
phonebookFile=sys.argv[1] 
name=sys.argv[2]
phone_list = {}
 
load_numbers(phone_list,phonebookFile)
print_numbers(phone_list)
number=lookup_number(phone_list,name)
print number
save_result(number,"result.txt")


phonebook.csv:
Giancarlo,0123456789
Gianpaolo,9876541234
Paolo,1231410414


phonebook result with Giancarlo:
For Giancarlo the number is 0123456789


Phonebook code:

File:Phonebook.zip

Matplotlib

Matplotlib require the $DISPLAY environment variable which means a running X server. DataMiner service do not allow a running X server session. So your code must be modified:

#
# author Giancarlo Panichi
#
# Write file image.png
# 
import matplotlib as mpl
mpl.use('Agg')
import matplotlib.pyplot as plt
import numpy as np
....
plt.imshow(image)
plt.savefig('image.png')

Urllib

On DataMiner services is installed Urllib3, see:

#
# author Giancarlo Panichi
#
# Request Url
# 
import urllib3
http = urllib3.PoolManager()
r = http.request('GET', 'http://httpbin.org/robots.txt')
 
print(r.status)
print(r.data)


StorageHub

StorageHub is the service for accessing to the user's D4Science Workspace. Below we show the StorageHubFacilityPython algorithm, it exhibits the interactions with StorageHub through its Rest API:
StorageHub Facility Python, SAI
Indicates the I/O parameters:
StorageHub Facility Python I/O parameters, SAI
Indicates the python version:
StorageHub Facility Python version, SAI
Indicates the main code python:
StorageHub Facility Python Code, SAI


DataMiner result:
StorageHub Facility Python in DataMiner, SAI
This algorithm shows 5 types of interactions with StorageHub:
  • Get Root Info
  • Get Item Info (requires an itemId as argument1)
  • Get Root Children
  • Get Item Children (requires an itemId as argument1)
  • Item Download (requires an itemId as argument1)
Python source code of StorageHubFacilityPython:
File:StorageHubFacilityPython.zip
View the demo in DataMiner:
StorageHub Facility Python