The EPIKH Project (Exchange Programme to advance e-infrastructure Know-How) GILDA Virtual Machine Riccardo Rotondo (riccardo.rotondo@garr.it) Consortium GARR Joint CHAIN/EPIKH School for Application Porting to Science Gateways Beijing, 11.04.2012 www.epikh.eu
VM Requirements Virtualization VMWare (Player(free), Workstation, etc ) OVF image file available for other platforms Hosting Machine X86_64 CPU Arch. (Overrides 4Gb of RAM limit) 1,9 Gb Disk space for compressed VMX 4 GB RAM Virtul Machine 1,5 GB RAM 20 GB Virtual disk (about 3GB at first boot) NAT/Bridged Network (default and suggested) Beijing, Asia 4, 11.04.2012 2
VM Services Liferay 6.0.6 on Glassfish 3.1 Liferay plugins sdk MySql v5 VMWare tools (Enables directory sharing) NTP for time synchronization mini_httpd (for Robot Proxy simulation) JSAGA 0.9.15 GridEngine 1.4.5 Beijing, Asia 4, 11.04.2012 3
VM Users&Credentials (MySQL) Glassfish/Liferay database backend Beijing, Asia 4, 11.04.2012 4
VM Users&Credentials (Glassfish) Glassfish administrator user credentials Administration page Web Address Beijing, Asia 4, 11.04.2012 5
VM Users&Credentials (Liferay) Liferay home page Administrator user credentials Beijing, Asia 4, 11.04.2012 6
VM Users&Credentials (ROOT) VM ROOT user credentials Beijing, Asia 4, 11.04.2012 7
VM Users&Credentials (liferayadmin) VM liferayadmin user credentials Used to develop Liferay portlets This user has been included in sudoers users Beijing, Asia 4, 11.04.2012 8
Glassfish http://gildavm:8130 User: liferayadmin Password: liferayadmin Beijing, Asia 4, 11.04.2012 9
Glassfish (Control Panel) Configure the Application server: Resources,Pool DB Connections, Log, other settings Beijing, Asia 4, 11.04.2012 10
Liferay (Sign In) http://gildavm:8162 User: test@liferay.com Password: test Beijing, Asia 4, 11.04.2012 11
Liferay (Application Space) Beijing, Asia 4, 11.04.2012 12
Liferay (MyJobs) Beijing, Asia 4, 11.04.2012 13
Liferay (ServerLog) Beijing, Asia 4, 11.04.2012 14
About Versioning and current patch Registration mail Generic information Important links (GILDA, Wiki, Training material) Licence information Beijing, Asia 4, 11.04.2012 15
Grid Engine Takes care of any Grid Transaction JAVA APIs + Users Tracking Database Acces to the UsersTrackingDB mysql u tracking_user D userstracking -p! (password: usertracking)! GridEngine tables +-------------------------+! Tables_in_userstracking! +-------------------------+! ActiveGridInteractions! GridInteractions! GridOperations! all_ces! +-------------------------+! GridOperation (for tests) +-------------+--------------+------+-----+---------+----------------+! Field Type Null Key Default Extra! +-------------+--------------+------+-----+---------+----------------+! id int(11) NO PRI NULL auto_increment! portal varchar(120) NO NULL! description varchar(200) NO NULL! +-------------+--------------+------+-----+---------+----------------+! GridEngine Library: /opt/glassfish3/glassfish/domains/liferay/lib/jsaga-jobmanagement-1.4.5.jar!!! Beijing, Asia 4, 11.04.2012 16
Development Environment (Working Directory) 1 st Define a VM shared folder link or (better) use a ssh OS integration app. 2 nd Define links to existing liferay-plugin-sdk and liferay libraries ln -s /opt/liferay-plugins-sdk-6.0.6/ \ /mnt/hgfs/<your_shared_folder>/liferay-plugins-sdk! ln -s /opt/glassfish3/glassfish/domains/liferay/lib/ /mnt/hgfs/<your_shared_folder>/liferay-lib! WARNING: On Windows and Mac machines it is not possible to work using symbolic links Copy the whole content of directories: /opt/liferay-plugins-sdk-6.0.6/! /opt/glassfish3/glassfish/domains/liferay/lib/! On Mac you may use Mac OSX Fuse+MacFusion to access directories via SSH 3 rd Define a VM readable hostname (optional, only with NAT) Insert into VM hosting machine /etc/hosts file the entry As root add: <NATIPAddress>!gildavm' For Windows machines add the same entru inside: %SystemRoot%\system32\drivers\etc\hosts! Beijing, Asia 4, 11.04.2012 17
Development Environment (Working Directory) Liferay Inc offers an SDK to develop plugins like portlet: /opt/liferay-plugins-sdk-6.0.6/! There is an sh script able to create a liferay plugin ready to be edited and deployed. Beijing, Asia 4, 11.04.2012 18
Development Environment (IDE) High level development tools can be integrated with the VM exploiting the shared directory; (NetBeans or Eclipse) See point 7 th of hostname-portlet Wiki content https://gilda.ct.infn.it/wikimain/-/wiki/main/hostname-portlet! Beijing, Asia 4, 11.04.2012 19
Compile your portlets As liferayadmin user connect to the VM and cd to the liferay-plugin-sdk/portlets/<your_portlet_dir>; then type: ant deploy! During the compilation, have a look on the produced server log with: tail -f /opt/glassfish3/glassfish/domains/liferay/logs/server.log! WARNING: Compilation may fail even the ant deploy command tells that the deployment is successful; pay attention to the produced log Beijing, Asia 4, 11.04.2012 20
Brief intro to hostname portlet Many Grid applications have the same requirements to run over a distributed environment: Specify the Executable Define Std-Out/Err files Speficy files for the Input and Output Sandoboxes Provide a BASH shell script that pilots the application execution on the remote host Configure distributed infrastructure services All these requirements have been collected into the hostname portlet: (Historically the hostname Job its the hello-world test application for Grids) Beijing, Asia 4, 11.04.2012 21
hostname portlet (MrBayes) First Grid application (proof of concept) Beijing, Asia 4, 11.04.2012 22
hostname portlet (GATE) Prototyping the first experience Beijing, Asia 4, 11.04.2012 23
hostname portlet (Template) Portlet Template The interface takes a input file or a text input. It just executes the hostname on the remote machine displaying information like: Working directory, Home directory and produces a files as well. It also simulates the creation of an output file. Beijing, Asia 4, 11.04.2012 24
hostname portlet (preferences) Portlet Template Preferences Information System Resource Manager ROBOT prroxy server Virtual Organization Proxy role Application Id (GridEngine) Requirements Pilot Script WARNING When the portlet first installs do not forget to place here the application pilot script Beijing, Asia 4, 11.04.2012 25
hostname portlet (about) Portlet Template About Licence Information Versioning Info Beijing, Asia 4, 11.04.2012 26
Multi Infrastructure Available only from Grid Engine 1.4.3 Same behavior of standard hostname-portlet New preference pane showing more infrastructures and allowing to add/remove/edit infrastructure settings Allows job submissions over different infrastructure. The selection mechanism actually done via a random choice Beijing, Asia 4, 11.04.2012 27
Portlet workflow doview! JSP page ACTION/VIEW Architecture Hostname code provides two Enum types: Actions and ProcessAction!!Views! doview! default JSP page JSP page ProcessAction! For each Action a different View can be selected and easily managed b switch statements The two enumerated types used to manage the portlet behavior Between portlet methods: doview and ProcessAction. Beijing, Asia 4, 11.04.2012 28
Testing VM Connect the VM as liferayadmin user Extract from GILDA software repository: svn checkout http://svn.ct.infn.it/svn/liferay/trunk/gilda/mi-hostname-portlet! WARNING if there exists already a hostname portlet; remove the directory and extract again the code. This ensures you will have the latest version.! Change directory to (mi-)hostname-portlet and compile: ant deploy! Open the Server Log and wait until appears the message: [# 2011-11-09T12:26:46.094+0100 INFO glassfish3.1 javax.enterprise.system.tools.deployment.org.glassfish.deployment.common _ThreadID=138;_ThreadName=Thread-1; [AutoDeploy] Successfully autodeployed : /opt/ glassfish3/glassfish/domains/liferay/autodeploy/hostname-portlet.war. #]! Connect to the Liferay Page, install and test the portlet! Do little changes on the code first and do several tests Beijing, Asia 4, 11.04.2012 29
Advanced topics: GILDA VM patches From the GILDA VM 1.4 on it is possible to keep updated the VM. As soon as new patches are available the user will be prompted: By a message into the VM main screen message By the welcome message at each remote connection (SSH) New changes checked via: A scheduled cron job starting each day at 9:00 am At each machine boot User may force the check with the script: /etc/gilda/vmpatcher.sh View content/apply patches: vmupdate -i/vmupdate (!Please do a backup before to apply any patch) Beijing, Asia 4, 11.04.2012 30
Advanced topics: Liferay domain How to start/stop/restart Liferay Liferay and Glassish automatically start upon boot. These services (together) can be managed through: /etc/init.d/liferay <start stop restart status>! Glassfish may be managed by root user though the utility (*) /opt/glassfish3/glassfish/bin/asadmin! Start/Stop Glassfish+Liferay: asadmin start-domain <domain-name> (liferay)! Asadmin stop-domain <domain-name> (liferay)! Show installed web applications asadmin --port <port> -u <admin-user> list-applications! Undeploy portlets sh asadmin --port <port> --user <admin-user> undeploy <portlet-name>! Create domain asadmin create-domain --portbase <port> <domain-name>! Getting help asadmin help! (*) Disable the liferay service if you like to manage Glassfish directly Beijing, Asia 4, 11.04.2012 31
Advanced topics: Robot proxy server simulator Liferay GILDA VM simulates internally a Robot Proxy server Production portlets work since /etc/hosts/ includes the defautl proxy server: myproxy.ct.infn.it! The GILDA VM uses a mini_httpd server to emulate the Robot Proxy server start/stop/status for mini_httpd services /etc/init.d/mini_httpd <start stop status>! You may use your own personal certificate proxy coping it to the file: /etc/gilda/etoken/proxy.txt! Copy a generated proxy from UI (use of a personal certificate) scp <remote-ui>:/tmp/x509up_u501 /etc/gilda/etoken/proxy.txt! Multi Infrastructure job submission needs a dedicated file into: /opt/mini_httpd-1.19/etokenserver/etoken/<proxy_id>/index.html Beijing, Asia 4, 11.04.2012 32
Advanced topics: Network Checker The VM provides a service able to restart the network as soon as network connection changes More useful when operating the VM in Bridged mode Called on startup by /etc/inittab! Script: /etc/gilda/networkchecker.sh! Once network settings change boot and ssh login messages will change accordingly Beijing, Asia 4, 11.04.2012 33
References GILDA VM https://gilda.ct.infn.it/services GILDA VM Instructions https://gilda.ct.infn.it/wikimain/-/wiki/main/gilda+liferay+virtual+machine Wiki hostname portlet template https://gilda.ct.infn.it/wikimain/-/wiki/main/hostname-portlet Beijing, Asia 4, 11.04.2012 34
Questions? Beijing, Asia 4, 11.04.2012 35