Geosciences Workflow
Project Leader: Robert Woodcock, CSIRO
Description
This project is implementing the specific interoperability standards of the geoscience community on the APAC Grid. The project acts as a connection point between the development of the APAC Grid and the larger Australian geoscience communities such as the Solid Earth and Environmental Sciences Grid (SEE Grid), the pmd*CRC and ACcESS MNRF. As a result, APAC Grid components and services of interest to the geoscience community will have interfaces that match the standards developed by the community and exhibit conceptual integrity and semantics understood by the geoscience community.
The project maintains a collaboration (twiki) website at: https://www.seegrid.csiro.au/twiki/bin/view/Compsrvices/WebHome#Grid_Technology_for_Geosciences.
Achievements
| 2004 |
- Candidate services have been identified as part of the project scenario and documented in the system specification.
- Candidate scenarios and associated services have been documented. A specific scenario has been selected but requires further review and detailed specification during 1Q2005. A Joint Application Development Workshop is scheduled for March 2005 to perform this activity.
- The first version of the system specification has been documented (on the twiki) and circulated for discussion to infrastructure project leaders and their teams.
- GplatesMarkup Language (GPML) is to form the transmission layer of the EarthBytes 4D data portal with Gplates at its centre. The GPML provides a standard conceptual model for the geophysics and geology community. It will build on the work of the OpenGIS consortiums GML and the SEE Grid XMML standards and become a part of the family of community information models.
|
| 1H2005 |
- Reviewed candidate APAC Grid middleware and architecture support and completed a technical report (see project’s Twiki site).
- Reviewed candidate APAC Grid e-Research services and completed technical report. The service requirements were developed during the Joint Application Development (JAD) workshop held March 2005 (see project’s Twiki site).
- Formalised requirements for Geoscience research community usage of APAC Grid and completed requirements document (see project’s Twiki site).
- Completed architectural design of APAC Grid e-Geoscience layer utilising SEE Grid open standards and interfaces and built on APAC base computing and e-Research layers. The design was documented and key middleware components configured and deployed to the development grid virtual machines. Unfortunately there was limited feedback during the reporting period from infrastructure projects to confirm suitability of the design. The situation was improving towards the end of the period and having a candidate architecture, partial deployment and well documented use-cases assisted communication.
- Further development of the GPML2 data standard.
- Completed selection of grid services to be e-Geoscience layer enabled. The JAD workshop completed selection of the services. There is a possibility that additional services will be developed by geoscience researchers outside of the original APAC Grid proposal (the seismic simulator is one example). These will probably be confirmed during 2H2005 as the grid infrastructure and development cycle mature.
- Demonstrated integration of computational services and applications with information access services – acceptance test pass and information is retrieved from SEE Grid compliant information service. This approach has been demonstrated with establishment of a national geochemical assay grid data service. This work was completed in collaboration with all state and territory geological surveys in a project under the SEE Grid banner and supported by AusIndustry. The services are available to geoscience researchers and industry worldwide.
- Investigating a registry system to allow for modelling results management. The registry service is based on similar work underway in the CCLRC Data portal project and a mapping for the schema used on that project to the ebRIM registry standard has been completed. At this time the project has selected the ebXML/RIM registry model and technology as its preferred method for providing this service. This is yet to be ratified by the infrastructure projects.
- Installed a 2Tbytes server at CSIRO WA.
- Installed SRB federation between CSIRO WA and CSIRO HPSC for use by Geoscience projects. The federation uses the same middleware and policies proposed for use on APAC Grid gateways.
|
| 2H2005 |
- Seismic Simulator Portal was deployed to the iVEC network. The portal was deployed to ngportal hosted at iVEC, which submits jobs to ng1. Input and output files are stored and transferred from the SRB federation hosted at CSIRO. StGermain/Snark seismic code successfully installed on iVEC’s new SGI Altix system.
- Design and implementation of first version of the Snark WS service. This is a ws-rf service to interface via the grid to the Snark geodynamics modelling code. The service can be used to submit jobs to the grid, stage files from SRB and get job information and result files during execution. Current service is available on the iVEC network (ng2.ivec.org) and is currently in testing status.
- Development and deployment of a generic APAC ws-rf service available for use by all Application Support Projects. This service greatly decreases the complexity in designing and implementing a ws-rf – based job-submission service for GT4 (ng2).
- Installation and configuring the iVEC gateway machines. Configuration and testing of ng1, ng2 and ngPortal for the iVEC network.
- Configuring and testing the gridftp/SRB driver for gridftp and data staging for job submission.
- Design and implementation of prototype Geoscience registry service based on ebXML registry server.
- Design and implementation of the Geoscience’s metadata model for datasets based on the CCLRC metadata model.
- Tuition of Elspeth Thorne (ACcESS MNRF) in grid methods for Finley.
- Extensive testing of the CSIRO SRB Federation for production based deployment to CSIRO Computational Geosciences community.
- Development and publishing of “How to get Grid for Developers” and “How to get Grid for Users” on the SeeGrid twiki site.
|
| 1H2006 |
- Snark has been deployed to 3 sites – iVEC, APAC NF and VPAC.
- Snark services has been deployed onto the ng2 machine at iVEC
- Finley deployed onto iVEC and APAC NF
- Finley service deployed at iVEC
- FastFlo deployed onto iVEC and CSIRO HPSC
- FasFlo service deployed onto ng2 at both iVEC and HPSC
- SRB federation operational between CSIRO Perth and HPSC in Melbourne
- Template codes for ws-rf services available to APAC Grid community
- “How to get Grid” guides for Geosciences online
- Workflow for Grid usage established and documented on SEE Grid twiki site
- Grid client tools for accessing Geoscience services, GridFTP and SRB deployed and in use by multiple users
- Grid jobs running at multiple sites with only service address change required to submission process
- Requirements for registry service completed
|
Plan for 2006
The plan for 2006 focuses on producing 3 outcomes intended to ensure the Geoscience Grid use and expansion is sustained:
- The establishment of a small user community who use the geoscience services developed.
- The deployment, to multiple grid sites, of the 3 geoscience services (based on Snark, Fastflo and Finley) and related grid services necessary for data storage and transport (a geoscience data federation).
- Documentation and how-to guides showing how the Grid operates and how to create and deploy your own services so others may continue the effort.
Milestones for 2006
| October |
- Finley portal deployed by ACCESS
|
| December |
- Registry service with client GUI for dataset registration and discovery
- Interface Development
- Testing and Deployment
- AcCESS MNRF facility and services available via UQ grid gateway
- Nimrod WS that is Globus compliant (ie – wsrf based, basic service)
- Integrated Nimrod WS into the pmd*CRC Desktop Modeling Toolkit (DMT) as a demonstration of the WS’s usage
- Finite element mesh generation service with support for parameter sweep based generation of related mesh structures
- Grid enable esys_particle service at iVEC and AcCESS MNRF
- User support for Geoscience services
|
Participating Organistions
The following organisations are participating in this project:
- CSIRO
- Curtin University of Technology
-
iVEC
-
AcCESS MNRF
-
University of Queensland
Resources for 2006
- Total 2006 resources available to the project are 1.80 efts (CSIRO: 0.80, iVEC: 1.00).
- APAC is providing funds to support 0.90 efts (CSIRO: 0.40, iVEC: 0.50).
