Generic Guides Generic Guides

What is a Science Gateway

Science Gateways are frameworks (or toolsets) which incorporate applications, data and tools to enable running applications on Grid infrastructures. They also provide services to support, upload, search, manage and download (or share) applications and data. The gateways are integrated via portals or sets of applications. Gateways enable user communities associated with a common discipline to use compute and data resources through a common  graphical user interface in an easy and intuitive way. As a result, users can focus on their applications instead of learning and managing the complex underlying infrastructure.  

 

Players in gateways

People around gateways have different role and a good gateway should provide support for all those people who work with the gateway. The first category is the gateway developers who develop the gateways. Here we have to distinguish SG framework developers and SG instance developers. The primary goal of SG framework developers is to develop the SG framework back-end in a portable way that enables SG instance developers to use it without modifications. Their second goal is to develop the generic part of the front-end. This is also important for the SG instance developers and some parts even for the SG instance users, too. The main task of the SG instance developers is to customize a SG framework for their user community. It means that the user interface of the SG framework should typically be extended with new application-specific portlets. However, if the SG instance is developed from scratch the SG instance developers have the same tasks as the SG framework developers and additionally, developing the application-specific portlets. That's why this second option is not a good practice. Once the SG frameworks or SG instances are developed they should be set up and operated. Here comes the role of gateway operators. They should be able to deploy, configure, run and maintain the gateway service for the user communities. For these purpose good gateways provides installation and configuration wizards, user management support interface, etc. These again – in good practice – can be developed in a generic way within a SG framework and just used (maybe adapted) by SG instances.

 

Once the SG frameworks or SG instances are set up and operated the user can come forward and start to use them. We have to distinguish two user categories: end-users and application developers. In fact, they need different front-ends. The application developers develop those DCI applications that are used by the end-users. The application developers are typically IT people or scientists (chemists, etc.) with good understanding of the underlying IT technology. They should get relatively detailed access information on the underlying DCIs while this information could partially or completely be hidden for the end-users. Therefore, the SG frameworks are primarily targeted to the application developers and the SG instances are typically designed for the end-users. Of course, this typical usage does not exclude the possibility that some SG framework can be used by end-users and SG instances can provide front-end necessary for DCI application development. However, the good practice is the clear separation of these two concepts.  

 

Building a Science Gateway

1. Create a list of requirements your SG should meet.

Here we provide you the major possible items on your list:

  • Target community
    • Focussed for a certain branch of science -> you need SG instance specialized for supporting that branch of science
    • Scientists from various fields (e.g. NGI SG) -> a generic purpose SG framework might do.
  • Size of the target community
    • Small number of scientists -> the scalability requirements are not so important
    • Large number of scientists -> the scalability requirements are very important
  • Supported applications
    • Focussed for one or very small number of predefined applications
      • -> you need SG instance specialized for supporting that application(s)
      • -> you need robot certificate
    • Large set of applications that can be defined by the users
      • -> you might need SG framework enabling the development of applications
      • -> you need certificate management
    • Large set of workflow applications that can be defined by the users
      • -> you might need SG framework supporting workflow creation and management
      • -> you need certificate management
      • -> support for SHIWA workflow repository access would be useful
  • Target DCIs
    • Focussed for one particular DCI
    • Enabling access to several different DCIs (e.g. an NGI where both clouds and different grids are supported)->  your SG should be derived from a SG framework that can support many different DCIs
  • Generic requirements
    • Robustness
      • You want only a prototype of your SG->  this is not an issue for you, you can develop your SG from scratch
      • You want to create a production SG service-> Choose a robust SG Framework and create your SG instance with customization of the selected SG Framework (if the target SG is extremely simple you can build it from scratch, too)
    • Supported functionalities
      • Limited functionalities: you can try to develop your SG instance from scratch
      • Rich functionalities: choose a SG Framework with rich functionalities and create your SG instance with customization of the selected SG Framework
    • Sustainability
      • You want to support the target user community only for the time of a project (2-3 years) ->you can try to develop your SG instance from scratch
      • You want to support the target user community for a long term beyond the development project life-time -> choose an open source based SG Framework behind which there is a large and active development and user community (this gives a chance that this SG Framework and your customized SG instance will be sustainable for a long term)
    • Extensibility
      • You are sure that you do not want to further develop your SG even if its users ask for it -> any
      • You assume that sooner or later your SG should be extended according to the new user needs -> choose a SG Framework with layered and SOA architecture that is easily extendible and create your SG instance with customization of the selected SG Framework
    • Scalability
      • Your target community is small and you know that it will not grow in the future -> scalability is not an issue for you
      • Your target community either large or will grow to be large -> choose a SG Framework with scalable architecture and create your SG instance with customization of the selected SG Framework

2. Choose technologies based on resources and time

Before you start to build your SG it is very important to carefully plan the use of your available resources and time.

  • Small number of PMS and short deadline -> because of the small number of PMs it is better to use or customize an existing SG framework 
  • Small number of PMS and long deadline -> because of the small number of PMs it is better to use or customize an existing SG framework
  • Large number of PMS and short deadline -> because of the short deadline it is better to use or customize an existing SG framework
  • Large number of PMS and long deadline -> you can develop your own SG instance from scratch if you like but consider the disadvantages mentioned in previous chapters and sections. You can even develop your own SG framework but it is risky

3. Building portals from reusable components

The Liferay technology enables quickly building portals using the Liferay portal framework and the existing Liferay portlets. There is a public Liferay portlet repository where you can find a large set of ready-to-use portlets. These help to build simple SG instances from scratch. The problem with such generic portal frameworks is that they do not provide back-ends that support DCI access. Creating the back-end with the necessary DCI access could be time-consuming and there are already existing SG frameworks and services that can be used for this purpose. E.g. the DCI Bridge service developed in the SHIWA and SCI-BUS projects can be easily connected to any gateway via a standard BES interface and it provides access to a large set of different DCIs (clusters, grids, desktop grids, clouds). It would be ideal to create a European SG portlet and service repository that contains portlets and services from which as building boxes SG developers could put together their desired SG instance. There are already such reusable portlets and services but the set of them is far not complete and their integration methods are not crystallized yet.

 

If you build a SG framework currently there are two options:

  • Build from scratch -> use standard portal building technologies (e.g. Liferay) that guarantees an initial framework and a large set of existing portlets. Use these portlets whenever it is possible and develop only those portlets that specialized for your needs.
  • Customize an existing open source SG framework: Usually they are built on top of Liferay so they have the same advantages as writing the SG directly on top of Liferay but they provide already many, useful further functionalities, portlets and back-end with access to various DCIs. However, it is better to collaborate with the developer community of the selected open source SG framework and jointly further develop that open source SG framework.

 

4. Plan for the long term

If your SG should be maintained for a long time it is very important to consider this requirement from the very beginning of designing your SG. This is a typically overlooked aspect of designing and establishing SGs. Many projects consider only the lifetime of their project and they do not care what will happen to the project products (including the developed SG) after the project is completed. As a result many SGs become unusable after the developing project is over. To avoid this pitfall the following recommendations should be followed: * If there is any existing open source SG framework that can be customized to your need always develop your SG instance with SG framework customization and do not start to develop it from scratch: o customization needs less development and maintenance cost than scratch-development o the open source community behind the SG framework has got chance to enforce the required future costs for sustainability * If your user community is small do not put much effort to the development of the SG instance: a small community is not able to enforce the required future costs for sustainability * Try to enlarge your user community: a large user community has got chance to enforce the required future costs for sustainability 8.6 Develop in stages It is very difficult to develop and provide exactly that kind of SG that your community wants. Instead of putting many efforts to find out completely what the user community wants it is better to build a simple SG instance for them with small development effort. Then provide SG service to the user community and ask for feedback. It is important that even this simple SG instance should be robust and user-friendly otherwise the user community will be disappointed, will leave you and will nevercome back to you. If the user community likes the simple SG then they can tell you what to improve and extend. Based on this feedback you can enter into the second stage of SG development. At this point it is very important that even the simple SG should be scalable and extendible. These features enable the fast extension of the SG and the fast grow of the number of the users. Do not put too many new services in the new version of the gateway. Every new version should provide some clear new functionalities and the new version should be as robust as the previous one was otherwise the users leave your SG. In summary: providing frequently small extensions as new releases is a better practice than rarely providing a new release with lots of extensions.  

 

General information:

Administrator

Developer

 

User

SCI-BUS

Supporters

Publications

http://guse.hu/about/publications

 



       This project has received funding from the European Union's Seventh Framework Programme for research, technological development and demonstration under grant agreement no 283481.

Site map | Contact | SCI-BUS in media