Chapter 4. Release notes

4.1. Release notes for OGSA-DAI 4.0
4.1.1. Acknowledgements
4.1.2. Main features
4.1.3. Backward compatibility and OGSA-DAI 3 versions
4.1.4. Java
4.1.5. Tested platforms and databases
4.1.6. Known problems
4.1.7. Changes and modifications in detail
4.1.8. Product stability
4.1.9. WSRF
4.1.10. Limitations

4.1. Release notes for OGSA-DAI 4.0

4.1.1. Acknowledgements

Work on OGSA-DAI 4.0 has been funded by the UK Engineering and Physical Sciences Research Council via OMII-UK and also by The University of Edinburgh. A full list of contributors and sponsors present and past is available at: http://sourceforge.net/apps/trac/ogsa-dai/wiki/Sponsors.

4.1.2. Main features

OGSA-DAI 4.0 has the following features:

  • OGSA-DAI SQL views components, previously released in December 2008, have now been refactored and fully-integrated. SQL views now use many of the same SQL parsing components as DQP.
  • Generic activities contributed by the ADMIRE project. These prototype activities allow you to do joins or transform a stream of tuples using a scripting language such as JavaScript.
  • JDBC-based relational resources that support connection pooling using the Apache Commons DBCP package.
  • Data sink and data source operations that support sequence numbers. This allows clients to be implemented that can resynchronize with a server in the event of a network outage.
  • Data sinks and data sources now support non-blocking read and write operations.
  • A trace facility to view the execution of an OGSA-DAI workflow via a JSP page has been added.
  • An example web server file caching activity that streams data into a server file and a RESTful interface to allow access to these files is included.
  • Example KML activities and supporting classes to convert OGSA-DAI tuples into KML documents are included.
  • Changes to the OGSA-DAI configuration and deployment make it easier to express complex configuration requirements and copy configurations across servers.
  • Java 1.6-compliance.
  • Numerous bug fixes.

4.1.3. Backward compatibility and OGSA-DAI 3 versions

Backwards compatibility between OGSA-DAI 4.0 and OGSA-DAI 3.x versions is as follows.

  • OGSA-DAI will not compile under Java 1.5 or 1.4. This is due to changes Java made to java.sql APIs that we implement between Java 1.5 and 1.6.
  • JNDI configuration of OGSA-DAI components has been replaced by Spring. This requires an ogsadai-context.xml file in the CLASSPATH. This also requires additional Spring-related JARs to be available in the CLASSPATH. These are bundled with OGSA-DAI.
  • uk.org.ogsadai.config.CommonComponentsConfiguration classes and the config.txt file have been removed. If you developed components that used this then these will no longer compile or run. You can now use the Spring file above to insert entries into the OGSA-DAI context. Any use of uk.org.ogsadai.config.CommonComponentsConfigurationManager can be replaced by use of the OGSA-DAI context.
  • We have added operations to the data source and data sink services.
  • We have not changed the resource or activity APIs on server or client side so existing server-side activity or data resource components, or existing clients should still be able to compile and run.
  • An OGSA-DAI 3.x client should be able to interact with an OGSA-DAI 4.0 server without any recoding or recompilation of the client being necessary.
  • An OGSA-DAI 4.0 client should be able to interact with an OGSA-DAI 3.x server providing it doesn't request 4.0-only functionality.

If you are an OGSA-DAI 3.x user and experience any problems in using OGSA-DAI 4.0 then please contact us (see Chapter 3, Help and support).

Chapter 5, How to upgrade to 4.0 for OGSA-DAI 3 users contains information on how to upgrade from OGSA-DAI 3 to 4.4.

4.1.4. Java

OGSA-DAI 4.0 binary releases are compiled using Sun Java 1.6.0_17.

4.1.5. Tested platforms and databases

OGSA-DAI 4.0 has been tested as follows. All tests are run on Linux Red Hat 9 using Apache ANT 1.7.0

ReleaseJava versionUnit testsDatabase specific testsRelease build testsClient-server tests
OGSA-DAI 4.0 GT 4.2.01.6.0_16YesYesYesYes - this includes security-related tests

Table 4.1. Tested platforms and databases


Database-specific tests have been run on all the databases listed in Chapter 25, Data resource products. Client-server tests were run using MySQL, eXist and Linux file system resources.

4.1.6. Known problems

For our current list of known problems and issues see:

4.1.7. Changes and modifications in detail

Activity changes:

  • uk.org.ogsadai.activity.extension.ConfigurableActivity's method configureActivity can now throw ActivityInitialisationException so activities can signal problems in their configuration properties.
  • The SQLNestedInClause activity now supports ?MIN and ?MAX templates in the expression.
  • The DeliverToDataSink activity now sends data to the remote server whenever it has data to send. It no longer waits for a specified number of blocks before sending the data.

Activity access to presentation-layer objects:

  • Now, any object can be passed from a presentation layer to an activity via a request resource, not just a security context.
  • The interface uk.org.ogsadai.resource.request.CandidateRequestDescriptor and its implementation SimpleCandidateRequestDescriptor now provide methods supporting the put and get of key-value pairs.
  • The key-value pairs are copied into a request resource by uk.org.ogsadai.resource.drer.SimpleDRER.

Backport Concurrency JAR:

  • Due to the move to Java 1.6 the dependency on the Backport Concurrency JAR has now been removed and this is no longer bundled. All references to edu.emory.mathcs.backport.java.util.concurrent have been replaced by java.util.concurrent.

Client handlers:

  • The Apache Axis client-side handlers file, client-config.wsdd, is now bundled in its own JAR so there is no need now to have the deploy directory in your CLASSPATH.

Deployment:

  • ANT commands for deploying resources and activities and changing their properties have been replaced by a single configure command. This takes in a script specifying one or more deployment and configuration-related operations. This allows complex configurations to be expressed within the scope of a single file.
  • The deployment commands buildGAR, deployGAR, buildAndDeployGARTomcat have been renamed to buildWebapp, deployWebapp, buildAndDeployWebapp respectively. These commands no longer build a GAR file but build a directory mirroring that of the GT wsrf webapp directory in Tomcat and containing just the OGSA-DAI XML Schema, WSDL, JARs and configuration files. Deployment now simply involves the appropriate command copying this directory to the GT wsrf directory in Tomcat.
  • The main method on uk.org.ogsadai.authorization.file.SimpleFileLoginProvider has been removed. The class uk.org.ogsadai.tools.LoginEditor now serves that purpose.

KML activities and supporting classes:

  • Two activities have been added which transforms tuples to KML documents (represented as lists of character arrays):
    • TupleToKMLPlacemarks - which converts tuples to KML documents. Each tuple is assumed to have a latitude and longitude field and these, with optional additional fields from the tuple, are converted into a KML placemark for the tuple.
    • SummaryTupleToKMLPlacemarks - this is similar to the above but each tuple is assumed to also have a "summary" field which is used to determine the nature of the KML placemark added.
  • Both activities are configurable - tuple to KML convertor implementations can be plugged-in. Relevant interfaces and sample implementations are in uk.org.ogsadai.convertors.tuple.kml.
  • Please note that these are not definitive implementations of tuple to KML convertor activities, rather they are examples as to how such activities can be implemented.

Product-specific meta-data extraction:

  • An SQL Server-specific meta-data handler implementing uk.org.ogsadai.resource.dataresource.jdbc.MetaDataHandler has been added - uk.org.ogsadai.resource.dataresource.jdbc.SQLServerMetaDataHandler . This addresses a problem arising from JDBC drivers providing incorrect ORDINAL_POSITION values.

Server configuration:

  • JNDI is no longer used to configure OGSA-DAI components that are held within the OGSA-DAI server context. This configuration is now done by a Spring XML configuration file which allows the same file format to be used across OGSA-DAI versions. See Appendix E, Appendix - OGSA-DAI context file for more information.
  • All classes relating to common components configuration and the server config.txt file have been removed.
  • Values that were provided in the config.txt file can now be provided as Environment entries in the new XML configuration above.

Web server file caching activity and REST API:

Web server information:

  • An empty uk.org.ogsadai.server.ServerContext interface has been added. An interface WebServerContext extends this with getters for a web server host, port, webapp and protocol. This is intended to be used as a container for this information for use by activities that construct WS-EPRs or servlet URLs.
  • uk.org.ogsadai.server.TomcatContext provides an implementation of this that extracts this information out of its current message context.
  • Web service presentation layers now deposit an instance of TomcatContext into the CandidateRequestDescriptor (see above) with key uk.org.ogsadai.server.WebServerConstants.WEB_SERVER_CONTEXT.
  • As a consequence, there is no need to specify service URLs or ports when deploying OGSA-DAI - URLs submitted by clients are used to determine URLs to return to the client.

Bug fixes and other changes:

  • TupleUnionAll now does type coercion between numeric columns if two input tuple lists contain different numeric types for the same column.
  • Bugs in GT 4.2 service security descriptors in the user doc and in the implementations of PIPs and PDPs have been fixed.
  • Erroneous activity input names in uk.org.ogsadai.dqp.execute.workflow.SortBuilder have been fixed.
  • A bug in uk.org.ogsadai.activity.delivery.DeliverToDataSinkActivity whereby it did not check the number of blocks specified until after the value was used (resulting in a server-side error if the block size was < 0) has been fixed.
  • Files ending with a tilde, ~, in the resource and resource template directories are now ignored.
  • A thread pooling bug in the TupleUnionAll activity has been fixed. See OGSA-DAI 3.2.2 software advisories for more details.
  • A bug that caused DQP to enter an infinite loop with deep query plans has been fixed. See OGSA-DAI 3.2.2 software advisories for more details.
  • A bug that could cause inaccurate joins in the TupleMergeJoin and SQLNestedInClauseJoin activities has been fixed. See OGSA-DAI 3.2.2 software advisories for more details.
  • A bug in DQP's use of a filtered table scan that could lead to the loss of data from a query result has been fixed. See OGSA-DAI 3.2.2 software advisories for more details.
  • A bug in DQP's use of ORDER BY has been fixed. See OGSA-DAI 3.2.2 software advisories for more details.
  • A bug in DQP's deserialisation of tuples containing dates has been fixed. See OGSA-DAI 3.2.2 software advisories for more details.

Unsupported features:

  • All APIs or classes marked deprecated in previous releases have been removed.
  • The extensibity point to specify a data sink pipe factory has been removed. This functionality is not longer required due to the addition of non-blocking data sink operations.
  • The Globus Toolkit stand-alone container is no longer supported.
  • MDS registration is no longer supported.
  • Globus Toolkit 4.0.8 is no longer supported.

4.1.8. Product stability

The following APIs may be assumed to be held stable in further OGSA-DAI 4 releases. The only exceptions to this are:

  • If a fundamental problem is encountered for a majority of users.
  • Methods we have marked as deprecated in 4.0.
  • In the case of an OGSA-DAI 5 release then any and all APIs may be refactored.

The list of APIs are as follows:

  • Activity API - API for activity developers.
  • Activity framework API.
  • Client toolkit API.
  • OGSA-DAI context.
  • Presentation layer WSDL. We will not remove operations or change namespaces but we may add operations.
  • Resource APIs.
  • Security context API.

The following components may be subject to change in the light of requirements to increase functionality or fix bugs. We may release bug fixed components or new versions of existing components. These will not incur API changes though. Note also that these are components that implement OGSA-DAI extensibility points so are designed to be pluggable and replaceable in this way.

  • Authorizer implementations. These are presentation layer and application specific so are intended to be replaceable. We may release new ones for different applications or demonstrating use of different third-party authorization infrastructures.
  • Activity implementations.
  • Activity manager implementation.
  • Persistence manager implementation. The current file-based implementation is simple and lightweight and may not be scalable.
  • Presentation layers.
  • Resource manager implementation.

The following APIs and components may be viewed as "alpha" versions which will be subject to change:

  • Configurable pipes.
  • Workflow transformation.
  • Remote resources.
  • JSP pages. We may release new versions. This has no implications on any other components.
  • Sub-workflow API. This may be engineered to be more usable in a future release. This will possible expose a client toolkit-like API for use in activities that create workflows.

4.1.9. WSRF

OGSA-DAI Axis services are compliant with the version of WSRF currently bundled with Globus Toolkit 4.0 releases. These are the 2004/06 OASIS WSRF and WSN working draft specifications with minor fixes to the 1.2-draft-01 published schemas and with the March 2004 version of the WS-Addressing specification.

OGSA-DAI GT services are compliant with the version of WSRF currently bundled with Globus Toolkit 4.2 releases which is the OASIS WSRF 1.2 standard.

4.1.10. Limitations

  • Some of our activities, e.g.
    uk.org.ogsadai.activity.sql.SQLBagActivity
    uk.org.ogsadai.activity.sql.SQLResilientActivity
    uk.org.ogsadai.activity.pipeline.AutomaticTee
    
    use hard-coded activity names to identify the activity or activities in sub-workflows they splice into workflows. This means that you should take care when renaming activities exposed by resources, especially block activities. This problem will be most likely manifested as an unsupported activity error.
  • ObtainFromGridFTP and DeliverToGridFTP do not allow the setting of certain GridFTP parameters.
  • The standard Oracle JDBC driver does not conform to the JDBC API for insertion of BLOBs and so OGSA-DAI does not currently support the addition of BLOBs to tables in Oracle databases.
  • The table-name tag in WebRowSet XML is empty in results produced by Oracle or SQLServer. This is a problem of the JDBC drivers that do not provide table names in the metadata of a java.sql.ResultSet.
  • Meta-data from a database, returned by ExtractTableSchema, can be case-sensitive depending on the database used. For example, MySQL might return a table called littleblackbook while DB2 returns LITTLEBLACKBOOK.
  • If an error occurs during execution of a relational activity then server-side information may be returned to the client. This is because OGSA-DAI does not, as yet, parse java.sql.SQLExceptions from database drivers to identify problems caused by a client (e.g. a syntactically-incorrect SQL query) and those due to server-side failures (e.g. loss of connection to a database).
  • Though this is not OGSA-DAI specific, a user encountered a problem using the standard MySQL driver class org.gjt.mm.mysql.Driver from the mysql-connector-java-5.0.5 driver JAR in conjunction with MySQL version mysql-4.1.20-1.RHEL4.1 running on Red Hat. Even though they had provided the database name as part of the database connection URL, e.g.
    jdbc:mysql://localhost:3306/dbname
    
    They had to specify the database name in their queries also, e.g.
    SELECT * FROM dbname.table
    
  • If a client does not provide a certificate then using deliverFromGridFTP and deliverToGridFTP in a request the transfer will fail.
  • There is a possible deadlock situation which can affect workflows which use naively the SQLNestedInClauseQuery activity. The issue is described in the activity documentation, SQLNestedInClauseQuery.
  • BLOBs and CLOBs are currently only held in-memory with consquent implications on memory usage.
  • OGSA-DAI data request execution resources and implementations of JDBC, XMLDB and file system data resources do not support use of the destroy operation. It is a no-op.
  • The default OGSA-DAI resource and activity managers

    uk.org.ogsadai.activity.SimpleActivityManager
    uk.org.ogsadai.resource.SimpleResourceManager
    

    cache activity specifications and resources in in-memory hash tables. They do not check for changes in any persisted configuration associated with the activity specifications or resources once the associated objects are in the cache.

  • The default OGSA-DAI file-based resource persistence and configuration manager uk.org.ogsadai.persistence.file.resource.SimpleFileResourceStateDAO only handles the persistence and restoration of resource properties or session state that are Strings.
  • The WSRF operation SetResourceProperty is not supported by OGSA-DAI services.
  • The OGSA-DAI GT wrapper class uk.org.ogsadai.service.gt.resource.GTResourceProperty methods
    add(X)
    clear()
    get(int)
    remove(Object)
    set(int, Object)
    
    are currently not supported. These would need to convert from XML fragments into objects.
  • There is no cleanup if client-service communication times out. Service will just reply as usual.
  • The uk.org.ogsadai.common.BinaryLob class contains unimplemented methods which throw java.lang.UnsupportedOperationException.
  • Conversion of java.sql.ResultSet to XML WebRowSet returns empty key-column and map properties in the properties element.