Prev   Next

Chapter 2. Release notes

2.1. Release notes for OGSA-DAI 3.2
2.1.1. Main features
2.1.2. Java
2.1.3. Tested platforms and databases
2.1.4. Backwards compatibility
2.1.5. Usage statistics and OGSA-DAI
2.1.6. WSRF
2.1.7. Changes and modifications in detail
2.1.8. Known problems and limitations
2.1.9. Product stability
2.2. Release notes for OGSA-DAI 3.1
2.2.1. Main features
2.2.2. Tested platforms and databases
2.2.3. Backwards compatibility
2.2.4. Usage statistics and OGSA-DAI
2.2.5. WSRF
2.2.6. Changes and modifications in detail
2.2.7. Known problems and limitations
2.2.8. Product stability
2.3. Release 3.0 notes
2.3.1. Usage statistics and OGSA-DAI
2.3.2. Changes from OGSA-DAI 2.2
2.3.3. Backwards compatibility
2.3.4. Product stability
2.3.5. Tested platforms and databases
2.3.6. WSRF

2.1. Release notes for OGSA-DAI 3.2

2.1.1. Main features

The main features of OGSA-DAI 3.2 are as follows.

OGSA-DAI is released under the Apache 2.0 licence.

OGSA-DAI 3.2 includes Distributed Query Processing (DQP):

  • DQP allows the tables from multiple distributed relational databases to be queried, using SQL, as if there were multiple tables in a single database. These databases are exposed via OGSA-DAI servers (either the same server as DQP or remote OGSA-DAI servers).
  • DQP was originally designed and developed by at the University of Manchester and the University of Newcastle upon Tyne. Previous versions of DQP were released as a separate product - OGSA-DQP.
  • DQP functionality has now been rewritten and is now simply an extension to OGSA-DAI rather than a separate product. DQP's functionality is now exposed as new OGSA-DAI resources and activities and accessed via OGSA-DAI's standard services - there are no DQP specific services that need to be deployed. This greatly simplifies deployment and enables extensibility within a single framework.
  • There is now support for a greater number of SQL operations, including GROUP BY, HAVING, EXISTS and SELECT *
  • There is improved extensiblity. Much of DQP can now be extended and adapted to suit specific project or application requirements.

A number of additional new OGSA-DAI components and capabilities are now supported. This includes:

  • A new request event activity extension interface so an activity can be informed when a request it belongs to completes.
  • A new extensibility point for data sink pipe factories is supported. A deployer can specify a data sink pipe factory which allows clients to write data to a data sink and not block, the data being cached server-side.
  • A new monitoring framework has been implemented which tracks the activity status and the number of blocks produced/consumed by an activity.
  • An example client that invokes a workflow that uses ExtractTable Schema to get database schema.
  • An example client that can convert a client toolkit workflow into a DOT graph description file which can then be rendered as an image using a tool such as GraphViz.
  • Support for relational database-specific meta-data extractors and mappers from JDBC column types to OGSA-DAI tuple types.
  • New classes that allow editing of configuration files that are located in an OGSA-DAI server, via command-line invocations. This provides an alternative to manually editing the files.

The following new activities have been added.

  • GetGSSCredentialFromSecurityContext - this extracts a client's credentials from the current server security context and outputs them.
  • GetGSSCredentialFromDelegationService - this retrieves a credential from a Globus Delegation Service. The credential output can serve as an input to other activities that require a credential to operate, such as the new GridFTP activities that are bundled this release.
  • ObtainFromGFTPExtended and DeliverToGFTPExtended - these are new GridFTP activities that make a more efficient use of Grid FTP and allow clients to specify a greater number of GridFTP control parameters. These activities can can also take a credential input for greater flexibility.
  • GetDataSinkResource and GetDataSourceResource - these create client toolkit proxies for data sinks and data sources exposed by remote OGSA-DAI servers. ObtainFromDataSource and DeliverToDataSink have been modified to accept these proxies as input (as an alternative to providing a URL and resource ID).

A number of bugs have been fixed, components made more efficient or robust.

Errors in the user doc have been fixed and new content added to reflect changes made.

Unlike previous versions of OGSA-DAI, OGSA-DAI 3.2 only compiles under Java 1.5 and not Java 1.4.

Apart from this, OGSA-DAI 3.2 is designed to be backwards compatible with OGSA-DAI 3.1 without the need for recompilation - data resource, activity and presentation layer APIs and service WSDLs remain the same.

We no longer release OGSA-DAI Axis 1.2.1 or GT 4.0.5 distributions, though cannot forsee any problems in upgrading OGSA-DAI 3.1 deployments of these versions with OGSA-DAI 3.2 JARs.

We no longer support the OMII container.

2.1.2. Java

OGSA-DAI 3.2 binary releases are compiled using Sun Java 1.5.0_07.

2.1.3. Tested platforms and databases

OGSA-DAI 3.2 has been tested as follows. All tests are run on Linux Red Hat 9 and using Apache ANT 1.7.0

ReleaseJava versionUnit testsDatabase specific testsRelease build testsClient-server tests
OGSA-DAI 3.2 GT 4.0.81.5.0_07YesYesYesYes - this includes security-related tests
OGSA-DAI 3.2 GT 4.2.01.5.0_07YesYesYesYes - this includes security-related tests
OGSA-DAI 3.2 GT 4.2.11.5.0_07YesYesYesYes - this includes security-related tests

Table 2.1. Tested platforms and databases


Database-specific tests have been run upon all the databases listed in Chapter 28, Data resource products with the exception of Oracle. Client-server tests were run using MySQL, eXist and file system resources.

Though OGSA-DAI 3.2 GT 4.0.8 has not been tested with earlier versions of the Globus Toolkit 4.0 we envisage no problems arising in upgrading an OGSA-DAI 3.1 GT 4.0.x deployment to use OGSA-DAI 3.1. GT 4.0.8 JARs.

2.1.4. Backwards compatibility

2.1.4.1. OGSA-DQP 3.2

OGSA-DAI 3.2's DQP components are not backwards compatible with earlier releases of DQP.

2.1.4.2. OGSA-DAI 3.0

Apart from a move to Java 1.5, OGSA-DAI 3.2 is designed to be backwards compatible with OGSA-DAI 3.1 and 3.0. What do we mean by this? We mean:

  • We have not changed any API we have promoted as a public extensibility point, that is the APIs used by activity or data resource developers.
  • A user should be able to replace all OGSA-DAI 3.0 JARs or 3.1 JARs (those with ogsadai-3.0 or ogsadai-3.1 prefixes) in their OGSA-DAI deployments with OGSA-DAI 3.2 JARs and not experience any errors about missing or undefined methods. A user should not need to change the configuration of their server at all. Furthermore, any OGSA-DAI 3.0 or 3.1 components they have written should continue to compile using 3.2 JARs without the need to rewrite any code.
  • An OGSA-DAI 3.0 or 3.1 client should be able to interact with an OGSA-DAI 3.2 server without any recoding or recompilation of the client being necessary.
  • An OGSA-DAI 3.2 client should be able to interact with an OGSA-DAI 3.0 or 3.1 server providing it doesn't request 3.2-only functionality.

If you are an OGSA-DAI 3.0 or 3.1 user and experience any problems in using OGSA-DAI 3.2 then please contact us (see Chapter 4, Information, help and support).

Chapter 5, How to upgrade to 3.2 for OGSA-DAI 3.0 and 3.1 users contains information on how to upgrade from OGSA-DAI 3.0 or 3.1 to 3.2.

2.1.4.3. OGSA-DAI 2.2 and earlier

As for OGSA-DAI 3.0, and 3.1 OGSA-DAI 3.2 is not backwards compatible with OGSA-DAI 2.2.

2.1.5. Usage statistics and OGSA-DAI

The OGSA-DAI 3.0 and 3.1 functionality to collect usage statistics has been removed. Globus Toolkit JARs related to this functionality are no longer bundled.

2.1.6. WSRF

OGSA-DAI GT 4.0 services are compliant with the version of WSRF currently bundled with Globus Toolkit 4.0 releases. From the Globus Toolkit user doc core facts ( http://www.globus.org/toolkit/docs/4.0/common/javawscore/JavaWSCoreFacts.html) these are

"...the 2004/06 OASIS WSRF and WSN working draft specifications (with minor fixes to the 1.2-draft-01 published schemas and with the March 2004 version of the WS-Addressing specification)"

OGSA-DAI GT 4.2 services are compliant with the version of WSRF currently bundled with Globus Toolkit 4.2 releases which is the OASIS WSRF 1.2 standard.

2.1.7. Changes and modifications in detail

2.1.7.1. Activities

DeliverToDataSink

  • The url and resourceID inputs are now optional. If omitted then the following input is expected.
  • A new input dataSinkResource is supported which is assumed to be a client toolkit uk.org.ogsadai.client.toolkit.DataSinkResource proxy.
  • This change affected
    • uk.org.ogsadai.activity.delivery.DeliverToDataSinkActivity
    • uk.org.ogsadai.client.toolkit.activities.delivery.DeliverToDataSink

ExtractTableSchema

  • uk.org.ogsadai.activity.sql.ExtractTableSchemaActivity
  • This now supports an activity configuration property which can be given a value that is a comma-separated list of types of table to return e.g. TABLE, TABLE,VIEW etc.
  • Uses new uk.org.ogsadai.activity.sql.MetaDataHandler API.

GetAvailableTables

  • uk.org.ogsadai.activity.sql.GetAvailableTablesActivity
  • This now supports an activity configuration property which can be given a value that is a comma-separated list of types of table to return e.g. TABLE, TABLE,VIEW etc.

ObtainFromDataSource

  • The url and resourceID inputs are now optional. If omitted then the following input is expected.
  • A new input dataSourceResource is supported which is assumed to be a client toolkit uk.org.ogsadai.client.toolkit.DataSourceResource proxy.
  • This change affected
    • uk.org.ogsadai.activity.delivery.ObtainFromDataSourceActivity
    • uk.org.ogsadai.client.toolkit.activities.delivery.ObtainFromDataSource

TableMetadataToXMLCharArrays

uk.org.ogsadai.client.toolkit.activities.transform.TableMetadataToXMLCharArrays

  • This activity and all the following were deprecated in the last: release and have now been removed. 
    uk.org.ogsadai.client.toolkit.activities.transform.TableMetadataToXMLCharArrays
uk.org.ogsadai.client.toolkit.activities.transform.XMLCharArraysToTableMetaDataRetriever
uk.org.ogsadai.activity.transform.TableMetadataToXMLCharArraysActivity
uk.org.ogsadai.converters.databaseschema.fromxml.XMLToSchemaConverter
uk.org.ogsadai.converters.databaseschema.fromxml.XMLToSchemaConverterImpl
uk.org.ogsadai.converters.databaseschema.toxml.SchemaToXMLConverter
uk.org.ogsadai.converters.databaseschema.toxml.SchemaToXMLConverterImpl
  • The implementation had 3 problems.
    • If table was non-existant then client would throw an exception when parsing the result data (which includes an empty OGSA-DAI list for non-existant tables).
    • If a wild card was used then the client would only return the first table due to a limitation in the parser.
    • Parsing was very inefficient, making use of regular expression matching.
  • It is recommended that the following activity implementation and client toolkit proxy and associated classes be used: 
    uk.org.ogsadai.activity.transform.TableMetadataToXMLCharArraysListActivity
uk.org.ogsadai.client.toolkit.activities.transform.TableMetadataToXMLCharArraysList
uk.org.ogsadai.converters.databaseschema.fromxml.XMLSchemaConvertor
uk.org.ogsadai.converters.databaseschema.fromxml.XMLParsingTableMetaDataImpl
uk.org.ogsadai.converters.databaseschema.DatabaseSchemaMetaData
uk.org.ogsadai.converters.databaseschema.toxml.SchemaToSingleXMLConverter
uk.org.ogsadai.converters.databaseschema.toxml.SchemaToXMLConverterBase

TupleSimpleMerge

  • uk.org.ogsadai.activity.transform.TupleSimpleMergeActivity
  • This activity has been fixed so that it correctly handles meta-data on each iteration of the activity. Previously it was aggregating meta-data on successive iterations resulting in the meta-data for the merged tuples becoming out of sync with the actual columns of the tuples themselves.

WebRowSetCharacterDataToTuple

  • uk.org.ogsadai.activity.transform.WebRowSetCharacterDataToTupleActivity
  • Now uses type name when mapping from JDBC types to OGSA-DAI types.

Activities changed to use new uk.org.ogsadai.resource.dataresource.jdbc.JDBCColumnTypeMapper API

  • uk.org.ogsadai.activity.remote.RemoteAsynchSQLQueryActivity
  • uk.org.ogsadai.activity.sql.ExtractTableSchemaActivity
  • uk.org.ogsadai.activity.sql.NestedQueryExecution
  • uk.org.ogsadai.activity.sql.SQLNestedInClauseJoinActivity
  • uk.org.ogsadai.activity.sql.SQLNestedInClauseQueryActivity
  • uk.org.ogsadai.activity.sql.SQLParameterisedQueryActivity
  • uk.org.ogsadai.activity.sql.SQLQueryActivity
  • uk.org.ogsadai.activity.sql.SQLUtilities

Activities changed to expect resources implementing uk.org.ogsadai.resource.dataresource.jdbc.EnhancedJDBCConnectionProvider .

  • uk.org.ogsadai.activity.sql.ExtractTableSchemaActivity
  • uk.org.ogsadai.activity.sql.SQLNestedInClauseJoinActivity
  • uk.org.ogsadai.activity.sql.SQLNestedInClauseQueryActivity
  • uk.org.ogsadai.activity.sql.SQLParameterisedQueryActivity
  • uk.org.ogsadai.activity.sql.SQLQueryActivity

uk.org.ogsadai.activity.sql.RelationalOrderedJoin

  • Code to get column indices from column IDs pulled into a utility method in uk.org.ogsadai.activity.sql.SQLUtilities

uk.org.ogsadai.activity.sql.SQLUtilities

  • Added utility method to get column indices from column IDs.
  • Changed to handle java.math.BigInteger as produced by MySQL unsigned BIGINT type.
  • Now uses type name when mapping from JDBC types to OGSA-DAI types.
  • Refactored to remove duplicate code.
  • Changes in use of java.util.ArrayList to prevent resizing and so causing problems with changing columns.

uk.org.ogsadai.activity.sql.SQLBulkLoadTupleActivity

  • The activity has been fixed so that it closes java.sql.PreparedStatement objects at the end of each iteration. Previously it was leaving these objects open until they are closed automatically.

2.1.7.2. Activity framework and engine

Default buffer size is now 20. This was updated in server configuration files:

  • $OGSADAI_HOME/deploy/jndi-config.xml

uk.org.ogsadai.activity.request.OGSADAIRequestConfiguration .

  • Adds block reader listeners to the array of already registered ones instead of replacing them.

Fixed problematic synchronisation in:

  • uk.org.ogsadai.activity.event.EventfulPipe
  • uk.org.ogsadai.activity.io.LinkedQueuePipe
  • uk.org.ogsadai.activity.io.LinkedQueuePipeFactory

uk.org.ogsadai.activity.io.BufferedPipe

  • Now uses an java.util.ArrayList instead of a java.util.LinkedList

uk.org.ogsadai.activity.io.BufferedPipeFactory

  • Made a minor robustness improvement to check buffer size > 0 and buffer size is now configurable.

uk.org.ogsadai.activity.io.GrowableInMemoryPipe

  • Now uses an java.util.ArrayList instead of a java.util.LinkedList
  • Fixed bug with data errors not being thrown if the pipe was closed due to an error.
  • Now throws a uk.org.ogsadai.activity.io.PipeClosedException if the pipe is closed by the consumer.

uk.org.ogsadai.activity.event.LoggingActivityListener

  • Now logs full stack traces in debug mode.

Added toString() overrides to aid debugging.

  • uk.org.ogsadai.activity.pipeline.ActivityInputLiteral
  • uk.org.ogsadai.activity.pipeline.ListLiteral
  • uk.org.ogsadai.activity.pipeline.SimpleActivityPipeline
  • uk.org.ogsadai.activity.pipeline.SimpleLiteral
  • uk.org.ogsadai.activity.pipeline.StreamLiteral

uk.org.ogsadai.activity.event.LoggingConsoleActivityListener

  • A new activity listener that logs to the console.

uk.org.ogsadai.activity.io.TypedOptionalActivityInput

  • setBlockReader now accepts null arguments and executes as if the method has never been called. This is useful as the caller can just do setBlockReader(getInput(INPUT_NAME)) and all will work well if the input does not exist. Without this the called needs to get the input, check it is not null and then call setBlockReader only when it is not null.

New methods to register and remove engine listeners added to:

  • uk.org.ogsadai.resource.drer.DRER
  • uk.org.ogsadai.engine.RequestQueueingEngine
  • uk.org.ogsadai.resource.drer.SimpleDRER
2.1.7.2.1. Request event activity extension interface

A new request event activity extension interface has been added so an activity can be informed when a request it belongs to completes.

  • Activities can implement this new interface: uk.org.ogsadai.activity.extension.RequestEventActivity
  • This new class initialises the activities: uk.org.ogsadai.activity.extension.RequestEventActivityInitialiser
  • And is now called by uk.org.ogsadai.activity.extension.DefaultActivityInitialisers
  • This new class is an inteface for the dispatcher for the events: uk.org.ogsadai.engine.event.ActivityRequestListener
2.1.7.2.2. Data sink pipe factory

A new extensibility point for data sink pipe factories is supported. A deployer can specify a data sink pipe factory which allows clients to write data to a data sink and not block, the data being cached server-side.

  • uk.org.ogsadai.resource.datasink.SimpleDataSinkResource now uses a pipe factory and a deployer can configure the pipes a data sink uses.
  • uk.org.ogsadai.activity.io.PrimitiveFilePipe provides a pipe implementation based on files.
  • uk.org.ogsadai.activity.io.PrimitiveFilePipeFactory provides a factory for the above.
  • uk.org.ogsadai.context.OGSADAIConstants and uk.org.ogsadai.context.OGSADAIContext now provide a constant for the data sink pipe factory and a getter method for the one currently in the OGSA-DAI server context.
  • OGSA-DAI server configuration files now include a support for specifying the data sink pipe factory to use via a ogsadai/misc/uk.org.ogsadai.DATA_SINK_PIPE_FACTORY JNDI entry:
    • $OGSADAI_HOME/deploy/jndi-config.xml
2.1.7.2.3. Activity status monitoring framework

A new monitoring framework has been implemented which tracks the activity status and the number of blocks produced/consumed by an activity.

  • The following classes implement the monitoring framework:
    • uk.org.ogsadai.monitoring.activity.ActivityProgressMonitoringFramework
    • uk.org.ogsadai.monitoring.activity.ActivityStatusPropertyCallback
    • uk.org.ogsadai.monitoring.activity.ActivityStatusTracker
    • uk.org.ogsadai.monitoring.activity.BlockReaderProgressListener
    • uk.org.ogsadai.monitoring.activity.OutputDetails
    • uk.org.ogsadai.monitoring.activity.PipeProgressListener
  • The following associated client toolkit classes have been added:
    • uk.org.ogsadai.client.toolkit.property.convertor.ActivityProgress
    • uk.org.ogsadai.client.toolkit.property.convertor.ActivityProgressXMLConvertor

2.1.7.3. Clients

Methods to manipulate workflow proxies:

  • uk.org.ogsadai.client.toolkit.ParallelWorkflow - now has methods to get and remove child workflows.
  • uk.org.ogsadai.client.toolkit.PipelineWorkflow - now has methods to remove activities.
  • uk.org.ogsadai.client.toolkit.SequenceWorkflow - now has methods to get and remove child workflows.

uk.org.ogsadai.client.toolkit.DataValueInputStream

  • Fixed bug which didn't shift the byte when reading a single byte value.

uk.org.ogsadai.client.toolkit.ServerFactory

  • Fixed concurrency bug by which the server map wasn't fully loaded when multiple clients accessed it.

uk.org.ogsadai.client.toolkit.activity.SimpleLiteralInputDescriptor and uk.org.ogsadai.client.toolkit.activity.SimplePipeInputDescriptor

  • Added equals and hashCode methods.

uk.org.ogsadai.client.toolkit.exception.ActivityInputsOverspecifiedException and uk.org.ogsadai.client.toolkit.exception.ActivityInputsUnderspecifiedException.

  • Fixed bugs in producing the error messages.

uk.org.ogsadai.client.toolkit.property.convertor.RequestStatusXMLConvertor

  • Moved getAttribute method to uk.org.ogsadai.util.xml.XML.

uk.org.ogsadai.client.toolkit.property.convertor.XMLDataValueFactory

  • Changed to handle empty character arrays in request status XML.

uk.org.ogsadai.client.toolkit.example.DocumentClient

  • Bug fix to sync|async command line flag and also to wait around for the request to finish if an asynchronous request is sent.

uk.org.ogsadai.client.toolkit.example.SQLClient

  • Now prints number of rows returned.
  • Now has a getServer() method for sub-classes to return presentation layer-specific server proxies if required.
  • uk.org.ogsadai.client.toolkit.gt.example.GTSecureSQLClient - now implements the above method to return uk.org.ogsadai.client.toolkit.presentation.gt.GTServer;

uk.org.ogsadai.client.toolkit.example.ServerClient

  • Now lists activities supported by DataRequestExecutionResource as well.

uk.org.ogsadai.client.toolkit.scenarios.ExtractTableSchemaScenario

  • Added tuple type to column metadata in extract table schema activity.

2.1.7.4. Common classes

XML utilities

  • uk.org.ogsadai.util.xml.XML
  • getAttribute method changed to handle namespaces or non-namespaced attributes.
  • Changes to the way XML attributes are handled to variants in XML parsers.

Relational table schema now includes information on OGSA-DAI tuple types (uk.org.ogsadai.tuple.TupleTypes) for each column. Associated updates made to:

  • uk.org.ogsadai.converters.databaseschema.ColumnMetaData
  • uk.org.ogsadai.converters.databaseschema.ColumnMetaDataImpl
  • uk.org.ogsadai.converters.databaseschema.DatabaseSchemaConstants
  • uk.org.ogsadai.converters.databaseschema.TableMetaDataImpl
  • uk.org.ogsadai.converters.databaseschema.fromxml.XMLParsingTableMetaDataImpl
  • uk.org.ogsadai.converters.databaseschema.fromxml.XMLSchemaConverter
  • uk.org.ogsadai.converters.databaseschema.toxml.SchemaToCharArraysConverterImpl
  • uk.org.ogsadai.converters.databaseschema.toxml.SchemaToXMLConverterBase
  • uk.org.ogsadai.converters.databaseschema.toxml.SchemaToXMLConverterImpl

Relational table schema functions now close metadata result sets which previously could cause issues on Oracle Databases due to cursor count exceeding maximum allowed. uk.org.ogsadai.converters.databaseschema.TableMetaDataImpl has now been updated to close the metadata result sets.

uk.org.ogsadai.converters.resultset.webrowset.TextColumnStrategy

  • Now uses uk.org.ogsadai.util.xml.XML.replaceSpecialCharacters() method instead of its own copy of this method, which has now been deleted.

uk.org.ogsadai.converters.webrowset.metadata.ColumnMetaData

  • This class now implements the method ResultSet.getColumnClassName(int) which returns the name of the class that a caller can expect to receive from ResultSet.getObject(int) method calls.
  • Added helper class uk.org.ogsadai.converters.webrowset.resultset.types.SQLTypesToJava which maps from SQL types (in the java.sql.Types) class into Java class names.

Sample data clients

  • uk.org.ogsadai.dbcreate.CreateTestMySQLTables now creates database and user correctly.
  • uk.org.ogsadai.dbcreate.CreateTestDB Ensures createDatabase() abstract methods in sub-classes is now called.

uk.org.ogsadai.physicalschema.impl.MySQLPhysicalSchemaProvider

  • The maximum join size returned can be too big to store in a java.lang.Long so for now it won't return the value if too big.

uk.org.ogsadai.util.BigDecimalUtil

  • New utility class for handling large decimal numbers.

uk.org.ogsadai.util.UniqueName

  • Now uses Java 1.5's java.util.UUID class.

uk.org.ogsadai.util.xml.XML

  • replaceSpecialCharacters() strips out problematic control characters as part of a conversion to XML.

uk.org.ogsadai.util.xml.XML

  • replaceSpecialCharacters() strips out problematic control characters as part of a conversion to XML.

uk.org.ogsadai.activity.request.status.AsynchronousRequestStatus, uk.org.ogsadai.activity.request.status.SimpleRequestStatus, uk.org.ogsadai.resource.request.RequestExecutionStatusListener, uk.org.ogsadai.resource.request.RequestStatus

  • Added request execution status listener.

uk.org.ogsadai.common.ID

  • Now implements java.lang.Comparable.

uk.org.ogsadai.converters.webrowset.tuple.ColumnStrategyFactory

  • Fixed bug in handling of null columns.

uk.org.ogsadai.resource.property.ActivitiesMetaDataPropertyConvertor

  • Changes to the way XML attributes are handled to variants in XML parsers.

uk.org.ogsadai.tuple.Null

  • Added toString() and a static public variable called VALUE.

uk.org.ogsadai.tuple.SimpleTuple

  • Made java.sql.Date handling more robust.

uk.org.ogsadai.tuple.SimpleTupleMetadata

  • Now addes table name, resource ID and DRES URL when creating product meta-data.

.uk.org.ogsadai.tuple.TupleTypes

  • Forgot about this one in the type casting check in.

uk.org.ogsadai.tuple.TupleUtilities

  • Exploits type name string so it can handle unsigned types. Also encodes mappings from OGSA-DAI types to Java classes.

2.1.7.5. Configuration and deployment - changes

$OGSADAI_HOME/setenv.sh.

  • Now correctly picks up JARs in $OGSADAI_HOME/thirdparty/globus/common.

$OGSADAI_HOME/build.xml .

  • permit and deny commands changed to allow spaces in resource IDs.

2.1.7.6. Core classes

uk.org.ogsadai.servlets.DataSourceRetrievalServlet. .

  • Fixed bug that ignored contentType. Now it is used.

Server configuration editor classes

  • The server configuration editor classes are in the package uk.org.ogsadai.tools.
  • There is an example of how they can be used from an ANT script in $OGSADAI_HOME/examples/build-server-editor.xml .
  • $OGSADAI_HOME/build-server.xml commands deployActivity and exposeResourceActivity now use these classes from ANT rather than deploying activities using an ANT regular expression replace. This avoids problems with duplicated activity declarations in configuration files.

uk.org.ogsadai.resource.ResourceFactory and uk.org.ogsadai.resource.SimpleResourceFactory

  • Added methods to create resource state and data resources from class name so the resource factory user has more control when creating a new data resource.

uk.org.ogsadai.resource.request.RequestStatusXML

  • Fixed concurrency bug when iterating through results data lists.

2.1.7.7. Resources

Support for relational database product-specific handling

  • A new interface, uk.org.ogsadai.resource.dataresource.jdbc.JDBCSettings is available for classes that provide access to advanced JDBC settings e.g. fetch size, schema names, OGSA-DAI meta-data handlers and OGSA-DAI JDBC column type mappers.
  • uk.org.ogsadai.resource.dataresource.jdbc.JDBCDataResource now implements this interface.
  • A new provider interface is available for activities that need to a relational resource that provides the above information - uk.org.ogsadai.resource.dataresource.jdbc.JDBCSettingsProvider
  • A new provider interface is available for activities that need to a relational resource that provides the above information as well as connections. This extends both uk.org.ogsadai.resource.dataresource.jdbc.JDBCConnectionProvider andJDBCSettingsProvider and is called uk.org.ogsadai.resource.dataresource.jdbc.EnhancedJDBCConnectionProvider
  • uk.org.ogsadai.resource.dataresource.jdbc.SimpleJDBCConnectionProvider now implements EnhancedJDBCConnectionProvider
  • uk.org.ogsadai.resource.dataresource.jdbc.JDBCDataResourceState and uk.org.ogsadai.resource.dataresource.jdbc.SimpleJDBCDataResourceState now provide corresponding methods to return the information required by JDBCSettings

Product-specific meta-data extraction.

  • A new interface, uk.org.ogsadai.resource.dataresource.jdbc.MetaDataHandler is available for classes to provide product-specific meta-data extaction. This includes providing product-specific uk.org.ogsadai.converters.databaseschema.TableMetaData and uk.org.ogsadai.tuple.TupleMetadata
  • A default implementation uk.org.ogsadai.resource.dataresource.jdbc.DefaultMetaDataHandler is provided.
  • Implementations are also provided for Oracle - uk.org.ogsadai.resource.dataresource.jdbc.OracleMetaDataHandler - PostgreSQL uk.org.ogsadai.resource.dataresource.jdbc.PostGreSQLMetaDataHandler - and MySQL - uk.org.ogsadai.resource.dataresource.jdbc.StreamingMySQLMetaDataHandler

Product-specific mappings from SQL types from JDBC to OGSA-DAI types.

  • A new interface, uk.org.ogsadai.resource.dataresource.jdbc.JDBCColumnTypeMapper is available for classes to provide product-specific mappings of column types.
  • A default implementation uk.org.ogsadai.resource.dataresource.jdbc.DefaultJDBCColumnTypeMapper is provided.
  • Implementations are also provided for MySQL - uk.org.ogsadai.resource.dataresource.jdbc.MySQLColumnTypeMapper
  • There is also an interface uk.org.ogsadai.resource.dataresource.jdbc.JDBCColumnTypeMapperProvider that can be implemented by classes returning JDBCColumnTypeMappers.

Relational resources and driver connection properties

  • uk.org.ogsadai.resource.dataresource.jdbc.JDBCDataResourceState.
  • This class has been changed to that an OGSA-DAI deployer can specify connection properties to be set on the driver within the relational resource configuration file. Any configuration property specified in this file that has a dai.connect. prefix is passed straight to the driver (with the prefix stripped off).
  • This allows Oracle 10g drivers to be forced to return JDBC java.sql.Timestamp instead of Oracle-specific org.oracle.TIMESTAMP via setting the following property in the CONFIG...END block of a relational resource configuration file: 
    dai.connect.oracle.jdbc.J2EE13Compliant=true
  • This allows MySQL drivers to be forced to convert zero-dates (dates where all fields are zero, which are valid in MySQL but not in JDBC) to null via setting the following property in the CONFIG...END block of a relational resource configuration file: 
    dai.connect.zeroDateTimeBehavior=convertToNull

Relational resources and driver SQL statements

  • uk.org.ogsadai.resource.dataresource.jdbc.JDBCDataResourceState.
  • This class has been changed to that an OGSA-DAI deployer can specify SQL statements, to be executed by a driver when a connection is made, within the relational resource configuration file. Any configuration property specified in this file that has a dai.statement. prefix is executed as an SQL statement by the driver.

2.1.7.8. OGSA-DAI GT-specific changes

In 3.1 the user doc for secure client examples mistakenly cited our non-secure SQLClient These have been corrected to cite the secure client GTSecureSQLClient.

New secure ExtractTableSchema client

  • uk.org.ogsadai.client.toolkit.gt.example.GTSecureExtractSchemaClient
  • A new example client that executes an ExtractTableSchema workflow on a secure OGSA-DAI server.

uk.org.ogsadai.service.gt.security.GTSecurityContext

  • Now gives access to the Apache Axis message context via implementation of a new interface uk.org.ogsadai.service.gt.security.MessageContextProvider

2.1.7.9. XML Schema

Request XML Schema - $OGSADAI_HOME/schema/ogsadai/types/Request.xsd

  • Changed so the output pipe name is an xsd:ID and the input pipe name is a xsd:IDREF. It was the other way round but this led to problems when using automatic Tee.
  • This resulted in small changes to

2.1.7.10. User doc

Errors in the user doc have been fixed and new content added to reflect changes made.

2.1.8. Known problems and limitations

Activities

  • The sub-workflow API by which an activity can spawn a sub-workflow (as used in the SQLBag activity for example) is a proof-of-concept only. We intend to refactor the API for invoking sub-workflows at a future date to make it more usable.
  • In the source code there are references to the notion of activity contracts. These are intended to help support the use of activities in different roles e.g. the SQLBag could be exposed as an SQLBag or as an SQLQuery activity. We will elaborate on this concept at a later date (when we have had a chance to experiment).
  • Some of our activities, e.g. 
    uk.org.ogsadai.activity.sql.SQLBagActivity
uk.org.ogsadai.activity.sql.SQLResilientActivity
uk.org.ogsadai.activity.pipeline.AutomaticTee
    use hard-coded activity names to identify the activity or activities in sub-workflows they splice into workflows. This means that you should take care when renaming activities exposed by resources, especially block activities. This problem will be most likely manifested as an unsupported activity error.
  • Creation or deletion of databases within DB2 is not currently supported.
  • Creation or deletion of databases within Oracle is not currently supported.
  • SQL Server IMAGE column type is not currently supported.
  • ObtainFromGridFTP and DeliverToGridFTP do not allow the setting of certain GridFTP parameters.
  • Client toolkit support for XPathQuery and XQuery activities does not currently parse data in a request status into an org.xmldb.api.base.ResourceSet object.

Deployment

  • The JSP pages are prototypes. We may revise and extend the functionality of these at a later date.

Interacting with databases

  • The standard Oracle JDBC driver does not conform to the JDBC API for insertion of BLOBs and so OGSA-DAI does not currently support the addition of BLOBs to tables in Oracle databases.
  • The table-name tag in WebRowSet XML is empty in results produced by Oracle or SQLServer. This is a problem of the JDBC drivers that do not provide table names in the metadata of a java.sql.ResultSet.
  • Meta-data from a database, returned by ExtractTableSchema, can be case-sensitive depending on the database used. For example, MySQL might return a table called littleblackbook while DB2 returns LITTLEBLACKBOOK.
  • If an error occurs during execution of a relational activity then server-side information may be returned to the client. This is because OGSA-DAI does not, as yet, parse java.sql.SQLExceptions from database drivers to identify problems caused by a client (e.g. a syntactically-incorrect SQL query) and those due to server-side failures (e.g. loss of connection to a database).
  • Though this is not OGSA-DAI specific, a user encountered a problem using the standard MySQL driver class org.gjt.mm.mysql.Driver from the mysql-connector-java-5.0.5 driver JAR in conjunction with MySQL version mysql-4.1.20-1.RHEL4.1 running on Red Hat. Even though they had provided the database name as part of the database connection URL, e.g. 
    jdbc:mysql://localhost:3306/dbname
    They had to specify the database name in their queries also, e.g. 
    SELECT * FROM dbname.table

Miscellaneous

  • The uk.org.ogsadai.common.msgs.DAILogger methods can sometimes log the wrong line numbers. Searching the logs will usually reveal where the problem actually arose.
  • If a client does not provide a certificate then using deliverFromGridFTP and deliverToGridFTP in a request the transfer will fail.
  • There is a possible deadlock situation which can affect workflows which use naively the SQLNestedInClauseQuery activity. The issue is described in the activity documentation, SQLNestedInClauseQuery.
  • BLOBs and CLOBs are currently only held in-memory with consquent implications on memory usage.
  • OGSA-DAI data request execution resources and implementations of JDBC, XMLDB and file system data resources do not support use of the destroy operation. It is a no-op.

  • The default OGSA-DAI resource and activity managers 

    uk.org.ogsadai.activity.SimpleActivityManager
uk.org.ogsadai.resource.SimpleResourceManager

    cache activity specifications and resources in in-memory hash tables. They do not check for changes in any persisted configuration associated with the activity specifications or resources once the associated objects are in the cache.

  • The default OGSA-DAI file-based resource persistence and configuration manager uk.org.ogsadai.persistence.file.resource.SimpleFileResourceStateDAO only handles the persistence and restoration of resource properties or session state that are Strings.
  • The WSRF operation SetResourceProperty is not supported by OGSA-DAI services.
  • The OGSA-DAI GT wrapper class uk.org.ogsadai.service.gt.resource.GTResourceProperty methods 
    add(X)
clear()
get(int)
remove(Object)
set(int, Object)
    are currently not supported. These would need to convert from XML fragments into objects.
  • There is no cleanup if client-service communication times out. Service will just reply as usual.
  • The uk.org.ogsadai.common.BinaryLob class contains unimplemented methods which throw java.lang.UnsupportedOperationException.
  • Conversion of java.sql.ResultSet to XML WebRowSet returns empty key-column and map properties in the properties element.

Platforms

  • OGSA-DAI 3.2 does not compile under Java 1.6. This is due to changes in java.sql APIs (in which both additional methods and classes have been added) from earlier versions of Java. The OGSA-DAI team may release a Java 1.6 patch in the near future.
  • The setenv scripts do not set CLASSPATHs correctly under Cygwin.

Security

  • Our implementations of inter-service delivery activities (ObtainFromDataSource and DeliverToDataSink) do not at present support secure inter-service communications. This does not however preclude us providing these at a future date.
  • Encrypted login files are unsupported.

2.1.9. Product stability

The following APIs may be assumed to be held stable in further OGSA-DAI 3 releases. The only exceptions to this are:

  • If a fundamental problem is encountered for a majority of users.
  • Methods we have marked as deprecated in 3.2.
  • In the case of an OGSA-DAI 4 release then any and all APIs may be refactored.

The list of APIs are as follows:

  • Activity API - API for activity developers.
  • Activity framework API.
  • Client toolkit API.
  • Common components configuration API.
  • OGSA-DAI context.
  • Persistence manager API.
  • Presentation layer JNDI API.
  • Presentation layer WSDL. However, we will release new presentation layers with different WSDL descriptions.
  • Resource APIs.
  • Security context API.

The following components may be subject to change in light of requirements to increase functionality or bug fixes. We may release bug fixed components or new versions of existing components. These will not incur API changes though. Note also that these are components that implement OGSA-DAI extensibility points so are designed to be pluggable and replaceable in this way.

  • Authorizer implementations. These are presentation layer and application specific so are intended to be replaceable. We may release new ones for different applications or demonstrating use of different third-party authorization infrastructures.
  • Activity implementations.
  • Activity manager implementation.
  • Common components configuration manager implementation.
  • Persistence manager implementation. The current file-based implementation is simple and lightweight and may not be scalable.
  • Presentation layers. We do not envisage changing existing presentation layers but may release new presentation layers for different applications or platforms.
  • Resource manager implementation.

The following APIs and components may be viewed as "alpha" versions which will be subject to change:

  • Configurable pipes.
  • Workflow transformation.
  • Remote resources.
  • JSP pages. We may release new versions. This has no implications on any other components.
  • Sub-workflow API. This may be engineered to be more usable in a future release. This will possible expose a client toolkit-like API for use in activities that create workflows.

2.2. Release notes for OGSA-DAI 3.1

2.2.1. Main features

The main features of OGSA-DAI 3.1 are as follows.

OGSA-DAI is now released under the Apache 2.0 licence.

OGSA-DAI 3.0 extension packs have been ingested. This includes:

  • Extension pack one's document-based workflow client, data source client and advanced SQL join activities.
  • Examples of dynamic resource creation activities.
  • Examples of remote resources and activities that run queries on remote OGSA-DAI servers.
  • The OGSA-DAI data source servlet.

A number of new components and capabilities are now supported. This includes:

  • A new activity extension interface allows activities to get the URLs of the OGSA-DAI server services.
  • Prototype support for pluggable workflow transformation components.
  • Prototype support for configurable pipes.
  • A cleaner API between presentation layers and the activity framework. The presentation layer can now translate a workflow expressed in a presentation layer-specific representation into a core OGSA-DAI workflow representation before the workflow begins execution. This replaces the late-translation approach of OGSA-DAI 3.0.
  • Resources can now be marked as "private" meaning they are hidden from clients and can only be used within sub-workflows.
  • Source code for example clients and scenarios are now shipped in binary releases.
  • An example workflow monitoring plug-in. The previous release shipped with one that did nothing. The new addition records events to an event list in the OGSA-DAI context which can be browsed via a JSP page.
  • Support for MDS registration.

The following new activities have been added.

  • From the ADMIRE project:
    • ListRandomSplit - splits an input (which may be a list or list of lists etc) into N random partitions.
    • RandomSplit - splits an input into N random partitions.
    • TupleUnionAll - performs a UNION ALL on input tuple lists.
  • From OGSA-DAI WS-DAIX 1.0 with modifications:
    • CharArraysToDOM - converts a list of character arrays into a DOM object.
    • DOMToCharArrays - converts a DOM object into a list of character arrays.
    • AddDocuments - adds XML documents - where each document is a list of XML character arrays.
    • CreateSubCollection - creates a sub-collection.
    • GetDocuments - gets XML documents. The XML documents are output as lists of character arrays.
    • RemoveDocuments - removes XML documents.
    • RemoveSubCollection - removes a sub-collection.
    • XMLListCollections - lists the collections in an XMLDB data resource or sub-collection thereof.
    • XMLListResources - lists the XMLDB resources in an XMLDB data resource or sub-collection thereof.
    • XUpdate - runs an XUpdate.
  • From OGSA-DQP 3.2 extensions for OGSA-DAI 3.0 release:
    • ExtractPhysicalSchemaToXMLActivity - outputs an XML representation of the physical schema of an XML database.
  • From OGSA-DAI 3.0 extension pack one:
    • TupleMergeJoin - does an inner or left outer join of two lists of ordered tuples.
    • SQLNestedInClauseJoin - does an inner or left outer join on tuples arising from execution of an SQL query.
    • SQLNestedInClauseQuery - runs an SQL query with an "IN" clause, populating this clause with tuples provided on its input.
    • Some changes to the extension pack classes were made.
      • Classes located in uk.org.ogsadai.util were relocated to uk.org.ogsadai.activity.util.
      • ErrorIDs defined in uk.org.ogsadai.RelationalJoins1.Messages.java were relocated to uk.org.ogsadai.exception.ErrorID.
      • uk.org.ogsadai.RelationalJoins1.Messages.java was not ingested.
  • From OGSA-DAI 3.0 dynamic resource creation extension pack:
    • CreateRelationalResource - creates a new relational resource when given a template resource ID from a client.
    • ExtendedCreateRelationalResource - creats a new relational resource when given a URL, driver class name, username and password from a client.
    • These activities are not deployed onto an OGSA-DAI server by default.
  • From OGSA-DAI 3.0 remote resources extension pack:
    • RemoteAsynchSQLQuery - runs a CreateDataSource=>DeliverToRequestStatus and then a SQLQuery=>TupleToWebRowSetCharArrays=>WriteToDataSource workflow on a remote OGSA-DAI server.
  • Miscellaneous:
    • TupleProjectByIDS - projects tuples where the column to project on can be specified either by integer index or column name.

OGSA-DAI 3.1 is designed to be backwards compatible with OGSA-DAI 3.0 without the need for recompilation - data resource, activity and presentation layer APIs and service WSDLs remain the same.

A number of bugs have been fixed, components made more efficient or robust.

The user doc has been extensively refactored and extended.

2.2.2. Tested platforms and databases

OGSA-DAI 3.1 has been tested as follows. All tests are run on Linux Red Hat 9 and using Apache ANT 1.7.0

ReleaseJava versionUnit testsDatabase specific testsRelease build testsClient-server tests
OGSA-DAI 3.0 GT 4.0.51.4.2_12YesYesYesYes - this includes security-related tests
OGSA-DAI 3.0 GT 4.0.81.4.2_12YesYesYesYes - this includes security-related tests

Table 2.2. Tested platforms and databases


Database-specific tests have been run upon all the databases listed in Chapter 28, Data resource products with the exception of Oracle. Client-server tests were run using MySQL, eXist and file system resources.

2.2.3. Backwards compatibility

2.2.3.1. OGSA-DAI 3.0

OGSA-DAI 3.1 is designed to be backwards compatible with OGSA-DAI 3.0. What do we mean by this? We mean:

  • We have not changed any API we have promoted as a public extensibility point, that is the APIs used by activity or data resource developers.
  • A user should be able to replace all OGSA-DAI 3.0 JARs (those with ogsadai-3.0) prefixes in their OGSA-DAI deployments with OGSA-DAI 3.1 JARs and not experience any errors about missing or undefined methods. A user should not need to change the configuration of their server at all. Furthermore, any OGSA-DAI 3.0 components they have written should continue to compile using 3.1 JARs without the need to rewrite any code.
  • An OGSA-DAI 3.0 client should be able to interact with an OGSA-DAI 3.1 server without any recoding or recompilation of the client being necessary.
  • An OGSA-DAI 3.1 client should be able to interact with an OGSA-DAI 3.0 server providing it doesn't request 3.1-only functionality.

If you are an OGSA-DAI 3.0 user and experience any problems in using OGSA-DAI 3.1 then please contact us (see Chapter 4, Information, help and support).

Chapter 5, How to upgrade to 3.2 for OGSA-DAI 3.0 and 3.1 users contains information on how to upgrade from OGSA-DAI 3.0 to 3.1.

2.2.3.2. OGSA-DAI 2.2 and earlier

As for OGSA-DAI 3.0, OGSA-DAI 3.1 is not backwards compatible with OGSA-DAI 2.2.

2.2.4. Usage statistics and OGSA-DAI

OGSA-DAI contains functionality that collects statistics on the use of OGSA-DAI and forwards this to the OGSA-DAI team. This is enabled by default in the release.

2.2.5. WSRF

OGSA-DAI services are compliant with the version of WSRF currently bundled with Globus Toolkit 4.0 releases. From the Globus Toolkit user doc core facts ( http://www.globus.org/toolkit/docs/4.0/common/javawscore/JavaWSCoreFacts.html) these are

"...the 2004/06 OASIS WSRF and WSN working draft specifications (with minor fixes to the 1.2-draft-01 published schemas and with the March 2004 version of the WS-Addressing specification)"

2.2.6. Changes and modifications in detail

2.2.6.1. Activities

uk.org.ogsadai.client.toolkit.activities.block.CharArraysResize.

  • addArraySizeInput(int) has been deprecated by addSizeInChars(int)
  • connectArraySizeInput(SingleActivityOutput) has been deprecated by connectSizeInCharsInput(SingleActivityOutput)
  • getOutput() has been deprecated by getResultOutput()

uk.org.ogsadai.activity.block.ListControlledRepeatActivity.

  • Fixed so that it checks that all inputs finish at the same time.

uk.org.ogsadai.activity.block.ListConcatenateActivity

  • When run in partial mode it outputs a list begin then list end marker if if all inputs fail to produce data.
  • No longer counts an error on one input multiple times.

uk.org.ogsadai.activity.block.ListRemoveActivity

  • Changed to support an optional "level" input so that the granularity of a list can be reduced (the list "flattened") by up to N levels.
  • Client toolkit proxy uk.org.ogsadai.client.toolkit.activities.block.ListRemove also changed.

uk.org.ogsadai.activity.delivery.DeliverToHTTPActivity.

  • This has been removed since it didn't do anything!

uk.org.ogsadai.activity.delivery.DeliverToSMTP.

  • Now reads messages from the SMTP server. This fixes a problem with some SMTP servers which close the connection if the responses are not read.
  • uk.org.ogsadai.client.toolkit.activities.delivery.DeliverToSMTP method addToAsArray(String[]) deprecated by addTo(String[]).

uk.org.ogsadai.activity.management.CreateResourceGroupActivity

  • Value of output contant OUTPUT is now deprecated. At the next release it will be renamed from output to result. Change will also apply to uk.org.ogsadai.client.toolkit.activities.management.CreateResourceGroup.
  • addResourceIdsAsArray(String[]) has been deprecated by addResourceIds(String[])
  • getOutput() has been deprecated by getResultOutput()

uk.org.ogsadai.activity.sql.ExtractTableSchemaActivity .

  • Changed so that if there are no tables of the specified name or pattern then it outputs an empty list rather than throwing an exception.
  • Changed so that if the getPrimaryKeys, getImportedKeys or getExportedKeys operations are unsupported by the driver currently in use, the activity continues rather than throw an exception.

uk.org.ogsadai.activity.sql.GetAvailableTablesActivity .

  • Changed so that if the getSchemas operation is unsupported by the driver currently in use, the activity continues, using a wild-card, rather than throw an exception.

uk.org.ogsadai.activity.sql.SQLBagActivity and uk.org.ogsadai.activity.sql.SQLResilientActivity.

  • Subject to major refactoring. A part of this was the removal of support for timeouts on the target resources which included removal of the timeout input and the TIMEOUT and DEFAULT_TIMEOUT constants.
  • Error handling was changed so that:
    • Errors arising from unknown resources in the resource group are handled.
    • A QUERY_FAILED_AT_ALL_RESOURCES error can be raised.
    • Errors in child SQLQuery activities are logged as warnings
    • uk.org.ogsadai.client.toolkit.activities.sql.SQLBag and uk.org.ogsadai.client.toolkit.activities.sql.SQLResilient. addTimeout and connectTimeoutInput methods are now deprecated.

uk.org.ogsadai.activity.sql.SQLQueryActivity and uk.org.ogsadai.activity.sql.SQLParameterisedQueryActivity.

  • Refactored so that if a request is terminated prematurely an attempt is made to cancel the query running in the database. This relies upon the JDBC driver for the database supporting this feature (it's optional for drivers to support this);

uk.org.ogsadai.activity.block.TeeActivity

  • Subject to a major refactoring. A part of this was that the constant CLONE was replaced by INPUT_CLONE.

uk.org.ogsadai.activity.transform.CVSToTupleActivity

  • Now raises an ActivityUserException if a parser error occurs.

uk.org.ogsadai.activity.transform.CVSToTupleActivity

  • Now has a final catch for all unhandled exceptions which wraps these in an ActivityProcessingException.

uk.org.ogsadai.activity.transform.TableMetadataToXMLCharArraysActivity

  • Changed so it can handle an empty list.

uk.org.ogsadai.activity.transform.TupleSplitActivity

  • Changed to support an optional "split" input which allows specific columns to be directed to specific outputs e.g. column 3 to activity output 1.
  • Change also applied to client toolkit proxy uk.org.ogsadai.client.toolkit.activities.transform.TupleSplit

uk.org.ogsadai.client.toolkit.activities.transform.TableMetadataToXMLCharArrays

  • This activity and all the following are deprecated: 
    uk.org.ogsadai.client.toolkit.activities.transform.TableMetadataToXMLCharArrays
uk.org.ogsadai.client.toolkit.activities.transform.XMLCharArraysToTableMetaDataRetriever
uk.org.ogsadai.activity.transform.TableMetadataToXMLCharArraysActivity
uk.org.ogsadai.converters.databaseschema.fromxml.XMLToSchemaConverter
uk.org.ogsadai.converters.databaseschema.fromxml.XMLToSchemaConverterImpl
uk.org.ogsadai.converters.databaseschema.toxml.SchemaToXMLConverter
uk.org.ogsadai.converters.databaseschema.toxml.SchemaToXMLConverterImpl
  • The implementation had 3 problems.
    • If table was non-existant then client would throw an exception when parsing the result data (which includes an empty OGSA-DAI list for non-existant tables).
    • If a wild card was used then the client would only return the first table due to a limitation in the parser.
    • Parsing was very inefficient, making use of regular expression matching.
  • It is recommended that the following activity implementation and client toolkit proxy and associated classes be used: 
    uk.org.ogsadai.activity.transform.TableMetadataToXMLCharArraysListActivity
uk.org.ogsadai.client.toolkit.activities.transform.TableMetadataToXMLCharArraysList
uk.org.ogsadai.converters.databaseschema.fromxml.XMLSchemaConvertor
uk.org.ogsadai.converters.databaseschema.fromxml.XMLParsingTableMetaDataImpl
uk.org.ogsadai.converters.databaseschema.DatabaseSchemaMetaData
uk.org.ogsadai.converters.databaseschema.toxml.SchemaToSingleXMLConverter
uk.org.ogsadai.converters.databaseschema.toxml.SchemaToXMLConverterBase

uk.org.ogsadai.activity.transform.TupleProjectionActivity

  • Changed to catch ColumnNotFoundException and rethrow as an ActivityUserException with an ErrorID of COLUMN_NOT_FOUND_ERROR.

uk.org.ogsadai.activity.transform.TupleProducingCSVHandler

  • No longer throws a NullPointerException if the first row input is null i.e. if there is no CSV input data.

2.2.6.2. Activity framework and engine

Service URL extension interface.

  • Activities can now implement the interface uk.org.ogsadai.activity.extension.ServiceAddressesActivity and be given an object containing the URLs of the OGSA-DAI server services.

A number of activity framework changes were made to allow for custom workflow transformation, factoring in a security context if required. This includes:

  • A workflow transformation interface has been added - uk.org.ogsadai.activity.request.WorkflowTransformation
  • An implementation - uk.org.ogsadai.activity.request.AutomaticTeeWorkflowTransformation is provided which inserts Tee activities where an activity output is connected to two activity inputs.
  • uk.org.ogsadai.context.OGSADAIContext now has a method, getWorkflowTransformation(), to to get any such component if present in the OGSA-DAI context.
  • A developer can provide their own workflow transformation component. If none is specified then the above AutomaticTeeWorkflowTransformation is used.

uk.org.ogsadai.activity.request.WorkflowRequest

  • Added constructor to take in the current security context.
  • This entailed changes to uk.org.ogsadai.service.gt.execution.workflow.GTRequestFactory
  • The original constructor now creates and uses a uk.org.ogsadai.authorization.NullSecurityContext.
  • uk.org.ogsadai.activity.request.WorkflowRequest now gets a WorkflowTransformation from the OGSA-DAI context and applies it - if none can be found then the default is used.

A number of activity framework changes were made to allow for pipes to be configurable. This includes:

  • Adding support for attributes on activity outputs.
  • Cleaning up activity pipe handling.
  • Changes to automatic Tee activity insertion.
  • Renaming uk.org.ogsadai.activity.io.GrowablePipe to uk.org.ogsadai.activity.io.GrowableInMemoryPipe.
  • Changes to allow an activity to specify attributes about pipes that are actually literal inputs rather than pipes.
  • uk.org.ogsadai.context.OGSADAIContext now has a method, getPipeFactory(), to to get any such component if present in the OGSA-DAI context.
  • A developer can provide their own pipe factory. If none is specified then the above uk.org.ogsadai.activity.io.BufferedPipeFactory is used.

uk.org.ogsadai.activity.io.ControlBlock

  • Now implements java.lang.Serializable. It now supports a private readResolve method used in Java object serialization and which returns the one of the three control block values.

uk.org.ogsadai.activity.block.InvalidInputsGranularityException

  • Replaced by two exceptions defined in: src/core/activities/uk/org/ogsadai/activity/block/. These are
    • uk.org.ogsadai.activity.block.InvalidInputGranularityException.
    • uk.org.ogsadai.activity.block.MismatchedInputsGranularityException.

Error ID uk.org.ogsadai.MISMATHCED_INPUTS_GRANULARITY

  • Removed. This incurred changes to: uk.org.ogsadai.activity.block.ListRemoveActivity and uk.org.ogsadai.activity.block.ListControlledRepeatActivity

uk.org.ogsadai.data.DataValue

  • Deprecated isFinalBlock(). It should not have been supported.

New DRER request descriptor

  • uk.org.ogsadai.resource.request.RequestDescriptor and all implementation classes of this (uk.org.ogsadai.resource.drer.EmbeddedWorkflowRequestDescriptor and uk.org.ogsadai.service.gt.execution.GTRequestDescriptor ) are now deprecated.
  • There is a new interface uk.org.ogsadai.resource.request.CandidateRequestDescriptor and class SimpleCandidateRequestDescriptor. This is much the same as RequestDescriptor but instead of a getRequest() method it provides a getWorkflow() method which returns a uk.org.ogsadai.activity.workflow.Workflow. This allows presentation layers to construct workflows built using core activity framework objects and is related to the deprecation of resource factories.
  • The uk.org.ogsadai.resource.drer.DRER interface and SimpleDRER class have a new execute(CandidateRequestDescriptor) method.execute(RequestDescriptor) is now marked as deprecated.
  • CandidateRequestDescriptor also provides a allowPrivateResources() method which returns a flag indicating whether private resources are allowed to be cited in the workflow.

uk.org.ogsadai.activity.request.RequestDescriptor

  • Added method, allowPrivateResources() returns a flag indicating whether private resources are allowed to be cited in the workflow.
  • Added method, getWorkflow() returns a uk.org.ogsadai.activity.workflow.Workflow. This allows presentation layers to construct workflows built using core activity framework objects and is related to the deprecation of resource factories.
  • getRequest() is supported for backwards compatibility but is now marked as deprecated.
  • SimpleRequestDescriptor can hold either a Workflow or (for backwards compatibility) a request Object. A new deprecated setRequest() method provides access to the latter.
  • Changes also affect class SimpleRequestDescriptor.
  • uk.org.ogsadai.activity.SimpleWorkflowRequestDescriptor, which was only used by our SQLBag and SQLResilient tests has been deleted.

2.2.6.3. Clients

uk.org.ogsadai.client.toolkit.presentation.gt.GTResource.

  • Unknown resource property names now throw uk.org.ogsadai.client.toolkit.exception.PropertyNameInvalidException instead of an incorrect ResourceCommsException

uk.org.ogsadai.client.toolkit.presentation.axis.AxisExceptionUtil.buildClientException() uk.org.ogsadai.client.toolkit.presentation.gt.GTExceptionUtil.buildClientException().

  • No longer throws a NullPointerException if an exception it is processing has no causes. This means, for example, that rejected requests from the server now yield a client toolkit ClientException.

uk.org.ogsadai.client.toolkit.example.SQLClient

  • No longer throws a NullPointerException if it tries to print a field with a NULL value. Now it displays null.

The document-based client toolkit classes from extension pack one have been ingested. These allow client applications to read OGSA-DAI workflows from XML files and send them to OGSA-DAI servers. It is an alternative to building the workflows programatically and can be a good way to test OGSA-DAI installations and demonstrate OGSA-DAI's functionality.

The simple data source client from extension pack one has been ingested. This client allows a data held in a data source to be streamed back and printed.

uk.org.ogsadai.client.toolkit.example.ServerClient

  • listResources now prints the resource type as well as the resource IDs.
  • Now allows invocation of destroy so that resources can be destroyed.

uk.org.ogsadai.client.toolkit.example.ClientBase

  • Now supports a printActivityStatus(Activity) method for printing the status of individual activities and any error messages.
  • The clients in uk.org.ogsadai.client.toolkit.example - SQLClient, XMLDBClient and FileClient now call ClientBase.printActivityStatus(Activity) on each activity in the workflow if any exception is thrown by the server.

uk.org.ogsadai.client.toolkit.example.SQLDataSourceClient has been added. This client creates an OGSA-DAI data source, runs an SQL query and exposes the results as CSV via the data source. This can be useful when used with the OGSA-DAI data source servlet.

Example clients in src/client/uk/org/ogsadai/client/toolkit/example and scenarios in src/client/uk/org/ogsadai/client/toolkit/scenarios are bundled in binary distributions in an examples/ directory.

2.2.6.4. Common classes

uk.org.ogsadai.converters.csv.CSVParser

  • No longer misses the last tuple if there is no newline character.

uk.org.ogsadai.converters.tuple.webrowset.TextColumnStrategy.

  • No longer throws a NullPointerException if the first tuple contains empty strings.

uk.org.ogsadai.metadata.MetadataWrapper

  • Now implements java.lang.Serializable.

uk.org.ogsadai.tuple.ColumnMetadata

  • columnNoNulls has been deprecated by COLUMN_NO_NULLS.
  • columnNullable has been deprecated by COLUMN_NULLABLE.
  • columnNullableUnknown has been deprecated by COLUMN_NULLABLE_UNKNOWN.

uk.org.ogsadai.tuple.Tuple, TupleMetadata, ColumnMetadata, SimpleBlob, SimpleClob, Null

  • Now implement or extend java.lang.Serializable.

uk.org.ogsadai.tuple.TupleMetadata

  • getColumnMetadataPosition no longer throws a ColumnNotFoundException.
  • No changes were needed to SimpleTupleMetadata since it always returned -1 in such cases.

uk.org.ogsadai.tuple.Null

  • Now supports the readResolve method used in Java object serialization and which returns the singleton instance held by the object.

2.2.6.5. Configuration and deployment - changes

build.xml.

  • permit and deny now can handle cases where path to OGSA-DAI container (and so to the logins file) has spaces.

localhost.

  • OGSA-DAI server and service deployment commands no longer accept localhost as the dai.host argument. This is because proper host names are required to construct WS-EPRs and ensure that inter-op with other OGSA-DAI, web and grid components (e.g. DQP) can work.

config.txt

  • Now records correct URLs for data source and data sink services after deployment of OGSA-DAI. Previously the data source service URL was set to be that of the data sink service).

deploy/config-blanks/JDBCResource

  • uk.org.ogsadai.SQLBulkLoadTuple activity name was mapped to a non-existent uk.org.ogsadai.SQLBulkLoadTuples activity ID. This has been corrected to be uk.org.ogsadai.SQLBulkLoadTuple.

dai-manager.jsp

  • Replaced by dai-index-gt.jsp.

JSP pages have been rewritten and new ones added. This should still be viewed as prototypes only.

  • Now take a dai.protocol (default http) property. This allows provision of https for secure deployments, for example.

uk.org.ogsadai.persistence.file.TextFileUtils

  • getLines methods now view lines in configuration files that start with # as comments and ignores these.
  • Methods now trim preceding and trailing whitespace from strings. Methods that do searches for blocks or lines of text likewise trim preceding and trailing whitespace. This means that trailing spaces in driver classnames, connection URLs, database usernames and passwords no longer cause problems.

DataRequestExecutionResource

  • Default values for dai.concurrency.queue.length and dai.concurrency.request.pool.size have been increased from 20 and 10 to 150 and 50 respectively, in response to performance evaluations on 3.0.
  • ListControlledRepeat activity is now supported. Its omission was an oversight.

config-blanks/JDBCResource and config-blanks/XMLDBResource

  • The three static resource properties now have an http:// prefix e.g.: 
    http://uk.org.ogsadai.resource.dataresource.product=MySQL
http://uk.org.ogsadai.resource.dataresource.vendor=MySQL
http://uk.org.ogsadai.resource.dataresource.version=5.0

    This is to avoid problems with using OGSA-DAI with security. No code relied upon these names so impact should be minimal.

2.2.6.6. Core classes

Tuples, meta data and WebRowSet

  • Tuple meta-data classes have been extended to suppoort the recording of source table names for data and and the resource IDs and URIs of OGSA-DAI resources and services upon which queries were run. This is useful for integrating OGSA-DAI with
  • Our WebRowSet implementation provides this in its catalog section.

uk.org.ogsadai.config.KeyValueProperties:

  • Added containsKey(Key) and put(KeyValueProperties).
  • This incurred changes to SimpleKeyValueProperties and uk.org.ogsadai.resource.session.SimpleSessionResource.

JDBC, column labels and column names

  • Calls to ResultSetMetaData.getColumnName() have been replaced by calls to ResultSetMetaData.getColumnLabel() at appropriate places as this allows for column aliases to be handled correctly. Currently the name field in tuple meta-data records the column label.

WebRowSet

  • The rowset-type in our implementation is now ResultSet.TYPE_FORWARD_ONLY rather than the integer value of this constant (1033). This is for inter-operability with Sun's WebRowSet.

2.2.6.7. Monitoring framework

An example workflow monitoring plug-in has been provided. The previous release shipped with one that did nothing. The new addition records events to an event list in the OGSA-DAI context.

2.2.6.8. Resources

uk.org.ogsadai.resource.SimpleResourcePropertySet.

  • Now raises ResourceEvent.UPDATE event if an existing resource property is overwritten and a ResourceEvent.ADD event if a new one is added (rather than the other way round!).

uk.org.ogsadai.resource.SimpleResourcePropertyValue

  • In setValue(). a cached DOM value is now cleared when this method is called - prevents value and DOM representation getting out of synch.

uk.org.ogsadai.resource.ResourceManager

  • Added listResources(ResourceType) method.
  • Also affected uk.org.ogsadai.resource.SimpleResourceManager

uk.org.ogsadai.resource.ResourceAccessorProvider

  • A new interface that uk.org.ogsadai.resource.DataResource implementations can implement. If a data resource plug-in implements this interface then it will be provided with the current session and request IDs, in addition to the current security context. These can be used to create a ResourceAccessor for the data resource that factors in the session and/or request, in addition to the security context. Any data resource implementing this interface will have the ResourceAccessorProvider.createResourceAccessor/3 method called instead of the ResourceAccessor.createResourceAccessor/1 method.
  • uk.org.ogsadai.activity.extension.ResourceActivityInitialiser was changed to call the appropriate method above.
  • uk.org.ogsadai.activity.extension.DefaultActivityInitialisers was changed to pass the session and request IDs to ResourceActivityInitialiser.

The OGSA-DAI 3.0 remote resources extension pack has been ingested. This consists of a new OGSA-DAI data resource plug-in which can communicate with a remote OGSA-DAI server. An activity - uk.org.ogsadai.activity.remote.RemoteAsynchSQLQueryActivity - which runs a CreateDataSource=>DeliverToRequestStatus and then a SQLQuery=>TupleToWebRowSetCharArrays=>WriteToDataSource workflow on the remote server has also been supplied.

Support for private resources

  • Resources have been extended to support an optional "private" configuration value. If provided and set to true then the resources will not be accessible via clients via OGSA-DAI services, servlets or within workflows they submit (though will still be accessible in sub-workflows that these workflows might spawn).
  • Interface uk.org.ogsadai.resource.ResourceState, and its sub-class class, SimpleResourceState, have been extended with isPrivateResource() and setPrivateResource() methods.
  • Interface uk.org.ogsadai.resource.ResourceManager, and its implementation class, SimpleResourceManager, have been extended with getPublicResource(ResourceID), getPublicResource(ResourceID, ResourceType), listPublicResources() and listPublicResources(ResourceType) methods.
  • Presentation layer classes and the OGSA-DAI data source servlet now use these new methods.
  • uk.org.ogsadai.activity.management.CreateResourceGroupActivity has been changed to that it will only succeed if all the resources to be members of the group are both known to the server and are marked as public resources.
  • Interface uk.org.ogsadai.activity.request.RequestConfiguration has a new allowPrivateResources() method which returns a boolean and its sub-class uk.org.ogsadai.activity.request.OGSADAIRequestConfiguration inherits the default value from the given uk.org.ogsadai.activity.RequestDescriptor.
  • uk.org.ogsadai.activity.request.OGSADAIChildRequestConfiguration, used in sub-workflows, sets the default value of this flag to true but provides a setAllowPrivateResources(boolean) method to allow this to be overridden (e.g. an activity could override it to prevent sub-workflows accessing private resources).
  • uk.org.ogsadai.activity.SimpleActivityFactory and uk.org.ogsadai.activity.extension.ResourceActivityInitialiser provide constructors which take a boolean about whether private resources can be used or not and respect this when assigning target resources to activities. If a private resource is targeted and private resources are not to be used then a uk.org.ogsadai.resource.ResourceUnknownException is thrown.
  • uk.org.ogsadai.activity.extension.DefaultActivityInitialisers now calls the new ResourceActivityInitialiser constructor with the allowPrivateResources value from the RequestConfiguration.

2.2.6.9. Deprecation of request factory

Interface uk.org.ogsadai.activity.request.RequestFactory is now deprecated.

  • uk.org.ogsadai.resource.initialize.FactoryResourceStateVisitor no longer looks for a RequestFactory with key uk.org.ogsadai.REQUEST_FACTORY in the OGSA-DAI context.In consequence, SimpleDRER and OGSADAIActivityFramework no longer take a RequestFactory as arguments to their constructors.
  • For backwards compatibility, the uk.org.ogsadai.activity.OGSADAIActivityFramework will look for a RequestFactory in the OGSA-DAI context. If it receives an activity.RequestDescriptor that has no Workflow object then it will get a request Object from this and pass it to the factory.
  • The deprecated SimpleDRER.execute(RequestDescriptor) constructs a SimpleRequestDescriptor holding a request Object.
  • The new SimpleDRER.execute(CandidateRequestDescriptor) constructs a SimpleRequestDescriptor holding a Workflow object.
  • uk.org.ogsadai.resource.drer.SimpleRequestFactory has been deprecated.
  • uk.org.ogsadai.service.gt.execution.workflow.GTRequestFactory is now a standalone class for converting beans into a Workflow. It no longer implements the deprecated RequestFactory interface.
  • uk.org.ogsadai.service.gt.execution.GTDataRequestExecutionProvider now uses GTRequestFactory to convert a request bean into a Workflow and also it builds a SimpleCandidateRequestDescriptor for passing to the DRER.
  • uk.org.ogsadai.service.gt.context.GTOGSADAIContextInitializer no longer adds the GTRequestFactory to the OGSA-DAI context.
  • uk.org.ogsadai.service.gt.execution.GTRequestDescriptor has now been removed as it's unused.
  • The constant uk.org.ogsadai.context.OGSADAIConstants.REQUEST_FACTORY has been deprecated.

2.2.6.10. OGSA-DAI context initialisation

uk.org.ogsadai.service.gt.context.GTOGSADAIContextInitializer now initializes OGSA-DAI managers before reading JNDI entries in the misc or logins contexts. This allows optional or application-specific components to use the managers when these components are first created.

2.2.6.11. Servlet

The servlet extension pack - which provides a servlet which allows access to data exposed via OGSA-DAI data sources - has been ingested. This is deployed by default in deployments of OGSA-DAI onto Tomcat. The following should be noted.

  • Concurrent access to a data source retrieval servlet will give unpredictable behaviour.
  • There is no support for secure access to data exposed via the data source servlet. This could lead to privileged data being accessible to non-privileged users.
  • The servlet does not function within the Globus Toolkit stand-alone container.

2.2.6.12. OGSA-DAI GT-specific changes

OGSA-DAI is now tested using GT 4.0.5 and GT 4.0.8.

OGSA-DAI GT now bundles the concurrent.jar and concurrent.LICENSE from the Globus Toolkit. This is required to allow clients to run under GT 4.0.8.

An OGSA-DAI GT PDP which authorizes on the basis of DN's matching a regular expression is available,

Support for MDS registration.

  • Support for MDS registration in OGSA-DAI GT has been added.
  • New classes to support this are now bundled. OGSA-DAI GT source distributions now bundle additional Globus Toolkit JARs to enable these classes to compile.
  • The OGSA-DAI GT JNDI file now includes an entry for the MDS components and example MDS registration files are shipped. Deployment commands have been added to associate MDS registrations with resources, to remove these registrations, to toggle registration on and off and to invoke Globus Toolkit scripts to get information about registered resources.

uk.org.ogsadai.service.gt.intrinsics.GTIntrinsicsProvider

  • If a resource disappears after its ID has been accessed but the resource itself has not yet been accesseed then listResources now just ignores the ID instead of throwing an exception.

2.2.6.13. User doc

The user doc has been refactored to (hopefully!) improve readability and usability and further examples have been added. A list of OGSA-DAI-related publications has also been added.

2.2.7. Known problems and limitations

Activities

  • The sub-workflow API by which an activity can spawn a sub-workflow (as used in the SQLBag activity for example) is a proof-of-concept only. We intend to refactor the API for invoking sub-workflows at a future date to make it more usable.
  • In the source code there are references to the notion of activity contracts. These are intended to help support the use of activities in different roles e.g. the SQLBag could be exposed as an SQLBag or as an SQLQuery activity. We will elaborate on this concept at a later date (when we have had a chance to experiment).
  • Some of our activities, e.g. 
    uk.org.ogsadai.activity.sql.SQLBagActivity
uk.org.ogsadai.activity.sql.SQLResilientActivity
uk.org.ogsadai.activity.pipeline.AutomaticTee
    use hard-coded activity names to identify the activity or activities in sub-workflows they splice into workflows. This means that you should take care when renaming activities exposed by resources, especially block activities. This problem will be most likely manifested as an unsupported activity error.
  • Creation or deletion of databases within DB2 is not currently supported.
  • Creation or deletion of databases within Oracle is not currently supported.
  • SQL Server IMAGE column type is not currently supported.
  • ObtainFromGridFTP and DeliverToGridFTP do not allow the setting of certain GridFTP parameters.
  • Client toolkit support for XPathQuery and XQuery activities does not currently parse data in a request status into an org.xmldb.api.base.ResourceSet object.

Deployment

  • The JSP pages are prototypes. We may revise and extend the functionality of these at a later date.

Interacting with databases

  • The standard Oracle JDBC driver does not conform to the JDBC API for insertion of BLOBs and so OGSA-DAI does not currently support the addition of BLOBs to tables in Oracle databases.
  • The table-name tag in WebRowSet XML is empty in results produced by Oracle or SQLServer. This is a problem of the JDBC drivers that do not provide table names in the metadata of a java.sql.ResultSet.
  • Meta-data from a database, returned by ExtractTableSchema, can be case-sensitive depending on the database used. For example, MySQL might return a table called littleblackbook while DB2 returns LITTLEBLACKBOOK.
  • If an error occurs during execution of a relational activity then server-side information may be returned to the client. This is because OGSA-DAI does not, as yet, parse java.sql.SQLExceptions from database drivers to identify problems caused by a client (e.g. a syntactically-incorrect SQL query) and those due to server-side failures (e.g. loss of connection to a database).
  • Though this is not OGSA-DAI specific, a user encountered a problem using the standard MySQL driver class org.gjt.mm.mysql.Driver from the mysql-connector-java-5.0.5 driver JAR in conjunction with MySQL version mysql-4.1.20-1.RHEL4.1 running on Red Hat. Even though they had provided the database name as part of the database connection URL, e.g. 
    jdbc:mysql://localhost:3306/dbname
    They had to specify the database name in their queries also, e.g. 
    SELECT * FROM dbname.table

Miscellaneous

  • The uk.org.ogsadai.common.msgs.DAILogger methods can sometimes log the wrong line numbers. Searching the logs will usually reveal where the problem actually arose.
  • If a client does not provide a certificate then using deliverFromGridFTP and deliverToGridFTP in a request the transfer will fail.
  • There is a possible deadlock situation which can affect workflows which use naively the SQLNestedInClauseQuery activity. The issue is described in the activity documentation, SQLNestedInClauseQuery.
  • BLOBs and CLOBs are currently only held in-memory with consquent implications on memory usage.
  • OGSA-DAI data request execution resources and implementations of JDBC, XMLDB and file system data resources do not support use of the destroy operation. It is a no-op.

  • The default OGSA-DAI resource and activity managers 

    uk.org.ogsadai.activity.SimpleActivityManager
uk.org.ogsadai.resource.SimpleResourceManager

    cache activity specifications and resources in in-memory hash tables. They do not check for changes in any persisted configuration associated with the activity specifications or resources once the associated objects are in the cache.

  • The default OGSA-DAI file-based resource persistence and configuration manager uk.org.ogsadai.persistence.file.resource.SimpleFileResourceStateDAO only handles the persistence and restoration of resource properties or session state that are Strings.
  • The WSRF operation SetResourceProperty is not supported by OGSA-DAI services.
  • The OGSA-DAI GT wrapper class uk.org.ogsadai.service.gt.resource.GTResourceProperty methods 
    add(X)
clear()
get(int)
remove(Object)
set(int, Object)
    are currently not supported. These would need to convert from XML fragments into objects.
  • There is no cleanup if client-service communication times out. Service will just reply as usual.
  • The uk.org.ogsadai.common.BinaryLob class contains unimplemented methods which throw java.lang.UnsupportedOperationException.
  • Conversion of java.sql.ResultSet to XML WebRowSet returns empty key-column and map properties in the properties element.

Platforms

  • OGSA-DAI 3.2 does not compile under Java 1.6. This is due to changes in java.sql APIs (in which both additional methods and classes have been added) from earlier versions of Java. The OGSA-DAI team may release a Java 1.6 patch in the near future.
  • The setenv scripts do not set CLASSPATHs correctly under Cygwin.

Security

  • Our implementations of inter-service delivery activities (ObtainFromDataSource and DeliverToDataSink) do not at present support secure inter-service communications. This does not however preclude us providing these at a future date.
  • Encrypted login files are unsupported.

2.2.8. Product stability

The following APIs may be assumed to be held stable in further OGSA-DAI 3 releases. The only exceptions to this are:

  • If a fundamental problem is encountered for a majority of users.
  • Methods we have marked as deprecated in 3.1.
  • In the case of an OGSA-DAI 4 release then any and all APIs may be refactored.

The list of APIs are as follows:

  • Activity API - API for activity developers.
  • Activity framework API.
  • Client toolkit API.
  • Common components configuration API.
  • OGSA-DAI context.
  • Persistence manager API.
  • Presentation layer JNDI API.
  • Presentation layer WSDL. However, we will release new presentation layers with different WSDL descriptions.
  • Resource APIs.
  • Security context API.

The following components may be subject to change in light of requirements to increase functionality or bug fixes. We may release bug fixed components or new versions of existing components. These will not incur API changes though. Note also that these are components that implement OGSA-DAI extensibility points so are designed to be pluggable and replaceable in this way.

  • Authorizer implementations. These are presentation layer and application specific so are intended to be replaceable. We may release new ones for different applications or demonstrating use of different third-party authorization infrastructures.
  • Activity implementations.
  • Activity manager implementation.
  • Common components configuration manager implementation.
  • Persistence manager implementation. The current file-based implementation is simple and lightweight and may not be scalable.
  • Presentation layers. We do not envisage changing existing presentation layers but may release new presentation layers for different applications or platforms.
  • Resource manager implementation.

The following APIs and components may be viewed as "alpha" versions which will be subject to change:

  • Configurable pipes.
  • Workflow transformation.
  • Remote resources.
  • JSP pages. We may release new versions. This has no implications on any other components.
  • Sub-workflow API. This may be engineered to be more usable in a future release. This will possible expose a client toolkit-like API for use in activities that create workflows.

2.3. Release 3.0 notes

2.3.1. Usage statistics and OGSA-DAI

OGSA-DAI contains functionality that collects statistics on the use of OGSA-DAI and forwards this to the OGSA-DAI team. This is enabled by default in the release.

2.3.2. Changes from OGSA-DAI 2.2

OGSA-DAI 3.0 is a complete rewrite of OGSA-DAI 2.2. A list of the changes made, why and advice on upgrading applications is given in Appendix AC, Appendix - Upgrading from OGSA-DAI 2.2.

2.3.3. Backwards compatibility

There is no backwards compatibility between OGSA-DAI 3.0 and previous releases of OGSA-DAI.

2.3.4. Product stability

The following APIs may be assumed to be held stable for the forseeable future (12 months). Changes will only be made if a fundamental problem is encountered for a majority of users.

  • Activity API - API for activity developers.
  • Activity framework API.
  • Client toolkit API.
  • Common components configuration API.
  • OGSA-DAI context.
  • Persistence manager API.
  • Presentation layer JNDI API.
  • Presentation layer WSDL. However, we will release new presentation layers with different WSDL descriptions.
  • Resource APIs.
  • Security context API.

The following components may be subject to change in light of requirements to increase functionality or bug fixes. We may release bug fixed components or new versions of existing components. These will not incur API changes though. Note also that these are components that implement OGSA-DAI extensibility points so are designed to be pluggable and replaceable in this way.

  • Authorizer implementations. These are presentation layer and application specific so are intended to be replaceable. We may release new ones for different applications or demonstrating use of different third-party authorization infrastructures.
  • Activity implementations.
  • Activity manager implementation.
  • Common components configuration manager implementation.
  • Persistence manager implementation. The current file-based implementation is simple and lightweight and may not be scalable.
  • Presentation layers. We do not envisage changing existing presentation layers but may release new presentation layers for different applications or platforms.
  • Resource manager implementation.

The following APIs and components may be viewed as "alpha" versions which will be subject to change:

  • JSP pages. We may release new versions. This has no implications on any other components.
  • Sub-workflow API. This may be engineered to be more usable in a future release. This will possible expose a client toolkit-like API for use in activities that create workflows.

2.3.5. Tested platforms and databases

OGSA-DAI 3.0 has been tested as follows. All tests are run on Linux Red Hat 9 under Java 1.4.2_12 and using Apache ANT 1.6.5.

ReleaseUnit testsDatabase specific testsRelease build testsClient-server tests
OGSA-DAI 3.0 GT 4.0.5YesYesYesYes - this includes security-related tests

Table 2.3. Tested platforms and databases


Database-specific tests have been run upon all the databases listed in Chapter 28, Data resource products. Client-server tests were run using MySQL, eXist and file system resources.

2.3.6. WSRF

OGSA-DAI services are compliant with the version of WSRF currently bundled with Globus Toolkit 4.0.5. From the Globus Toolkit user doc core facts ( http://www.globus.org/toolkit/docs/4.0/common/javawscore/JavaWSCoreFacts.html) these are

"...the 2004/06 OASIS WSRF and WSN working draft specifications (with minor fixes to the 1.2-draft-01 published schemas and with the March 2004 version of the WS-Addressing specification)"

Prev   Next
Chapter 1. User doc conventions Home Chapter 3. Licence and copyright