Chapter 2. Release notes

Prev		Next

2.1. Release notes for OGSA-DAI 3.1

2.1.1. Main features

The main features of OGSA-DAI 3.1 are as follows.

OGSA-DAI is now released under the Apache 2.0 licence.

OGSA-DAI 3.0 extension packs have been ingested. This includes:

Extension pack one's document-based workflow client, data source client and advanced SQL join activities.
Examples of dynamic resource creation activities.
Examples of remote resources and activities that run queries on remote OGSA-DAI servers.
The OGSA-DAI data source servlet.

A number of new components and capabilities are now supported. This includes:

A new activity extension interface allows activities to get the URLs of the OGSA-DAI server services.
Prototype support for pluggable workflow transformation components.
Prototype support for configurable pipes.
A cleaner API between presentation layers and the activity framework. The presentation layer can now translate a workflow expressed in a presentation layer-specific representation into a core OGSA-DAI workflow representation before the workflow begins execution. This replaces the late-translation approach of OGSA-DAI 3.0.
Resources can now be marked as "private" meaning they are hidden from clients and can only be used within sub-workflows.
Source code for example clients and scenarios are now shipped in binary releases.
An example workflow monitoring plug-in. The previous release shipped with one that did nothing. The new addition records events to an event list in the OGSA-DAI context which can be browsed via a JSP page.
Support for deployment on Tomcat 5.0 or Tomcat 5.5

The following new activities have been added.

From the ADMIRE project:
- ListRandomSplit - splits an input (which may be a list or list of lists etc) into N random partitions.
- RandomSplit - splits an input into N random partitions.
- TupleUnionAll - performs a UNION ALL on input tuple lists.
From OGSA-DAI WS-DAIX 1.0 with modifications:
- CharArraysToDOM - converts a list of character arrays into a DOM object.
- DOMToCharArrays - converts a DOM object into a list of character arrays.
- AddDocuments - adds XML documents - where each document is a list of XML character arrays.
- CreateSubCollection - creates a sub-collection.
- GetDocuments - gets XML documents. The XML documents are output as lists of character arrays.
- RemoveDocuments - removes XML documents.
- RemoveSubCollection - removes a sub-collection.
- XMLListCollections - lists the collections in an XMLDB data resource or sub-collection thereof.
- XMLListResources - lists the XMLDB resources in an XMLDB data resource or sub-collection thereof.
- XUpdate - runs an XUpdate.
From OGSA-DQP 3.2 extensions for OGSA-DAI 3.0 release:
- ExtractPhysicalSchemaToXMLActivity - outputs an XML representation of the physical schema of an XML database.
From OGSA-DAI 3.0 extension pack one:
- TupleMergeJoin - does an inner or left outer join of two lists of ordered tuples.
- SQLNestedInClauseJoin - does an inner or left outer join on tuples arising from execution of an SQL query.
- SQLNestedInClauseQuery - runs an SQL query with an "IN" clause, populating this clause with tuples provided on its input.
- Some changes to the extension pack classes were made.
  - Classes located in uk.org.ogsadai.util were relocated to uk.org.ogsadai.activity.util.
  - ErrorIDs defined in uk.org.ogsadai.RelationalJoins1.Messages.java were relocated to uk.org.ogsadai.exception.ErrorID.
  - uk.org.ogsadai.RelationalJoins1.Messages.java was not ingested.
From OGSA-DAI 3.0 dynamic resource creation extension pack:
- CreateRelationalResource - creates a new relational resource when given a template resource ID from a client.
- ExtendedCreateRelationalResource - creats a new relational resource when given a URL, driver class name, username and password from a client.
- These activities are not deployed onto an OGSA-DAI server by default.
From OGSA-DAI 3.0 remote resources extension pack:
- RemoteAsynchSQLQuery - runs a CreateDataSource=>DeliverToRequestStatus and then a SQLQuery=>TupleToWebRowSetCharArrays=>WriteToDataSource workflow on a remote OGSA-DAI server.
Miscellaneous:
- TupleProjectByIDS - projects tuples where the column to project on can be specified either by integer index or column name.

OGSA-DAI 3.1 is designed to be backwards compatible with OGSA-DAI 3.0 without the need for recompilation - data resource, activity and presentation layer APIs and service WSDLs remain the same.

A number of bugs have been fixed, components made more efficient or robust.

The user doc has been extensively refactored and extended.

2.1.2. Tested platforms and databases

OGSA-DAI 3.1 has been tested as follows. All tests are run on Linux Red Hat 9 and using Apache ANT 1.7.0

Release	Java version	Unit tests	Database specific tests	Release build tests	Client-server tests
OGSA-DAI 3.0 Axis 1.4	1.4.2_12	Yes	Yes	Yes	Yes - with Tomcat 5.0.28 and Tomcat 5.5.26
OGSA-DAI 3.0 Axis 1.2.1	1.4.2_12	Yes	Yes	Yes	No

Table 2.1. Tested platforms and databases

Database-specific tests have been run upon all the databases listed in Chapter 27, Data resource products with the exception of Oracle. Client-server tests were run using MySQL, eXist and file system resources.

2.1.3. Backwards compatibility

2.1.3.1. OGSA-DAI 3.0

OGSA-DAI 3.1 is designed to be backwards compatible with OGSA-DAI 3.0. What do we mean by this? We mean:

We have not changed any API we have promoted as a public extensibility point, that is the APIs used by activity or data resource developers.
A user should be able to replace all OGSA-DAI 3.0 JARs (those with ogsadai-3.0) prefixes in their OGSA-DAI deployments with OGSA-DAI 3.1 JARs and not experience any errors about missing or undefined methods. A user should not need to change the configuration of their server at all. Furthermore, any OGSA-DAI 3.0 components they have written should continue to compile using 3.1 JARs without the need to rewrite any code.
An OGSA-DAI 3.0 client should be able to interact with an OGSA-DAI 3.1 server without any recoding or recompilation of the client being necessary.
An OGSA-DAI 3.1 client should be able to interact with an OGSA-DAI 3.0 server providing it doesn't request 3.1-only functionality.

If you are an OGSA-DAI 3.0 user and experience any problems in using OGSA-DAI 3.1 then please contact us (see Chapter 4, Information, help and support).

Chapter 32, How to upgrade to 3.1 for OGSA-DAI 3.0 users contains information on how to upgrade from OGSA-DAI 3.0 to 3.1.

2.1.3.2. OGSA-DAI 2.2 and earlier

As for OGSA-DAI 3.0, OGSA-DAI 3.1 is not backwards compatible with OGSA-DAI 2.2.

2.1.4. Usage statistics and OGSA-DAI

OGSA-DAI contains functionality that collects statistics on the use of OGSA-DAI and forwards this to the OGSA-DAI team. This is enabled by default in the release. For information on the statistics collected and how to disable this functionality please see Section 34.1, “Usage statistics”.

2.1.5. WSRF

OGSA-DAI services are compliant with the version of WSRF currently bundled with Globus Toolkit 4.0 releases. From the Globus Toolkit user doc core facts ( http://www.globus.org/toolkit/docs/4.0/common/javawscore/JavaWSCoreFacts.html) these are

"...the 2004/06 OASIS WSRF and WSN working draft specifications (with minor fixes to the 1.2-draft-01 published schemas and with the March 2004 version of the WS-Addressing specification)"

2.1.6. Changes and modifications in detail

2.1.6.1. Activities

uk.org.ogsadai.client.toolkit.activities.block.CharArraysResize.

addArraySizeInput(int) has been deprecated by addSizeInChars(int)
connectArraySizeInput(SingleActivityOutput) has been deprecated by connectSizeInCharsInput(SingleActivityOutput)
getOutput() has been deprecated by getResultOutput()

uk.org.ogsadai.activity.block.ListControlledRepeatActivity.

Fixed so that it checks that all inputs finish at the same time.

uk.org.ogsadai.activity.block.ListConcatenateActivity

When run in partial mode it outputs a list begin then list end marker if if all inputs fail to produce data.
No longer counts an error on one input multiple times.

uk.org.ogsadai.activity.block.ListRemoveActivity

Changed to support an optional "level" input so that the granularity of a list can be reduced (the list "flattened") by up to N levels.
Client toolkit proxy uk.org.ogsadai.client.toolkit.activities.block.ListRemove also changed.

uk.org.ogsadai.activity.delivery.DeliverToHTTPActivity.

This has been removed since it didn't do anything!

uk.org.ogsadai.activity.delivery.DeliverToSMTP.

Now reads messages from the SMTP server. This fixes a problem with some SMTP servers which close the connection if the responses are not read.
uk.org.ogsadai.client.toolkit.activities.delivery.DeliverToSMTP method addToAsArray(String[]) deprecated by addTo(String[]).

uk.org.ogsadai.activity.management.CreateResourceGroupActivity

Value of output contant OUTPUT is now deprecated. At the next release it will be renamed from output to result. Change will also apply to uk.org.ogsadai.client.toolkit.activities.management.CreateResourceGroup.
addResourceIdsAsArray(String[]) has been deprecated by addResourceIds(String[])
getOutput() has been deprecated by getResultOutput()

uk.org.ogsadai.activity.sql.ExtractTableSchemaActivity.

Changed so that if there are no tables of the specified name or pattern then it outputs an empty list rather than throwing an exception.
Changed so that if the getPrimaryKeys, getImportedKeys or getExportedKeys operations are unsupported by the driver currently in use, the activity continues rather than throw an exception.

uk.org.ogsadai.activity.sql.GetAvailableTablesActivity.

Changed so that if the getSchemas operation is unsupported by the driver currently in use, the activity continues, using a wild-card, rather than throw an exception.

uk.org.ogsadai.activity.sql.SQLBagActivity and uk.org.ogsadai.activity.sql.SQLResilientActivity.

Subject to major refactoring. A part of this was the removal of support for timeouts on the target resources which included removal of the timeout input and the TIMEOUT and DEFAULT_TIMEOUT constants.
Error handling was changed so that:
- Errors arising from unknown resources in the resource group are handled.
- A QUERY_FAILED_AT_ALL_RESOURCES error can be raised.
- Errors in child SQLQuery activities are logged as warnings
- uk.org.ogsadai.client.toolkit.activities.sql.SQLBag and uk.org.ogsadai.client.toolkit.activities.sql.SQLResilient. addTimeout and connectTimeoutInput methods are now deprecated.

uk.org.ogsadai.activity.sql.SQLQueryActivity and uk.org.ogsadai.activity.sql.SQLParameterisedQueryActivity.

Refactored so that if a request is terminated prematurely an attempt is made to cancel the query running in the database. This relies upon the JDBC driver for the database supporting this feature (it's optional for drivers to support this);

uk.org.ogsadai.activity.block.TeeActivity

Subject to a major refactoring. A part of this was that the constant CLONE was replaced by INPUT_CLONE.

uk.org.ogsadai.activity.transform.CVSToTupleActivity

Now raises an ActivityUserException if a parser error occurs.

uk.org.ogsadai.activity.transform.CVSToTupleActivity

Now has a final catch for all unhandled exceptions which wraps these in an ActivityProcessingException.

uk.org.ogsadai.activity.transform.TableMetadataToXMLCharArraysActivity

Changed so it can handle an empty list.

uk.org.ogsadai.activity.transform.TupleSplitActivity

Changed to support an optional "split" input which allows specific columns to be directed to specific outputs e.g. column 3 to activity output 1.
Change also applied to client toolkit proxy uk.org.ogsadai.client.toolkit.activities.transform.TupleSplit

uk.org.ogsadai.client.toolkit.activities.transform.TableMetadataToXMLCharArrays

This activity and all the following are deprecated:

uk.org.ogsadai.client.toolkit.activities.transform.TableMetadataToXMLCharArrays
uk.org.ogsadai.client.toolkit.activities.transform.XMLCharArraysToTableMetaDataRetriever
uk.org.ogsadai.activity.transform.TableMetadataToXMLCharArraysActivity
uk.org.ogsadai.converters.databaseschema.fromxml.XMLToSchemaConverter
uk.org.ogsadai.converters.databaseschema.fromxml.XMLToSchemaConverterImpl
uk.org.ogsadai.converters.databaseschema.toxml.SchemaToXMLConverter
uk.org.ogsadai.converters.databaseschema.toxml.SchemaToXMLConverterImpl

The implementation had 3 problems.
- If table was non-existant then client would throw an exception when parsing the result data (which includes an empty OGSA-DAI list for non-existant tables).
- If a wild card was used then the client would only return the first table due to a limitation in the parser.
- Parsing was very inefficient, making use of regular expression matching.

It is recommended that the following activity implementation and client toolkit proxy and associated classes be used:

uk.org.ogsadai.activity.transform.TableMetadataToXMLCharArraysListActivity
uk.org.ogsadai.client.toolkit.activities.transform.TableMetadataToXMLCharArraysList
uk.org.ogsadai.converters.databaseschema.fromxml.XMLSchemaConvertor
uk.org.ogsadai.converters.databaseschema.fromxml.XMLParsingTableMetaDataImpl
uk.org.ogsadai.converters.databaseschema.DatabaseSchemaMetaData
uk.org.ogsadai.converters.databaseschema.toxml.SchemaToSingleXMLConverter
uk.org.ogsadai.converters.databaseschema.toxml.SchemaToXMLConverterBase

uk.org.ogsadai.activity.transform.TupleProjectionActivity

Changed to catch ColumnNotFoundException and rethrow as an ActivityUserException with an ErrorID of COLUMN_NOT_FOUND_ERROR.

uk.org.ogsadai.activity.transform.TupleProducingCSVHandler

No longer throws a NullPointerException if the first row input is null i.e. if there is no CSV input data.

2.1.6.2. Activity framework and engine

Service URL extension interface.

Activities can now implement the interface uk.org.ogsadai.activity.extension.ServiceAddressesActivity and be given an object containing the URLs of the OGSA-DAI server services.

A number of activity framework changes were made to allow for custom workflow transformation, factoring in a security context if required. This includes:

A workflow transformation interface has been added - uk.org.ogsadai.activity.request.WorkflowTransformation
An implementation - uk.org.ogsadai.activity.request.AutomaticTeeWorkflowTransformation is provided which inserts Tee activities where an activity output is connected to two activity inputs.
uk.org.ogsadai.context.OGSADAIContext now has a method, getWorkflowTransformation(), to to get any such component if present in the OGSA-DAI context.
A developer can provide their own workflow transformation component. If none is specified then the above AutomaticTeeWorkflowTransformation is used.

uk.org.ogsadai.activity.request.WorkflowRequest

Added constructor to take in the current security context.
This entailed changes to uk.org.ogsadai.service.axis.execution.workflow.AxisRequestFactory
The original constructor now creates and uses a uk.org.ogsadai.authorization.NullSecurityContext.
uk.org.ogsadai.activity.request.WorkflowRequest now gets a WorkflowTransformation from the OGSA-DAI context and applies it - if none can be found then the default is used.

A number of activity framework changes were made to allow for pipes to be configurable. This includes:

Adding support for attributes on activity outputs.
Cleaning up activity pipe handling.
Changes to automatic Tee activity insertion.
Renaming uk.org.ogsadai.activity.io.GrowablePipe to uk.org.ogsadai.activity.io.GrowableInMemoryPipe.
Changes to allow an activity to specify attributes about pipes that are actually literal inputs rather than pipes.
uk.org.ogsadai.context.OGSADAIContext now has a method, getPipeFactory(), to to get any such component if present in the OGSA-DAI context.
A developer can provide their own workflow transformation component. If none is specified then the above uk.org.ogsadai.activity.io.BufferedPipeFactory is used.

uk.org.ogsadai.activity.io.ControlBlock

Now implements java.lang.Serializable. It now supports a private readResolve method used in Java object serialization and which returns the one of the three control block values.

uk.org.ogsadai.activity.block.InvalidInputsGranularityException

Replaced by two exceptions defined in: src/core/activities/uk/org/ogsadai/activity/block/. These are
- uk.org.ogsadai.activity.block.InvalidInputGranularityException.
- uk.org.ogsadai.activity.block.MismatchedInputsGranularityException.

Error ID uk.org.ogsadai.MISMATHCED_INPUTS_GRANULARITY

Removed. This incurred changes to: uk.org.ogsadai.activity.block.ListRemoveActivity and uk.org.ogsadai.activity.block.ListControlledRepeatActivity

uk.org.ogsadai.data.DataValue

Deprecated isFinalBlock(). It should not have been supported.

New DRER request descriptor

uk.org.ogsadai.resource.request.RequestDescriptor and all implementation classes of this (uk.org.ogsadai.resource.drer.EmbeddedWorkflowRequestDescriptor and uk.org.ogsadai.service.axis.execution.AxisRequestDescriptor ) are now deprecated.
There is a new interface uk.org.ogsadai.resource.request.CandidateRequestDescriptor and class SimpleCandidateRequestDescriptor. This is much the same as RequestDescriptor but instead of a getRequest() method it provides a getWorkflow() method which returns a uk.org.ogsadai.activity.workflow.Workflow. This allows presentation layers to construct workflows built using core activity framework objects and is related to the deprecation of resource factories.
The uk.org.ogsadai.resource.drer.DRER interface and SimpleDRER class have a new execute(CandidateRequestDescriptor) method.execute(RequestDescriptor) is now marked as deprecated.
CandidateRequestDescriptor also provides a allowPrivateResources() method which returns a flag indicating whether private resources are allowed to be cited in the workflow.

uk.org.ogsadai.activity.request.RequestDescriptor

Added method, allowPrivateResources() returns a flag indicating whether private resources are allowed to be cited in the workflow.
Added method, getWorkflow() returns a uk.org.ogsadai.activity.workflow.Workflow. This allows presentation layers to construct workflows built using core activity framework objects and is related to the deprecation of resource factories.
getRequest() is supported for backwards compatibility but is now marked as deprecated.
SimpleRequestDescriptor can hold either a Workflow or (for backwards compatibility) a request Object. A new deprecated setRequest() method provides access to the latter.
Changes also affect class SimpleRequestDescriptor.
uk.org.ogsadai.activity.SimpleWorkflowRequestDescriptor, which was only used by our SQLBag and SQLResilient tests has been deleted.

2.1.6.3. Clients

uk.org.ogsadai.client.toolkit.presentation.axis.AxisResource .

Unknown resource property names now throw uk.org.ogsadai.client.toolkit.exception.PropertyNameInvalidException instead of an incorrect ResourceCommsException

uk.org.ogsadai.client.toolkit.presentation.axis.AxisExceptionUtil.buildClientException() uk.org.ogsadai.client.toolkit.presentation.gt.GTExceptionUtil.buildClientException().

No longer throws a NullPointerException if an exception it is processing has no causes. This means, for example, that rejected requests from the server now yield a client toolkit ClientException.

uk.org.ogsadai.client.toolkit.example.SQLClient

No longer throws a NullPointerException if it tries to print a field with a NULL value. Now it displays null.

The document-based client toolkit classes from extension pack one have been ingested. These allow client applications to read OGSA-DAI workflows from XML files and send them to OGSA-DAI servers. It is an alternative to building the workflows programatically and can be a good way to test OGSA-DAI installations and demonstrate OGSA-DAI's functionality.

The simple data source client from extension pack one has been ingested. This client allows a data held in a data source to be streamed back and printed.

uk.org.ogsadai.client.toolkit.example.ServerClient

listResources now prints the resource type as well as the resource IDs.
Now allows invocation of destroy so that resources can be destroyed.

uk.org.ogsadai.client.toolkit.example.ClientBase

Now supports a printActivityStatus(Activity) method for printing the status of individual activities and any error messages.
The clients in uk.org.ogsadai.client.toolkit.example - SQLClient, XMLDBClient and FileClient now call ClientBase.printActivityStatus(Activity) on each activity in the workflow if any exception is thrown by the server.

uk.org.ogsadai.client.toolkit.example.SQLDataSourceClient has been added. This client creates an OGSA-DAI data source, runs an SQL query and exposes the results as CSV via the data source. This can be useful when used with the OGSA-DAI data source servlet.

Example clients in src/client/uk/org/ogsadai/client/toolkit/example and scenarios in src/client/uk/org/ogsadai/client/toolkit/scenarios are bundled in binary distributions in an examples/ directory.

2.1.6.4. Common classes

uk.org.ogsadai.converters.csv.CSVParser

No longer misses the last tuple if there is no newline character.

uk.org.ogsadai.converters.tuple.webrowset.TextColumnStrategy.

No longer throws a NullPointerException if the first tuple contains empty strings.

uk.org.ogsadai.metadata.MetadataWrapper

Now implements java.lang.Serializable.

uk.org.ogsadai.tuple.ColumnMetadata

columnNoNulls has been deprecated by COLUMN_NO_NULLS.
columnNullable has been deprecated by COLUMN_NULLABLE.
columnNullableUnknown has been deprecated by COLUMN_NULLABLE_UNKNOWN.

uk.org.ogsadai.tuple.Tuple, TupleMetadata, ColumnMetadata, SimpleBlob, SimpleClob, Null

Now implement or extend java.lang.Serializable.

uk.org.ogsadai.tuple.TupleMetadata

getColumnMetadataPosition no longer throws a ColumnNotFoundException.
No changes were needed to SimpleTupleMetadata since it always returned -1 in such cases.

uk.org.ogsadai.tuple.Null

Now supports the readResolve method used in Java object serialization and which returns the singleton instance held by the object.

2.1.6.5. Configuration and deployment - changes

build.xml.

permit and deny now can handle cases where path to OGSA-DAI container (and so to the logins file) has spaces.

localhost.

OGSA-DAI server and service deployment commands no longer accept localhost as the dai.host argument. This is because proper host names are required to construct WS-EPRs and ensure that inter-op with other OGSA-DAI, web and grid components (e.g. DQP) can work.

config.txt

Now records correct URLs for data source and data sink services after deployment of OGSA-DAI. Previously the data source service URL was set to be that of the data sink service).

deploy/config-blanks/JDBCResource

uk.org.ogsadai.SQLBulkLoadTuple activity name was mapped to a non-existent uk.org.ogsadai.SQLBulkLoadTuples activity ID. This has been corrected to be uk.org.ogsadai.SQLBulkLoadTuple.

dai-manager.jsp

Replaced by dai-index-axis.jsp .

JSP pages have been rewritten and new ones added. This should still be viewed as prototypes only.

OGSA-DAI 3.0 Axis deployment targets.

Now take a dai.protocol (default http) property. This allows provision of https for secure deployments, for example.

uk.org.ogsadai.persistence.file.TextFileUtils

getLines methods now view lines in configuration files that start with # as comments and ignores these.
Methods now trim preceding and trailing whitespace from strings. Methods that do searches for blocks or lines of text likewise trim preceding and trailing whitespace. This means that trailing spaces in driver classnames, connection URLs, database usernames and passwords no longer cause problems.

DataRequestExecutionResource

Default values for dai.concurrency.queue.length and dai.concurrency.request.pool.size have been increased from 20 and 10 to 150 and 50 respectively, in response to performance evaluations on 3.0.
ListControlledRepeat activity is now supported. Its omission was an oversight.

config-blanks/JDBCResource and config-blanks/XMLDBResource

The three static resource properties now have an http:// prefix e.g.:
```
http://uk.org.ogsadai.resource.dataresource.product=MySQL
http://uk.org.ogsadai.resource.dataresource.vendor=MySQL
http://uk.org.ogsadai.resource.dataresource.version=5.0
```
This is to avoid problems with using OGSA-DAI with security. No code relied upon these names so impact should be minimal.

2.1.6.6. Core classes

Tuples, meta data and WebRowSet

Tuple meta-data classes have been extended to suppoort the recording of source table names for data and and the resource IDs and URIs of OGSA-DAI resources and services upon which queries were run. This is useful for integrating OGSA-DAI with
Our WebRowSet implementation provides this in its catalog section.

uk.org.ogsadai.config.KeyValueProperties:

Added containsKey(Key) and put(KeyValueProperties).
This incurred changes to SimpleKeyValueProperties and uk.org.ogsadai.resource.session.SimpleSessionResource.

JDBC, column labels and column names

Calls to ResultSetMetaData.getColumnName() have been replaced by calls to ResultSetMetaData.getColumnLabel() at appropriate places as this allows for column aliases to be handled correctly. Currently the name field in tuple meta-data records the column label.

WebRowSet

The rowset-type in our implementation is now ResultSet.TYPE_FORWARD_ONLY rather than the integer value of this constant (1033). This is for inter-operability with Sun's WebRowSet.

2.1.6.7. Monitoring framework

An example workflow monitoring plug-in has been provided. The previous release shipped with one that did nothing. The new addition records events to an event list in the OGSA-DAI context.

2.1.6.8. Resources

uk.org.ogsadai.resource.SimpleResourcePropertySet.

Now raises ResourceEvent.UPDATE event if an existing resource property is overwritten and a ResourceEvent.ADD event if a new one is added (rather than the other way round!).

uk.org.ogsadai.resource.SimpleResourcePropertyValue

In setValue(). a cached DOM value is now cleared when this method is called - prevents value and DOM representation getting out of synch.

uk.org.ogsadai.resource.ResourceManager

Added listResources(ResourceType) method.
Also affected uk.org.ogsadai.resource.SimpleResourceManager

uk.org.ogsadai.resource.ResourceAccessorProvider

A new interface that uk.org.ogsadai.resource.DataResource implementations can implement. If a data resource plug-in implements this interface then it will be provided with the current session and request IDs, in addition to the current security context. These can be used to create a ResourceAccessor for the data resource that factors in the session and/or request, in addition to the security context. Any data resource implementing this interface will have the ResourceAccessorProvider.createResourceAccessor/3 method called instead of the ResourceAccessor.createResourceAccessor/1 method.
uk.org.ogsadai.activity.extension.ResourceActivityInitialiser was changed to call the appropriate method above.
uk.org.ogsadai.activity.extension.DefaultActivityInitialisers was changed to pass the session and request IDs to ResourceActivityInitialiser.

The OGSA-DAI 3.0 remote resources extension pack has been ingested. This consists of a new OGSA-DAI data resource plug-in which can communicate with a remote OGSA-DAI server. An activity - uk.org.ogsadai.activity.remote.RemoteAsynchSQLQueryActivity - which runs a CreateDataSource=>DeliverToRequestStatus and then a SQLQuery=>TupleToWebRowSetCharArrays=>WriteToDataSource workflow on the remote server has also been supplied.

Support for private resources

Resources have been extended to support an optional "private" configuration value. If provided and set to true then the resources will not be accessible via clients via OGSA-DAI services, servlets or within workflows they submit (though will still be accessible in sub-workflows that these workflows might spawn).
Interface uk.org.ogsadai.resource.ResourceState, and its sub-class class, SimpleResourceState, have been extended with isPrivateResource() and setPrivateResource() methods.
Interface uk.org.ogsadai.resource.ResourceManager, and its implementation class, SimpleResourceManager, have been extended with getPublicResource(ResourceID), getPublicResource(ResourceID, ResourceType), listPublicResources() and listPublicResources(ResourceType) methods.
Presentation layer classes and the OGSA-DAI data source servlet now use these new methods.
uk.org.ogsadai.activity.management.CreateResourceGroupActivity has been changed to that it will only succeed if all the resources to be members of the group are both known to the server and are marked as public resources.
Interface uk.org.ogsadai.activity.request.RequestConfiguration has a new allowPrivateResources() method which returns a boolean and its sub-class uk.org.ogsadai.activity.request.OGSADAIRequestConfiguration inherits the default value from the given uk.org.ogsadai.activity.RequestDescriptor.
uk.org.ogsadai.activity.request.OGSADAIChildRequestConfiguration, used in sub-workflows, sets the default value of this flag to true but provides a setAllowPrivateResources(boolean) method to allow this to be overridden (e.g. an activity could override it to prevent sub-workflows accessing private resources).
uk.org.ogsadai.activity.SimpleActivityFactory and uk.org.ogsadai.activity.extension.ResourceActivityInitialiser provide constructors which take a boolean about whether private resources can be used or not and respect this when assigning target resources to activities. If a private resource is targeted and private resources are not to be used then a uk.org.ogsadai.resource.ResourceUnknownException is thrown.
uk.org.ogsadai.activity.extension.DefaultActivityInitialisers now calls the new ResourceActivityInitialiser constructor with the allowPrivateResources value from the RequestConfiguration.

2.1.6.9. Deprecation of request factory

Interface uk.org.ogsadai.activity.request.RequestFactory is now deprecated.

uk.org.ogsadai.resource.initialize.FactoryResourceStateVisitor no longer looks for a RequestFactory with key uk.org.ogsadai.REQUEST_FACTORY in the OGSA-DAI context.In consequence, SimpleDRER and OGSADAIActivityFramework no longer take a RequestFactory as arguments to their constructors.
For backwards compatibility, the uk.org.ogsadai.activity.OGSADAIActivityFramework will look for a RequestFactory in the OGSA-DAI context. If it receives an activity.RequestDescriptor that has no Workflow object then it will get a request Object from this and pass it to the factory.
The deprecated SimpleDRER.execute(RequestDescriptor) constructs a SimpleRequestDescriptor holding a request Object.
The new SimpleDRER.execute(CandidateRequestDescriptor) constructs a SimpleRequestDescriptor holding a Workflow object.
uk.org.ogsadai.resource.drer.SimpleRequestFactory has been deprecated.
uk.org.ogsadai.service.axis.execution.workflow.AxisRequestFactory is now a standalone class for converting beans into a Workflow. It no longer implements deprecated RequestFactory interface.
uk.org.ogsadai.service.axis.execution.AxisDataRequestExecutionProvider now uses AxisRequestFactory to convert a request bean into a Workflow and also it builds a SimpleCandidateRequestDescriptor for passing to the DRER.
uk.org.ogsadai.service.axis.context.AxisOGSADAIContextInitializer no longer adds the AxisRequestFactory to the OGSA-DAI context.
uk.org.ogsadai.service.axis.execution.AxisRequestDescriptor has now been removed as it's unused.
The constant uk.org.ogsadai.context.OGSADAIConstants.REQUEST_FACTORY has been deprecated.

2.1.6.10. OGSA-DAI context initialisation

uk.org.ogsadai.service.axis.context.AxisOGSADAIContextInitializer now initializes OGSA-DAI managers before reading JNDI entries in the misc or logins contexts. This allows optional or application-specific components to use the managers when these components are first created.

2.1.6.11. Servlet

The servlet extension pack - which provides a servlet which allows access to data exposed via OGSA-DAI data sources - has been ingested. This is deployed by default in deployments of OGSA-DAI onto Tomcat. The following should be noted.

Concurrent access to a data source retrieval servlet will give unpredictable behaviour.
There is no support for secure access to data exposed via the data source servlet. This could lead to privileged data being accessible to non-privileged users.

2.1.6.12. OGSA-DAI Axis-specific changes

OGSA-DAI Axis supports deployment on Tomcat 5.0 or Tomcat 5.5

The binary distribution file deploy/war-context.xml has been replaced by deploy/war-context-Tomcat-5.0.xml and deploy/war-context-Tomcat-5.5.xml to support this.
Deployment targets support a dai.tomcat.version flag to specify whether 5.0 or 5.5 is being assumed. For backwards compatibility 5.0 is the default.
We will no longer explicitly support 5.0 from the next release since it has now been archived by Apache.

uk.org.ogsadai.service.axis.intrinsics.AxisIntrinsicsProvider

If a resource disappears after its ID has been accessed but the resource itself has not yet been accesseed then listResources now just ignores the ID instead of throwing an exception.

2.1.6.13. User doc

The user doc has been refactored to (hopefully!) improve readability and usability and further examples have been added. A list of OGSA-DAI-related publications has also been added.

2.1.7. Known problems and limitations

Activities

The sub-workflow API by which an activity can spawn a sub-workflow (as used in the SQLBag activity for example) is a proof-of-concept only. We intend to refactor the API for invoking sub-workflows at a future date to make it more usable.
In the source code there are references to the notion of activity contracts. These are intended to help support the use of activities in different roles e.g. the SQLBag could be exposed as an SQLBag or as an SQLQuery activity. We will elaborate on this concept at a later date (when we have had a chance to experiment).
Some of our activities, e.g.
```
uk.org.ogsadai.activity.sql.SQLBagActivity
uk.org.ogsadai.activity.sql.SQLResilientActivity
uk.org.ogsadai.activity.pipeline.AutomaticTee
```
use hard-coded activity names to identify the activity or activities in sub-workflows they splice into workflows. This means that you should take care when renaming activities exposed by resources, especially block activities. This problem will be most likely manifested as an unsupported activity error.
Creation or deletion of databases within DB2 is not currently supported.
Creation or deletion of databases within Oracle is not currently supported.
SQL Server IMAGE column type is not currently supported.
deliverFromGridFTP and deliverToGridFTP do not allow the setting of certain GridFTP parameters.
Client toolkit support for XPathQuery and XQuery activities does not currently parse data in a request status into an org.xmldb.api.base.ResourceSet object.

Deployment

The JSP pages are prototypes. We may revise and extend the functionality of these at a later date.

Interacting with databases

The standard Oracle JDBC driver does not conform to the JDBC API for insertion of BLOBs and so OGSA-DAI does not currently support the addition of BLOBs to tables in Oracle databases.
The table-name tag in WebRowSet XML is empty in results produced by Oracle or SQLServer. This is a problem of the JDBC drivers that do not provide table names in the metadata of a java.sql.ResultSet.
Meta-data from a database, returned by ExtractTableSchema, can be case-sensitive depending on the database used. For example, MySQL might return a table called littleblackbook while DB2 returns LITTLEBLACKBOOK.
If an error occurs during execution of a relational activity then server-side information may be returned to the client. This is because OGSA-DAI does not, as yet, parse java.sql.SQLExceptions from database drivers to identify problems caused by a client (e.g. a syntactically-incorrect SQL query) and those due to server-side failures (e.g. loss of connection to a database).
Though this is not OGSA-DAI specific, a user encountered a problem using the standard MySQL driver class org.gjt.mm.mysql.Driver from the mysql-connector-java-5.0.5 driver JAR in conjunction with MySQL version mysql-4.1.20-1.RHEL4.1 running on Red Hat. Even though they had provided the database name as part of the database connection URL, e.g.
```
jdbc:mysql://localhost:3306/dbname
```
They had to specify the database name in their queries also, e.g.
```
SELECT * FROM dbname.table
```

Miscellaneous

The uk.org.ogsadai.common.msgs.DAILogger methods can sometimes log the wrong line numbers. Searching the logs will usually reveal where the problem actually arose.
If a client does not provide a certificate then using deliverFromGridFTP and deliverToGridFTP in a request the transfer will fail.
There is a possible deadlock situation which can affect workflows which use naively the SQLNestedInClauseQuery activity. The issue is described in the activity documentation, SQLNestedInClauseQuery.
BLOBs and CLOBs are currently only held in-memory with consquent implications on memory usage.
OGSA-DAI data request execution resources and implementations of JDBC, XMLDB and file system data resources do not support use of the destroy operation. It is a no-op.
The default OGSA-DAI resource and activity managers
```
uk.org.ogsadai.activity.SimpleActivityManager
uk.org.ogsadai.resource.SimpleResourceManager
```
cache activity specifications and resources in in-memory hash tables. They do not check for changes in any persisted configuration associated with the activity specifications or resources once the associated objects are in the cache.
The default OGSA-DAI file-based resource persistence and configuration manager uk.org.ogsadai.persistence.file.resource.SimpleFileResourceStateDAO only handles the persistence and restoration of resource properties or session state that are Strings.
The WSRF operation SetResourceProperty is not supported by OGSA-DAI services.
The OGSA-DAI GT wrapper class uk.org.ogsadai.service.gt.resource.GTResourceProperty methods
```
add(X)
clear()
get(int)
remove(Object)
set(int, Object)
```
are currently not supported. These would need to convert from XML fragments into objects.
There is no cleanup if client-service communication times out. Service will just reply as usual.
The uk.org.ogsadai.common.BinaryLob class contains unimplemented methods which throw java.lang.UnsupportedOperationException.
Conversion of java.sql.ResultSet to XML WebRowSet returns empty key-column and map properties in the properties element.

Platforms

OGSA-DAI 3.1 does not compile under Java 1.6. This is due to changes in java.sql APIs (in which both additional methods and classes have been added) from earlier versions of Java. The OGSA-DAI team may release a Java 1.6 patch in the near future.
The setenv scripts do not set CLASSPATHs correctly under Cygwin.

Security

Our implementations of inter-service delivery activities (ObtainFromDataSource and DeliverToDataSink) do not at present support secure inter-service communications. This does not however preclude us providing these at a future date.
Encrypted login files are unsupported.

2.1.8. Product stability

The following APIs may be assumed to be held stable in further OGSA-DAI 3 releases. The only exceptions to this are:

If a fundamental problem is encountered for a majority of users.
Methods we have marked as deprecated in 3.1.
In the case of an OGSA-DAI 4 release then any and all APIs may be refactored.

The list of APIs are as follows:

Activity API - API for activity developers.
Activity framework API.
Client toolkit API.
Common components configuration API.
OGSA-DAI context.
Persistence manager API.
Presentation layer JNDI API.
Presentation layer WSDL. However, we will release new presentation layers with different WSDL descriptions.
Resource APIs.
Security context API.

The following components may be subject to change in light of requirements to increase functionality or bug fixes. We may release bug fixed components or new versions of existing components. These will not incur API changes though. Note also that these are components that implement OGSA-DAI extensibility points so are designed to be pluggable and replaceable in this way.

Authorizer implementations. These are presentation layer and application specific so are intended to be replaceable. We may release new ones for different applications or demonstrating use of different third-party authorization infrastructures.
Activity implementations.
Activity manager implementation.
Common components configuration manager implementation.
Persistence manager implementation. The current file-based implementation is simple and lightweight and may not be scalable.
Presentation layers. We do not envisage changing existing presentation layers but may release new presentation layers for different applications or platforms.
Resource manager implementation.

The following APIs and components may be viewed as "alpha" versions which will be subject to change:

Configurable pipes.
Workflow transformation.
Remote resources.
JSP pages. We may release new versions. This has no implications on any other components.
Sub-workflow API. This may be engineered to be more usable in a future release. This will possible expose a client toolkit-like API for use in activities that create workflows.

2.2. Release 3.0 notes

2.2.1. Usage statistics and OGSA-DAI

2.2.2. Changes from OGSA-DAI 2.2

OGSA-DAI 3.0 is a complete rewrite of OGSA-DAI 2.2. A list of the changes made, why and advice on upgrading applications is given in Appendix Y, Appendix - Upgrading from OGSA-DAI 2.2.

2.2.3. Backwards compatibility

There is no backwards compatibility between OGSA-DAI 3.0 and previous releases of OGSA-DAI.

2.2.4. Product stability

The following APIs may be assumed to be held stable for the forseeable future (12 months). Changes will only be made if a fundamental problem is encountered for a majority of users.

Activity API - API for activity developers.
Activity framework API.
Client toolkit API.
Common components configuration API.
OGSA-DAI context.
Persistence manager API.
Presentation layer JNDI API.
Presentation layer WSDL. However, we will release new presentation layers with different WSDL descriptions.
Resource APIs.
Security context API.

Authorizer implementations. These are presentation layer and application specific so are intended to be replaceable. We may release new ones for different applications or demonstrating use of different third-party authorization infrastructures.
Activity implementations.
Activity manager implementation.
Common components configuration manager implementation.
Persistence manager implementation. The current file-based implementation is simple and lightweight and may not be scalable.
Presentation layers. We do not envisage changing existing presentation layers but may release new presentation layers for different applications or platforms.
Resource manager implementation.

The following APIs and components may be viewed as "alpha" versions which will be subject to change:

JSP pages. We may release new versions. This has no implications on any other components.
Sub-workflow API. This may be engineered to be more usable in a future release. This will possible expose a client toolkit-like API for use in activities that create workflows.

2.2.5. Tested platforms and databases

OGSA-DAI 3.0 has been tested as follows. All tests are run on Linux Red Hat 9 under Java 1.4.2_12 and using Apache ANT 1.6.5.

Release	Unit tests	Database specific tests	Release build tests	Client-server tests
OGSA-DAI 3.0 Axis 1.4	Yes	Yes	Yes	Yes - with Tomcat 5.0
OGSA-DAI 3.0 Axis 1.2.1	Yes	Yes	Yes	No

Table 2.2. Tested platforms and databases

Database-specific tests have been run upon all the databases listed in Chapter 27, Data resource products. Client-server tests were run using MySQL, eXist and file system resources.

Prev		Next
Chapter 1. User doc conventions	Home	Chapter 3. Licence and copyright