com.univocity.api.config
Class MetadataSettings

java.lang.Object
  extended by com.univocity.api.config.MetadataSettings

public final class MetadataSettings
extends Object

This class provides the required configuration for uniVocity's metadata handling. If this configuration is not provided by the user, then a in-memory instance of a database with the essential metadata structure will be created automatically. This of course, means any metadata information will be lost when the application is shut down.

uniVocity requires two database tables only: by default, these have the names "univocity_metadata" and "univocity_tmp". The scripts to create these tables and associated indexes are provided with the univocity implementation package, under the "metadata" folder.

Important: always refer to the script in the implementation package as it could have been modified to adapt to the specific implementation requirements.

Just as a reference, the table creation script should be similar to the following:


        CREATE TABLE univocity_metadata (
                source                          VARCHAR(256) NOT NULL,
                source_id                       VARCHAR(256) NOT NULL,
                source_hash                     INTEGER,
                destination                     VARCHAR(256) NOT NULL,
                destination_id          VARCHAR(256) NOT NULL,
                destination_hash        INTEGER,
                flag                            CHARACTER(1) DEFAULT 'N',
                batch_id                        BIGINT DEFAULT NULL,
                CONSTRAINT pk_univocity_md PRIMARY KEY (source, destination, source_id, destination_id)
        );

        CREATE INDEX u_md_dst_idx ON univocity_metadata (source, destination, destination_id);

  CREATE TABLE univocity_tmp (
                table_name                      VARCHAR(256),
                table_id                        VARCHAR(256),
                batch_id                        BIGINT,
                CONSTRAINT pk_univocity_tmp PRIMARY KEY (table_name, table_id, batch_id)
        )
 

Author:
uniVocity Software Pty Ltd - dev@univocity.com
See Also:
EngineConfiguration

Constructor Summary
MetadataSettings(DataSource dataSource)
          Creates a new metadata setting configuration with a DataSource.
 
Method Summary
 int getBatchSize()
          Obtains the number of metadata rows to be persisted in a single batch execution.
 DatabaseCapabilities getDatabaseInformation()
          Obtains an object describing the capabilities of the database that stores uniVocity metadata.
 DataSource getDataSource()
           
 int getFetchSize()
          Obtains the number of rows to return for an open ResultSet in a each each trip to the database.
 String getMetadataTableName()
          Obtains the configured metadata table name configured for uniVocity.
 String getTemporaryTableName()
          Obtains the configured table name for storing temporary data produced during uniVocity's metadata operations.
 int getTransactionIsolationLevel()
          Obtains the transaction isolation level used when persisting metadata.
 int getTransactionTimeout()
          Obtains the timeout, in number of seconds, for transactions created for metadata manipulation.
 void setBatchSize(int batchSize)
          Defines the number of metadata rows to be persisted in a single batch execution.
 void setDatabaseInformation(DatabaseCapabilities databaseInformation)
          Defines the capabilities of the database that stores uniVocity metadata.
 void setFetchSize(int fetchSize)
          Defines the number of rows to return for an open ResultSet in a each each trip to the database.
 void setMetadataTableName(String metadataTableName)
          Defines the name of the table configured for uniVocity metadata storage.
 void setTemporaryTableName(String temporaryTableName)
          Defines the name of the table that should be used by uniVocity's to store temporary data for its metadata operations.
 void setTransactionIsolationLevel(int transactionIsolationLevel)
          Defines the transaction isolation level used when persisting metadata.
 void setTransactionTimeout(int transactionTimeout)
          Defines a timeout, in number of seconds, for transactions created for metadata manipulation.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

MetadataSettings

public MetadataSettings(DataSource dataSource)
Creates a new metadata setting configuration with a DataSource. The data source must provide a connection to the database that contains the metadata tables.

Parameters:
dataSource - the factory that provides connections to a database configured to use the necessary metadata tables.
Method Detail

getBatchSize

public final int getBatchSize()
Obtains the number of metadata rows to be persisted in a single batch execution.

This setting has an effect only when DatabaseCapabilities.isBatchSupported() evaluates to true.

Batching database operations greatly improves performance in general, but you might want to adjust the batch size to better control memory usage and batch duration.

Defaults to 10,000 rows

Returns:
the batch size to use when persisting metadata records into the configured metadata table
See Also:
DatabaseCapabilities

setBatchSize

public final void setBatchSize(int batchSize)
Defines the number of metadata rows to be persisted in a single batch execution.

This setting has an effect only when DatabaseCapabilities.isBatchSupported() evaluates to true.

Batching database operations greatly improves performance in general, but you might want to adjust the batch size to better control memory usage and batch duration.

Parameters:
batchSize - the batch size to use when persisting metadata records into the configured metadata table
See Also:
DatabaseCapabilities

getMetadataTableName

public final String getMetadataTableName()
Obtains the configured metadata table name configured for uniVocity.

Defaults to "univocity_metadata"

Returns:
the name of the metadata table

setMetadataTableName

public final void setMetadataTableName(String metadataTableName)
Defines the name of the table configured for uniVocity metadata storage.

Parameters:
metadataTableName - the name of the metadata table

getTemporaryTableName

public final String getTemporaryTableName()
Obtains the configured table name for storing temporary data produced during uniVocity's metadata operations.

Defaults to "univocity_tmp"

Returns:
the name of the temporary metadata table

setTemporaryTableName

public final void setTemporaryTableName(String temporaryTableName)
Defines the name of the table that should be used by uniVocity's to store temporary data for its metadata operations.

Parameters:
temporaryTableName - the name of the temporary metadata table

getDatabaseInformation

public final DatabaseCapabilities getDatabaseInformation()
Obtains an object describing the capabilities of the database that stores uniVocity metadata.

By default, uniVocity tries to detect this information automatically.

Returns:
the database capabilities information

setDatabaseInformation

public final void setDatabaseInformation(DatabaseCapabilities databaseInformation)
Defines the capabilities of the database that stores uniVocity metadata.

By default, uniVocity tries to detect this information automatically. The configuration you set here takes precedence over any auto-detected data.

Parameters:
databaseInformation - the database information.

getDataSource

public final DataSource getDataSource()
Returns:
the DataSource that provides connections to a database configured to use the necessary metadata tables.

getTransactionTimeout

public final int getTransactionTimeout()
Obtains the timeout, in number of seconds, for transactions created for metadata manipulation.

Transactions will be created automatically by uniVocity while performing data mapping operations between two entities.
A new transaction will be created for each data mapping cycle executed with DataIntegrationEngine.executeCycle().

Defaults to -1 (no timeout)

Returns:
the transaction timeout.

setTransactionTimeout

public final void setTransactionTimeout(int transactionTimeout)
Defines a timeout, in number of seconds, for transactions created for metadata manipulation.

Transactions will be created automatically by uniVocity while performing data mapping cycles that involve multiple entity mappings.
A new transaction will be created for mapping cycle executed with DataIntegrationEngine.executeCycle().

Parameters:
transactionTimeout - the transaction timeout.

getTransactionIsolationLevel

public final int getTransactionIsolationLevel()
Obtains the transaction isolation level used when persisting metadata.
The transaction isolation level code returned by this method is passed on to Connection.setTransactionIsolation(int) before writing to the metadata table

Defaults to Connection.TRANSACTION_REPEATABLE_READ

Returns:
the transaction isolation level

setTransactionIsolationLevel

public final void setTransactionIsolationLevel(int transactionIsolationLevel)
Defines the transaction isolation level used when persisting metadata.
The transaction isolation level code returned by this method is passed on to Connection.setTransactionIsolation(int) before writing to the metadata table

Only the constants defined in java.sql.Connection are accepted.

Parameters:
transactionIsolationLevel - the transaction isolation level

getFetchSize

public final int getFetchSize()
Obtains the number of rows to return for an open ResultSet in a each each trip to the database.

uniVocity passes this value directly to your database driver using Statement.setFetchSize(int).

For performance reasons, you might want to adjust the fetch size to reflect the common number of rows returned for metadata operations. A big fetch size number might consume too many resources and will be excessive for a small number of records. A small fetch size for too many rows will cause slowness (e.g. a fetch size of 100 to read 10,000 rows and will generate 100 roundtrips to the database).

Defaults to 10,000 rows

Returns:
the fetch size to use when reading metadata.
See Also:
Statement

setFetchSize

public final void setFetchSize(int fetchSize)
Defines the number of rows to return for an open ResultSet in a each each trip to the database.

uniVocity passes this value directly to your database driver using Statement.setFetchSize(int).

For performance reasons, you might want to adjust the fetch size to reflect the common number of rows returned for metadata operations. A big fetch size number might consume too many resources and will be excessive for a small number of records. A small fetch size for too many rows will cause slowness (e.g. a fetch size of 100 to read 10,000 rows and will generate 100 roundtrips to the database).

Parameters:
fetchSize - the fetch size to use when reading metadata.
This parameter is not validated as some database drivers accept special settings for the fetch size, such as Integer.MIN_VALUE.
See Also:
Statement


Copyright © 2015 uniVocity Software Pty Ltd. All rights reserved.