NPE when using DatabaseSequenceFilter, QueryDataSet and ForwardOnlyResultSetTableFactory on Oracle

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

NPE when using DatabaseSequenceFilter, QueryDataSet and ForwardOnlyResultSetTableFactory on Oracle

Justin Florentine
I have a need to extract filtered data into xml, sequenced correctly for insertion into an empty database.  Eventually, this will need to extract a large amount of data, and from what I saw in the FAQ, using the ForwardOnlyResultSetTableFactory is the recommended means of speeding that up.

The code I'm listing below runs fine;  I get the expected filtered rows in the correct dependent table order.

public class DatabaseExportSample
{

    private static final Logger log = Logger.getLogger(DatabaseExportSample.class);
    public static void main(String[] args) throws Exception
    {
        // database connection
        Properties props = new Properties();
        try {
            FileInputStream fr = new FileInputStream("src/main/resources/testDS.properties");
            props.load(fr);
        } catch (IOException e) {
            log.error("Couldn't load props", e);
            System.exit(-1);
        }
        DriverManager.registerDriver(new OracleDriver());
        Connection jdbcConnection = DriverManager.getConnection(props.getProperty("url"),
                props);

        IDatabaseConnection connection = new DatabaseConnection(jdbcConnection);
        DatabaseConfig dbconf = connection.getConfig();
        //dbconf.setProperty(DatabaseConfig.PROPERTY_RESULTSET_TABLE_FACTORY,
        //                    new ForwardOnlyResultSetTableFactory());
        dbconf.setProperty(DatabaseConfig.PROPERTY_DATATYPE_FACTORY,
                new OracleDataTypeFactory());

        // partial database export
        QueryDataSet partialDataSet = new QueryDataSet(connection);
        partialDataSet.addTable("FMG_DD", "SELECT * FROM FMG_DD WHERE FMG_LOCK='Y'");
        partialDataSet.addTable("FMG_DD_TABLES", "SELECT * FROM FMG_DD_TABLES WHERE FMG_LOCK='Y'");


        final ITableFilter depFilter = new DatabaseSequenceFilter(connection,
                new String[]{"FMG_DD", "FMG_DD_TABLES"});
        final IDataSet dependents = new FilteredDataSet(depFilter,
                partialDataSet);

        XmlDataSet.write(dependents, new FileOutputStream("partial.xml"));

    }
}


If I uncomment out the call that specifies the use of the ForwardOnlyResultSetTableFactory, I get an NPE:

Exception in thread "main" java.lang.NullPointerException
    at org.dbunit.database.ForwardOnlyResultSetTable.getValue(ForwardOnlyResultSetTable.java:86)
    at org.dbunit.dataset.stream.DataSetProducerAdapter.produce(DataSetProducerAdapter.java:102)
    at org.dbunit.dataset.xml.XmlDataSetWriter.write(XmlDataSetWriter.java:130)
    at org.dbunit.dataset.xml.XmlDataSet.write(XmlDataSet.java:104)
    at org.dbunit.dataset.xml.XmlDataSet.write(XmlDataSet.java:91)
    at com.fmgtech.services.util.data.DatabaseExportSample.main(DatabaseExportSample.java:70)
Disconnected from the target VM, address: '127.0.0.1:53166', transport: 'socket'

Any ideas what I'm doing wrong here?  Is there another means of speeding the extract up that I should be looking at instead?



------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge  
This is your chance to win up to $100,000 in prizes! For a limited time,
vendors submitting new applications to BlackBerry App World(TM) will have
the opportunity to enter the BlackBerry Developer Challenge. See full prize  
details at: http://p.sf.net/sfu/Challenge
_______________________________________________
dbunit-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dbunit-user
Reply | Threaded
Open this post in threaded view
|

Re: NPE when using DatabaseSequenceFilter, QueryDataSet and ForwardOnlyResultSetTableFactory on Oracle

Matthias Gommeringer
Hi Justin,

you may also want to increase the values for the JDBC batch and fetch size: http://www.dbunit.org/properties.html
I'm not sure whether the batchedStatement feature has any effect but it may also be worth a try.

I believe the the ForwardOnlyResultSetTableFactory would not bring a big performance gain since it is also used internally by the default CachedResultSetTableFactory.

Did you have a look on the performance figures in detail using a profiler? Maybe it is the XML export that consumes most of the time...

rgds,
matthias

> -----Ursprüngliche Nachricht-----
> Von: "Justin Florentine" <[hidden email]>
> Gesendet: 13.07.09 23:10:42
> An: [hidden email]
> Betreff: [dbunit-user] NPE when using DatabaseSequenceFilter, QueryDataSet and ForwardOnlyResultSetTableFactory on Oracle

I have a need to extract filtered data into xml, sequenced correctly

> for insertion into an empty database. Eventually, this will need to
> extract a large amount of data, and from what I saw in the FAQ, using
> the ForwardOnlyResultSetTableFactory is the recommended means of
> speeding that up.
>
> The code I'm listing below runs fine; I get the expected filtered
> rows in the correct dependent table order.
>
> public class DatabaseExportSample
> {
>
>  private static final Logger log = Logger.getLogger(DatabaseExportSampl
> e.class);
>  public static void main(String[] args) throws Exception
>  {
>  // database connection
>  Properties props = new Properties();
>  try {
>  FileInputStream fr = new FileInputStream("src/main/resources/testDS.
> properties");
>  props.load(fr);
>  } catch (IOException e) {
>  log.error("Couldn't load props", e);
>  System.exit(-1);
>  }
>  DriverManager.registerDriver(new OracleDriver());
>  Connection jdbcConnection = DriverManager.getConnection(props.
> getProperty("url"),
>  props);
>
>  IDatabaseConnection connection = new DatabaseConnection(jdbcConnection
> );
>  DatabaseConfig dbconf = connection.getConfig();
>  //dbconf.setProperty(DatabaseConfig.PROPERTY_RESULTSET_TABLE_FACTORY,
>  // new ForwardOnlyResultSetTableFactory());
>  dbconf.setProperty(DatabaseConfig.PROPERTY_DATATYPE_FACTORY,
>  new OracleDataTypeFactory());
>
>  // partial database export
>  QueryDataSet partialDataSet = new QueryDataSet(connection);
>  partialDataSet.addTable("FMG_DD", "SELECT * FROM FMG_DD WHERE FMG_
> LOCK='Y'");
>  partialDataSet.addTable("FMG_DD_TABLES", "SELECT * FROM FMG_DD_
> TABLES WHERE FMG_LOCK='Y'");
>
>  final ITableFilter depFilter = new DatabaseSequenceFilter(connection,
>  new String[]{"FMG_DD", "FMG_DD_TABLES"});
>  final IDataSet dependents = new FilteredDataSet(depFilter,
>  partialDataSet);
>
>  XmlDataSet.write(dependents, new FileOutputStream("partial.xml"));
>
>  }
> }
>
> If I uncomment out the call that specifies the use of the ForwardOnlyRe
> sultSetTableFactory, I get an NPE:
>
> Exception in thread "main" java.lang.NullPointerException
>  at org.dbunit.database.ForwardOnlyResultSetTable.getValue(ForwardOnlyR
> esultSetTable.java:86)
>  at org.dbunit.dataset.stream.DataSetProducerAdapter.produce(
> DataSetProducerAdapter.java:102)
>  at org.dbunit.dataset.xml.XmlDataSetWriter.write(XmlDataSetWriter.
> java:130)
>  at org.dbunit.dataset.xml.XmlDataSet.write(XmlDataSet.java:104)
>  at org.dbunit.dataset.xml.XmlDataSet.write(XmlDataSet.java:91)
>  at com.fmgtech.services.util.data.DatabaseExportSample.main(
> DatabaseExportSample.java:70)
> Disconnected from the target VM, address: '127.0.0.1:53166',
> transport: 'socket'
>
> Any ideas what I'm doing wrong here? Is there another means of
> speeding the extract up that I should be looking at instead?
>
> ----------------------------------------------------------------------
> -------- Enter the BlackBerry Developer Challenge This is your chance
> to win up to $100,000 in prizes! For a limited time, vendors
> submitting new applications to BlackBerry App World(TM) will have the
> opportunity to enter the BlackBerry Developer Challenge. See full
> prize details at: http://p.sf.net/sfu/Challenge_______________________
> ________________________ dbunit-user mailing list dbunit-user@lists.
> sourceforge.net https://lists.sourceforge.net/lists/listinfo/dbunit-
>
> user


______________________________________________________
GRATIS für alle WEB.DE-Nutzer: Die maxdome Movie-FLAT!
Jetzt freischalten unter http://movieflat.web.de


------------------------------------------------------------------------------
Enter the BlackBerry Developer Challenge  
This is your chance to win up to $100,000 in prizes! For a limited time,
vendors submitting new applications to BlackBerry App World(TM) will have
the opportunity to enter the BlackBerry Developer Challenge. See full prize  
details at: http://p.sf.net/sfu/Challenge
_______________________________________________
dbunit-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dbunit-user