slow metadata query

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

slow metadata query

David Kensche
Hello dbunit users,
I have a problem with the performance of dbUnit table import and export.
It takes up about 1/3 of the whole time needed for executing my unit tests. I have the impression that this is a bit much as the tables that I use in my tests are very small. I used a profiler to track down the problem and found that the query for table metadata is extremely slow. In particular DatabaseTableMetaData.
tableExists(..) takes about 30% of the runtime. Maybe an expensive join is being made here, so this would be DBMS-specific (BTW, I am using "MS SQL Server 2008 R2").
Actually, I therefore fear that this problem cannot be fixed as the metadata query is part of the jdbc driver. However, maybe it would be already helpful to allow switching off the following lines from org.dbunit.database.DatabaseTableMetaData.DatabaseTableMetaData(..) as a non-existent table will cause an understandable exception during import anyhow and this one call to tableExists seems to be the bottleneck.

                     if(!metadataHandler.tableExists(databaseMetaData, schemaName, plainTableName))
                     {
                         throw new NoSuchTableException("Did not find table '" + plainTableName + "' in schema '" + schemaName + "'");
                     }

Or maybe I have just missed some configuration option?
Has anyone else faced this problem?
 
Below I attached the stack back trace together with the fractions of time spend in the methods. Maybe this is of help.

  Thanks beforehand,
    David


     com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(TDSCommand) (The method calls itself recursively)
    380.642    100 %
         com.microsoft.sqlserver.jdbc.SQLServerStatement.executeCommand(TDSCommand)
    374.595    98 %
             com.microsoft.sqlserver.jdbc.SQLServerStatement.executeStatement(TDSCommand)
    370.367    97 %
                 com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeQueryInternal()
    144.177    38 %
                     com.microsoft.sqlserver.jdbc.SQLServerDatabaseMetaData.getResultSetFromStoredProc(String, SQLServerDatabaseMetaData$CallableHandles, String[])
       
                         com.microsoft.sqlserver.jdbc.SQLServerDatabaseMetaData.getResultSetWithProvidedColumnNames(String, SQLServerDatabaseMetaData$CallableHandles, String[], String[])
       
                             com.microsoft.sqlserver.jdbc.SQLServerDatabaseMetaData.getTables(String, String, String, String[])
    120.276    32 %
                                 org.apache.commons.dbcp.DelegatingDatabaseMetaData.getTables(String, String, String, String[])
       
                                     org.dbunit.database.DefaultMetadataHandler.tableExists(DatabaseMetaData, String, String)
    113.394    30 %
                                         org.dbunit.database.DatabaseTableMetaData.<init>(String, IDatabaseConnection, boolean, boolean)
       
                                             org.dbunit.database.DatabaseDataSet.getTableMetaData(String)
       
                                                 org.dbunit.database.DatabaseTableIterator.getTableMetaData()
    58.552    15 %
                                                     org.dbunit.dataset.filter.AbstractTableFilter$FilterIterator.next()
       
                                                         org.dbunit.dataset.stream.DataSetProducerAdapter.produce()
       
                                                             org.dbunit.dataset.xml.FlatXmlWriter.write(IDataSet)
       
                                                                 org.dbunit.dataset.xml.FlatXmlDataSet.write(IDataSet, OutputStream)
       
                                                                     com.tn_ag.util.test.DbUnitUtil.selectedTableExport(OutputStream, String[], String[])
       
                                                 org.dbunit.operation.DeleteAllOperation.execute(IDatabaseConnection, IDataSet)
    54.842    14 %
                                                     org.dbunit.operation.CompositeOperation.execute(IDatabaseConnection, IDataSet)
       
                                                         com.tn_ag.util.test.DbUnitUtil.databaseImport(String)
       
                                                             com.tn_ag.tnp.structure.dao.ClassDaoTest.setUp()
    21.495    6 %
                                                             com.tn_ag.tnp.structure.dao.StructureTransferSchemaDaoTest.establishPrecondition()
    16.775    4 %
                                                             com.tn_ag.tnp.structure.dao.CustomAttributeDaoTest.establishPrecondition()
    16.572    4 %
                                     org.dbunit.database.DefaultMetadataHandler.getTables(DatabaseMetaData, String, String[])
    6.881    2 %
                             com.microsoft.sqlserver.jdbc.SQLServerDatabaseMetaData.getColumns(String, String, String, String)
    18.023    5 %
                             com.microsoft.sqlserver.jdbc.SQLServerDatabaseMetaData.getPrimaryKeys(String, String, String)
    5.877    2 %
                 com.microsoft.sqlserver.jdbc.SQLServerStatement.execute(String)
    111.655    29 %
                 com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.execute()
    69.500    18 %
                 com.microsoft.sqlserver.jdbc.SQLServerStatement.executeQuery(String)
    34.773    9 %
                 com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeBatch()
    4.653    1 %
                 com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeQuery()
    3.862    1 %
                 com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.executeUpdate()
    1.743    0 %
             com.microsoft.sqlserver.jdbc.SQLServerPreparedStatement.closePreparedHandle()
    4.227    1 %
         com.microsoft.sqlserver.jdbc.SQLServerConnection.connectionCommand(String, String)
    5.883    2 %
         com.microsoft.sqlserver.jdbc.SQLServerConnection.connectHelper(ServerPortPlaceHolder, int, int)
    164    0 %

------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2dcopy2
_______________________________________________
dbunit-user mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/dbunit-user