0038269: parallel export.sampledata can get stuck

Description

When export.sampledata is run with multiple threads in parallel, the process can randomly get stuck.

Steps To Reproduce

Note the problem reproduces randomly

1. Run ant export.sample.data -Dclient="F&B International Group"
-> It can get stuck

Proposed Solution

When this problem occurs, the stack trace is similar to:

"pool-2-thread-3" prio=10 tid=0x00007f8780067000 nid=0x309d runnable [0x00007f875c3a3000]
   java.lang.Thread.State: RUNNABLE
        at java.util.HashMap.put(HashMap.java:495)
        at org.postgresql.jdbc.CacheMetadata.setCache(CacheMetadata.java:37)
        at org.postgresql.jdbc.PgResultSetMetaData.fetchFieldMetaData(PgResultSetMetaData.java:268)
        at org.postgresql.jdbc.PgResultSetMetaData.getBaseTableName(PgResultSetMetaData.java:318)
        at org.postgresql.jdbc.PgResultSetMetaData.getTableName(PgResultSetMetaData.java:313)
        at org.apache.ddlutils.platform.ModelBasedResultSetIterator.initFromMetaData(ModelBasedResultSetIterator.java:123)
        at org.apache.ddlutils.platform.ModelBasedResultSetIterator.<init>(ModelBasedResultSetIterator.java:100)
        at org.apache.ddlutils.platform.PlatformImplBase.createResultSetIterator(PlatformImplBase.java:2888)
        at org.apache.ddlutils.io.DatabaseDataIO.readRowsFromTableList(DatabaseDataIO.java:785)
        at org.apache.ddlutils.io.DatabaseDataIO.writeDataForTableToXML(DatabaseDataIO.java:455)
        at org.apache.ddlutils.io.DatabaseDataIO.exportDataSet(DatabaseDataIO.java:399)
        at org.openbravo.ddlutils.task.ExportSampledata$ExportDataSetRunner.run(ExportSampledata.java:411)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1152)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:622)
        at java.lang.Thread.run(Thread.java:748)

The problem is caused by PostgreSQL JDBC metadata cache which is implemented in current jdbc version (9.4.1207) as an static field [1].

This is fixed in newer JDBC versions (starting from 9.4.1209) PR551 [2] and PR552 [3].

So proposed solution is to update jdbc to latest version.

---
[1] https://github.com/pgjdbc/pgjdbc/blob/REL9.4.1207/pgjdbc/src/main/java/org/postgresql/jdbc/CacheMetadata.java#L12 [^]
[2] https://github.com/pgjdbc/pgjdbc/pull/551 [^]
[3] https://github.com/pgjdbc/pgjdbc/pull/552 [^]

Tags

No tags attached.

Attached Files

Relationships [ Relation Graph ] [ Dependency Graph ]

related to	feature request	0035832		closed	AugustoMauch	Support using several threads to import and export the sample data
causes	defect	0038271		closed	alostale	pg connections set "PostgreSQL JDBC Driver" as application name

Notes
(0103669) hgbot (developer) 2018-04-04 14:11	Repository: erp/devel/pi Changeset: f0156943cb5e389e280ff5cfc82a62bbcbec97a1 Author: Asier Lostalé <asier.lostale <at> openbravo.com> Date: Tue Apr 03 15:53:31 2018 +0200 URL: http://code.openbravo.com/erp/devel/pi/rev/f0156943cb5e389e280ff5cfc82a62bbcbec97a1 [^] fixed bug 38269: parallel export.sampledata can get stuck Updated PostgreSQL JDBC from version 9.4.1207 to 42.2.2, which solves static metadata cache that caused export.sampledata to get randomly stuck when it was executed in multi-thread. --- M build.xml M legal/Licensing.txt A lib/runtime/postgresql-42.2.2.jre7.jar R lib/runtime/postgresql-9.4.1207.jre6.jar ---

(0104362) hudsonbot (developer) 2018-05-09 19:29	A changeset related to this issue has been promoted main and to the Central Repository, after passing a series of tests. Promotion changeset: https://code.openbravo.com/erp/devel/main/rev/2be7d3efe606 [^] Maturity status: Test

Issue History
Date Modified	Username	Field	Change
2018-04-04 14:05	alostale	New Issue
2018-04-04 14:05	alostale	Assigned To	=> alostale
2018-04-04 14:05	alostale	Modules	=> Core
2018-04-04 14:05	alostale	Triggers an Emergency Pack	=> No
2018-04-04 14:07	alostale	Relationship added	related to 0035832
2018-04-04 14:08	alostale	Review Assigned To	=> AugustoMauch
2018-04-04 14:08	alostale	Steps to Reproduce Updated	View Revisions
2018-04-04 14:09	alostale	Steps to Reproduce Updated	View Revisions
2018-04-04 14:11	hgbot	Checkin
2018-04-04 14:11	hgbot	Note Added: 0103669
2018-04-04 14:11	hgbot	Status	new => resolved
2018-04-04 14:11	hgbot	Resolution	open => fixed
2018-04-04 14:11	hgbot	Fixed in SCM revision	=> http://code.openbravo.com/erp/devel/pi/rev/f0156943cb5e389e280ff5cfc82a62bbcbec97a1 [^]
2018-04-04 16:59	alostale	Relationship added	causes 0038271
2018-04-05 11:28	AugustoMauch	Status	resolved => closed
2018-04-05 11:28	AugustoMauch	Fixed in Version	=> 3.0PR18Q3
2018-05-09 19:29	hudsonbot	Checkin
2018-05-09 19:29	hudsonbot	Note Added: 0104362