Openbravo Issue Tracking System - Openbravo ERP
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0036573Openbravo ERPY. DBSourceManagerpublic2017-08-02 10:032017-09-21 16:49
Reporteralostale 
Assigned Toalostale 
PriorityimmediateSeveritymajorReproducibilityrandom
StatusclosedResolutionfixed 
PlatformOS5OS Version
Product Version 
Target VersionFixed in Version3.0PR17Q4 
Merge Request Status
Review Assigned ToAugustoMauch
OBNetwork customer
Web browser
ModulesCore
Support ticket
Regression levelQA functional test
Regression date2017-05-25
Regression introduced in releasemain
Regression introduced by commithttp://code.openbravo.com/erp/devel/pi/rev/49ac427a0d6689181afa29b95d7384f78eab2dca [^]
Triggers an Emergency PackNo
Summary0036573: import sample data can get stuck when executed by multiple threads
DescriptionImporting sample data concurrently with several threads can cause locks at JVM level due to usage of non thread safe internal caches.

This behavior is unlike to occur, but if it happens, import sample data process lasts forever.
Steps To ReproduceWithin a multi-core run: ant install.source

Randomly it can get stuck at importing sample data step.

Checking stack traces (jstack):

"pool-1-thread-2" prio=10 tid=0x00007fac2c83f800 nid=0x4c3a runnable [0x00007fac2203f000]
   java.lang.Thread.State: RUNNABLE
    at java.util.HashMap.getEntry(HashMap.java:465)
    at java.util.HashMap.get(HashMap.java:417)
    at org.apache.ddlutils.dynabean.DynaClassCache.getDynaClass(DynaClassCache.java:99)
    at org.apache.ddlutils.dynabean.DynaClassCache.createNewInstance(DynaClassCache.java:50)
    at org.apache.ddlutils.model.Database.createDynaBeanFor(Database.java:1342)
    at org.apache.ddlutils.io.DynaSqlCreateRule.begin(DynaSqlCreateRule.java:61)
    at org.apache.commons.digester.Digester.startElement(Digester.java:1464)
    at org.apache.xerces.parsers.AbstractSAXParser.startElement(Unknown Source)
    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanStartElement(Unknown Source)
    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
    at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
    at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
    at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
    at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
    at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
    at org.apache.commons.digester.Digester.parse(Digester.java:1743)
    at org.apache.ddlutils.io.DatabaseDataIO.writeDataToDatabase(DatabaseDataIO.java:722)
    at org.openbravo.ddlutils.task.ImportSampledata$ImportRunner.importXmlFile(ImportSampledata.java:357)
    at org.openbravo.ddlutils.task.ImportSampledata$ImportRunner.run(ImportSampledata.java:340)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

"pool-1-thread-1" prio=10 tid=0x00007fac2c83f000 nid=0x4c39 runnable [0x00007fac2213f000]
   java.lang.Thread.State: RUNNABLE
    at java.util.HashMap.put(HashMap.java:494)
    at org.apache.ddlutils.dynabean.DynaClassCache.getDynaClass(DynaClassCache.java:103)
    at org.apache.ddlutils.dynabean.DynaClassCache.createNewInstance(DynaClassCache.java:50)
    at org.apache.ddlutils.model.Database.createDynaBeanFor(Database.java:1342)
    at org.apache.ddlutils.io.DynaSqlCreateRule.begin(DynaSqlCreateRule.java:61)
    at org.apache.commons.digester.Digester.startElement(Digester.java:1464)
    at org.apache.xerces.parsers.AbstractSAXParser.startElement(Unknown Source)
    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanStartElement(Unknown Source)
    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
    at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
    at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
    at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
    at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
    at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source)
    at org.apache.commons.digester.Digester.parse(Digester.java:1743)
    at org.apache.ddlutils.io.DatabaseDataIO.writeDataToDatabase(DatabaseDataIO.java:722)
    at org.openbravo.ddlutils.task.ImportSampledata$ImportRunner.importXmlFile(ImportSampledata.java:357)
    at org.openbravo.ddlutils.task.ImportSampledata$ImportRunner.run(ImportSampledata.java:340)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)

Both threads are manipulating the same HashMap concurrently causing, in some cases, infinite loop.
Proposed Solution
Additional Information
TagsNo tags attached.
Relationships
depends on backport 00365743.0PR17Q3 closed alostale import sample data can get stuck when executed by multiple threads 
caused by feature request 0035832 closed AugustoMauch Support using several threads to import and export the sample data 
blocks design defect 0034664 acknowledged Triage Platform Base multi thread unsafeties 
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2017-08-02 10:03alostaleNew Issue
2017-08-02 10:03alostaleAssigned To => alostale
2017-08-02 10:03alostaleModules => Core
2017-08-02 10:03alostaleRegression level => QA functional test
2017-08-02 10:03alostaleRegression date => 2017-05-25
2017-08-02 10:03alostaleRegression introduced in release => main
2017-08-02 10:03alostaleRegression introduced by commit => http://code.openbravo.com/erp/devel/pi/rev/49ac427a0d6689181afa29b95d7384f78eab2dca [^]
2017-08-02 10:03alostaleTriggers an Emergency Pack => No
2017-08-02 10:03alostaleRelationship addedcaused by 0035832
2017-08-02 10:04alostalePrioritynormal => immediate
2017-08-02 10:04alostaleRelationship addedblocks 0034664
2017-08-02 10:05alostaleStatusnew => scheduled
2017-08-02 10:30hgbotCheckin
2017-08-02 10:30hgbotNote Added: 0098322
2017-08-02 10:30hgbotStatusscheduled => resolved
2017-08-02 10:30hgbotResolutionopen => fixed
2017-08-02 10:30hgbotFixed in SCM revision => http://code.openbravo.com/erp/devel/dbsm-main/rev/354fa4d03f404fa6feb04ad691153d249429b1cc [^]
2017-08-02 10:31hgbotCheckin
2017-08-02 10:31hgbotNote Added: 0098323
2017-08-02 10:31hgbotFixed in SCM revisionhttp://code.openbravo.com/erp/devel/dbsm-main/rev/354fa4d03f404fa6feb04ad691153d249429b1cc [^] => http://code.openbravo.com/erp/devel/pi/rev/f258c39df153c6203cda0f84fb26add44b0ec5b0 [^]
2017-08-02 10:32alostaleReview Assigned To => AugustoMauch
2017-08-03 11:35AugustoMauchNote Added: 0098362
2017-08-03 11:35AugustoMauchStatusresolved => closed
2017-08-03 11:35AugustoMauchFixed in Version => 3.0PR17Q4
2017-09-21 16:49hudsonbotCheckin
2017-09-21 16:49hudsonbotNote Added: 0099290

Notes
(0098322)
hgbot   
2017-08-02 10:30   
Repository: erp/devel/dbsm-main
Changeset: 354fa4d03f404fa6feb04ad691153d249429b1cc
Author: Asier Lostalé <asier.lostale <at> openbravo.com>
Date: Wed Aug 02 10:28:46 2017 +0200
URL: http://code.openbravo.com/erp/devel/dbsm-main/rev/354fa4d03f404fa6feb04ad691153d249429b1cc [^]

fixed bug 36573: import sample data can get stuck if executed concurrently

  It was caused by a concurrent modification/read of cache HashMap in DynaClassCache.

  As getDynaClass is the only method that reads and modifies this internal cache,
  the fix is to make it synchronized.

  Checked also, while importing sample data this is the only place where shared
  objects are modified.

---
M src/org/apache/ddlutils/dynabean/DynaClassCache.java
M src/org/openbravo/ddlutils/task/ImportSampledata.java
---
(0098323)
hgbot   
2017-08-02 10:31   
Repository: erp/devel/pi
Changeset: f258c39df153c6203cda0f84fb26add44b0ec5b0
Author: Asier Lostalé <asier.lostale <at> openbravo.com>
Date: Wed Aug 02 10:30:22 2017 +0200
URL: http://code.openbravo.com/erp/devel/pi/rev/f258c39df153c6203cda0f84fb26add44b0ec5b0 [^]

fixed bug 36573: import sample data can get stuck if executed concurrently

  It was caused by a concurrent modification/read of cache HashMap in DynaClassCache.

  As getDynaClass is the only method that reads and modifies this internal cache,
  the fix is to make it synchronized.

  Checked also, while importing sample data this is the only place where shared
  objects are modified.

---
M src-db/database/lib/dbsourcemanager.jar
---
(0098362)
AugustoMauch   
2017-08-03 11:35   
Code reviewed and verified in pi@f258c39df153
(0099290)
hudsonbot   
2017-09-21 16:49   
A changeset related to this issue has been promoted main and to the
Central Repository, after passing a series of tests.

Promotion changeset: https://code.openbravo.com/erp/devel/main/rev/9750b78d3e5c [^]
Maturity status: Test