Openbravo Issue Tracking System - Modules
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0043309ModulesExternal Integration Infrastructurepublic2020-02-24 16:462020-02-27 09:00
Reportercaristu 
Assigned Tocaristu 
PriorityhighSeveritycriticalReproducibilityhave not tried
StatusclosedResolutionfixed 
PlatformOS5OS Version
Product Version 
Target VersionFixed in Version 
Merge Request Status
Regression date
Regression introduced by commit
Regression level
Review Assigned ToAugustoMauch
Support ticket
OBNetwork customer
Regression introduced in release
Summary0043309: Memory leak in SBOImporterStore and SBOExporterStore
DescriptionThe SBOImporterStore is an @ApplicationScoped bean that centralizes the generation of SBOImporters. To do so, it has an injection to point to retrieve the SBOImporter beans (which are @Dependant scoped) and retrieves the correct instance depending on an Entity Mapping in particular (this is because currently we support two kind of mapping types: ManualBOB and RetailAPI)

The problem is that all these created instances are living as long as the bean instance they are injected in is living. As the SBOImporterStore is @ApplicationScoped they are never being garbage collected which can eventually lead to OutOfMemory errors (see attached images with a memory dump of an environment affected by this issue).

This very same problem also happens with the SBOExporterStore.

See here[1] for a more detailed explanation regarding this problem.

[1] https://blog.akquinet.de/2017/01/04/dont-get-trapped-into-a-memory-leak-using-cdi-instance-injection/ [^]
Steps To ReproduceIn description
Proposed SolutionAvoid injecting the SBOImporter and SBOExportes in their corresponding application scoped stores.
Additional Information
TagsNo tags attached.
Relationships
depends on backport 0043313 closed caristu Memory leak in SBOImporterStore and SBOExporterStore 
related to feature request 0043314 new Triage Platform Base Get SBOImporters/SBOExporters dynamically 
Attached Filespng image_2020_02_24T15_51_14_891Z.png (23,797) 2020-02-24 16:53
https://issues.openbravo.com/file_download.php?file_id=14080&type=bug
png

png image_2020_02_24T15_52_31_473Z.png (157,291) 2020-02-24 16:54
https://issues.openbravo.com/file_download.php?file_id=14081&type=bug
png
Issue History
Date ModifiedUsernameFieldChange
2020-02-24 16:46caristuNew Issue
2020-02-24 16:46caristuAssigned To => platform
2020-02-24 16:46caristuAssigned Toplatform => caristu
2020-02-24 16:46caristuReview Assigned To => AugustoMauch
2020-02-24 16:53caristuDescription Updatedbug_revision_view_page.php?rev_id=20429#r20429
2020-02-24 16:53caristuFile Added: image_2020_02_24T15_51_14_891Z.png
2020-02-24 16:54caristuFile Added: image_2020_02_24T15_52_31_473Z.png
2020-02-24 17:23hgbotCheckin
2020-02-24 17:23hgbotNote Added: 0118058
2020-02-24 17:23hgbotStatusnew => resolved
2020-02-24 17:23hgbotResolutionopen => fixed
2020-02-24 17:23hgbotFixed in SCM revision => http://code.openbravo.com/erp/pmods/org.openbravo.service.external.integration/rev/07d0b6dbdfed468985296bbbbf8e2b04028862f1 [^]
2020-02-24 17:24hgbotCheckin
2020-02-24 17:24hgbotNote Added: 0118059
2020-02-24 17:24hgbotCheckin
2020-02-24 17:24hgbotNote Added: 0118060
2020-02-24 17:24hgbotCheckin
2020-02-24 17:24hgbotNote Added: 0118061
2020-02-24 17:27hgbotCheckin
2020-02-24 17:27hgbotNote Added: 0118062
2020-02-24 18:26caristuIssue cloned0043313
2020-02-24 18:27caristuRelationship addeddepends on 0043313
2020-02-25 08:19caristuIssue cloned0043314
2020-02-25 08:19caristuRelationship addedrelated to 0043314
2020-02-25 11:52hgbotCheckin
2020-02-25 11:52hgbotNote Added: 0118091
2020-02-25 12:32egoitzIssue Monitored: egoitz
2020-02-25 14:05hgbotCheckin
2020-02-25 14:05hgbotNote Added: 0118095
2020-02-27 09:00AugustoMauchNote Added: 0118144
2020-02-27 09:00AugustoMauchStatusresolved => closed

Notes
(0118058)
hgbot   
2020-02-24 17:23   
Repository: erp/pmods/org.openbravo.service.external.integration
Changeset: 07d0b6dbdfed468985296bbbbf8e2b04028862f1
Author: Carlos Aristu <carlos.aristu <at> openbravo.com>
Date: Mon Feb 24 17:15:45 2020 +0100
URL: http://code.openbravo.com/erp/pmods/org.openbravo.service.external.integration/rev/07d0b6dbdfed468985296bbbbf8e2b04028862f1 [^]

fixes BUG-43309: Memory leak in SBOImporterStore and SBOExporterStore

  The SBOImporterStore was an @ApplicationScoped bean that centralizes the generation of SBOImporters. To handle this creation it used an injection point to retrieve dynamically the SBOImporter instances and the selecting one based on the Integration API type (ManualBOB or RetailAPI) of an specific EntityMapping.

  The SBOImporters are @Dependent scoped, and this means that they are living as long as the bean instance they are injected in is living. As the SBOImporterStore is @ApplicationScoped the SBOImporters instances created with it were never being garbage collected which could eventually lead to OutOfMemory errors in some cases.

  To fix this problem now the SBOImporterStore is not using the injection point and it instantiates manually the proper SBOImporter according to its Integration API type.

  Note that this very same problem affected to the export flow and the same solution has been applied.

  See here[1] for a detailed explanation of this problem.

  Finally note that we have reduced the visibility of the class. The idea is to replace that manual implementation with a dynamic one but to implement it we need to update the module dependency with core so we can use the CDI.current() method available starting from the Weld upgrade[2].

  [1] https://blog.akquinet.de/2017/01/04/dont-get-trapped-into-a-memory-leak-using-cdi-instance-injection/ [^]
  [2] https://issues.openbravo.com/view.php?id=40193 [^]

---
M src/org/openbravo/service/external/integration/edl/BaseImportItemProcessor.java
M src/org/openbravo/service/external/integration/export/SynchronizableBusinessObjectExporter.java
M src/org/openbravo/service/external/integration/export/SynchronizableBusinessObjectExporterStore.java
M src/org/openbravo/service/external/integration/importer/DalSynchronizableBusinessObjectImporter.java
M src/org/openbravo/service/external/integration/importer/RetailApiSynchronizableBusinessObjectImporter.java
M src/org/openbravo/service/external/integration/importer/SynchronizableBusinessObjectImporter.java
M src/org/openbravo/service/external/integration/importer/SynchronizableBusinessObjectImporterStore.java
M src/org/openbravo/service/external/integration/mapping/EntityPropertyMapping.java
M src/org/openbravo/service/external/integration/mapping/JsonEntityPropertyMapping.java
---
(0118059)
hgbot   
2020-02-24 17:24   
Repository: erp/pmods/org.openbravo.service.external.integration.tests
Changeset: ea99a8f098d65b255c007672ad1f044b211b188c
Author: Carlos Aristu <carlos.aristu <at> openbravo.com>
Date: Mon Feb 24 17:20:39 2020 +0100
URL: http://code.openbravo.com/erp/pmods/org.openbravo.service.external.integration.tests/rev/ea99a8f098d65b255c007672ad1f044b211b188c [^]

related to BUG-43309: new way of getting instances of SBOImporters/SBOExporters

---
M src-test/org/openbravo/service/external/integration/test/_import/BaseImportTest.java
M src-test/org/openbravo/service/external/integration/test/export/CountryExportTest.java
M src-test/org/openbravo/service/external/integration/test/export/ExporterSetExternalIdentifierTest.java
---
(0118060)
hgbot   
2020-02-24 17:24   
Repository: erp/pmods/org.openbravo.service.integration.sapecc
Changeset: bd971c8f595c095e64ca439eda3c1b7ef9958b67
Author: Carlos Aristu <carlos.aristu <at> openbravo.com>
Date: Mon Feb 24 17:22:11 2020 +0100
URL: http://code.openbravo.com/erp/pmods/org.openbravo.service.integration.sapecc/rev/bd971c8f595c095e64ca439eda3c1b7ef9958b67 [^]

related to BUG-43309: new way of getting instances of SBOImporters/SBOExporters

---
M src-test/org/openbravo/service/integration/sapecc/edl/IdocImportTestHelper.java
---
(0118061)
hgbot   
2020-02-24 17:24   
Repository: erp/pmods/org.openbravo.service.integration.sap.ecc.mappings
Changeset: 6985f9aa6a83670af563af919c119a86bf0852cb
Author: Carlos Aristu <carlos.aristu <at> openbravo.com>
Date: Mon Feb 24 17:21:52 2020 +0100
URL: http://code.openbravo.com/erp/pmods/org.openbravo.service.integration.sap.ecc.mappings/rev/6985f9aa6a83670af563af919c119a86bf0852cb [^]

related to BUG-43309: new way of getting instances of SBOImporters/SBOExporters

---
M src-test/org/openbravo/service/integration/sap/ecc/mappings/inbound/InboundTest.java
---
(0118062)
hgbot   
2020-02-24 17:27   
Repository: erp/pmods/org.openbravo.service.external.integration
Changeset: 9e5f05a148ac909da66696665370c895423cadfe
Author: Carlos Aristu <carlos.aristu <at> openbravo.com>
Date: Mon Feb 24 17:27:39 2020 +0100
URL: http://code.openbravo.com/erp/pmods/org.openbravo.service.external.integration/rev/9e5f05a148ac909da66696665370c895423cadfe [^]

related to BUG-43309: remove no longer needed annotations

---
M src/org/openbravo/service/external/integration/export/DalSynchronizableBusinessObjectExporter.java
M src/org/openbravo/service/external/integration/export/RetailApiSynchronizableBusinessObjectExporter.java
---
(0118091)
hgbot   
2020-02-25 11:52   
Repository: erp/pmods/org.openbravo.service.integration.magento.core
Changeset: 17d9aac1944053581156a663cc708013e0d6a258
Author: Carlos Aristu <carlos.aristu <at> openbravo.com>
Date: Tue Feb 25 11:48:36 2020 +0100
URL: http://code.openbravo.com/erp/pmods/org.openbravo.service.integration.magento.core/rev/17d9aac1944053581156a663cc708013e0d6a258 [^]

related to BUG-43309: new way of getting instances of SBOImporters/SBOExporters

---
M src-test/org/openbravo/service/integration/magento/core/base/MagentoImportTest.java
---
(0118095)
hgbot   
2020-02-25 14:05   
Repository: erp/pmods/org.openbravo.warehouse.advancedwarehouseoperations.edl
Changeset: f1fd9a3ec2776c3a4e652ddf6e7f50a6f907bda9
Author: Carlos Aristu <carlos.aristu <at> openbravo.com>
Date: Tue Feb 25 14:05:03 2020 +0100
URL: http://code.openbravo.com/erp/pmods/org.openbravo.warehouse.advancedwarehouseoperations.edl/rev/f1fd9a3ec2776c3a4e652ddf6e7f50a6f907bda9 [^]

related to BUG-43309: new way of getting instances of SBOImporters/SBOExporters

---
M src-test/org/openbravo/warehouse/advancedwarehouseoperations/edl/test/AwoEdlImportTestHelper.java
---
(0118144)
AugustoMauch   
2020-02-27 09:00   
Code reviewed and verified