Anonymous | Login
Project:
RSS
  
News | My View | View Issues | Roadmap | Summary

View Issue Details[ Jump to Notes ] [ Issue History ] [ Print ]
ID
0024506
TypeCategorySeverityReproducibilityDate SubmittedLast Update
defect[Openbravo ERP] A. Platformmajorsometimes2013-08-07 14:302013-08-16 20:31
ReportercaristuView Statuspublic 
Assigned Toshuehner 
PriorityurgentResolutionfixedFixed in Version
StatusclosedFix in branchFixed in SCM revisionfcf4d70d6019
ProjectionnoneETAnoneTarget Version
OSAnyDatabaseAnyJava version
OS VersionDatabase versionAnt version
Product VersionSCM revision 
Merge Request Status
Review Assigned Tocaristu
OBNetwork customerNo
Web browser
ModulesCore
Support ticket
Regression level
Regression date
Regression introduced in release
Regression introduced by commit
Triggers an Emergency PackNo
Summary

0024506: Performance problem when importing bank statements

DescriptionThe import bank statement process is working very slow with some files. There is a performance problem with this process because a strange gc pattern appears in the cpu usage.

The flush operations in the FIN_BankStatementImport java class are where most part of the time is spent.
Steps To Reproduce1- Configure tomcat with the following parameters: -Xms384M -Xmx1024M -XX:MaxPermSize=256M
2- Go to the [Financial Account] window
3- Choose an account and click on "Import Bank Statement"
4- Use the csv importer algorithm and load a bank statement with about 250 lines.
Proposed SolutionThe root of the problem is that the process is loading too many business partners into the memory from the database.

Most of those business partners are not modified during the process.

So there is no need of keeping those unmodified business partners in the DAL session. They should be evicted directly to reduce size of the session and thus greatly reducing time spend on flush iterating over entities in the session.
TagsPerformance
Attached Files

- Relationships [ Relation Graph ] [ Dependency Graph ]
has duplicate defect 0024246 closedcaristu Modules Error OutOfMemory when you try to import a CSV with 800 lines 

-  Notes
(0060533)
hgbot (developer)
2013-08-07 15:44

 Repository: erp/devel/pi
Changeset: fcf4d70d60194e7afdca63dcbf4725ffb3201f07
Author: Stefan Hühner <stefan.huehner <at> openbravo.com>
Date: Wed Aug 07 15:43:49 2013 +0200
URL: http://code.openbravo.com/erp/devel/pi/rev/fcf4d70d60194e7afdca63dcbf4725ffb3201f07 [^]

Fixed 24506: Improve performance/reduce gc time of bankstatement import.
The code searching for matching business partners in the import of
bankstatementlines is loading a very high number of bp entities into
memory over the course of an import (example 10k bp's loaded for importing
250 lines).
That causes sub-sequent calls to flush() to be very slow (>=30 seconds) as those
need to iterate over all the objects in the hibernate session.

As all those bp's are not modified during the import they can be directly removed
(evicted) from the hibernate session as there's no need to keep them there and
check if they have been modified at flush() time.

---
M modules/org.openbravo.advpaymentmngt/src/org/openbravo/advpaymentmngt/utility/FIN_BankStatementImport.java
---
(0060543)
caristu (viewer)
2013-08-08 11:54

 Code Review + Testing OK
(0060632)
hudsonbot (viewer)
2013-08-16 20:31

 A changeset related to this issue has been promoted main and to the
Central Repository, after passing a series of tests.

Promotion changeset: https://code.openbravo.com/erp/devel/main/rev/59a1180e7f4f [^]

Maturity status: Test

- Issue History
Date Modified Username Field Change
2013-08-07 14:30 caristu New Issue
2013-08-07 14:30 caristu Assigned To => shuehner
2013-08-07 14:30 caristu Modules => Core
2013-08-07 14:30 caristu OBNetwork customer => No
2013-08-07 14:30 caristu Triggers an Emergency Pack => No
2013-08-07 14:30 caristu Issue Monitored: networkb
2013-08-07 14:31 caristu Tag Attached: Performance
2013-08-07 14:35 shuehner Proposed Solution updated
2013-08-07 15:44 hgbot Checkin
2013-08-07 15:44 hgbot Note Added: 0060533
2013-08-07 15:44 hgbot Status new => resolved
2013-08-07 15:44 hgbot Resolution open => fixed
2013-08-07 15:44 hgbot Fixed in SCM revision => http://code.openbravo.com/erp/devel/pi/rev/fcf4d70d60194e7afdca63dcbf4725ffb3201f07 [^]
2013-08-08 11:54 caristu Review Assigned To => caristu
2013-08-08 11:54 caristu Note Added: 0060543
2013-08-08 11:54 caristu Status resolved => closed
2013-08-16 20:31 hudsonbot Checkin
2013-08-16 20:31 hudsonbot Note Added: 0060632
2013-09-10 19:19 caristu Relationship added has duplicate 0024246

Copyright © 2000 - 2009 MantisBT Group
Powered by Mantis Bugtracker