Openbravo Issue Tracking System - Openbravo ERP
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0024430Openbravo ERPA. Platformpublic2013-07-29 18:352014-03-18 09:24
Reporteregoitz 
Assigned Toeduardo_Argal 
PriorityurgentSeveritymajorReproducibilityalways
StatusclosedResolutionfixed 
PlatformOS5OS Version
Product Version 
Target VersionFixed in Version 
Merge Request Status
Review Assigned ToSandrahuguet
OBNetwork customerOBPS
Web browser
ModulesCore
Support ticket
Regression level
Regression date
Regression introduced in release
Regression introduced by commit
Triggers an Emergency PackNo
Summary0024430: The ReportGeneralLedger report take long to be printed with big amount of data and consumes a lot of resources
DescriptionThe ReportGeneralLedger report take long to be printed with big amount of data, and while the report is being executed the server can not be used because a lot of resources are being consumed by the report.


When taking from fact_acct 500000 recoreds it takes 40 seconds to execute the select but 6 minutes to execute fillREport method on Httpsecureappservlert and 1,5 minutes to execute exportreporttopdf.

Steps To Reproduce-Execute a report on an environment with a arround 5.000.000 rows on the fact_acct for a interval where 500.000 rows are get.
Proposed Solutionhttp://stackoverflow.com/questions/15875199/jasperreports-fillreport-too-slow-and-resource-consuming [^]

http://community.jaspersoft.com/questions/536842/jasperreports-too-slow [^]

http://stackoverflow.com/questions/472522/jasperreport-taking-too-much-resources [^]
Additional Information
TagsNo tags attached.
Relationships
related to defect 0030694 closed AtulOpenbravo The ReportInvoiceDiscountJR report consume many resources when printing big amount of data. 
Attached Files? 1MProducts.jrxml (2,545) 2013-10-06 16:09
https://issues.openbravo.com/file_download.php?file_id=6439&type=bug
diff issue24430.diff (1,284) 2013-10-06 16:09
https://issues.openbravo.com/file_download.php?file_id=6440&type=bug
Issue History
Date ModifiedUsernameFieldChange
2013-07-29 18:35egoitzNew Issue
2013-07-29 18:35egoitzAssigned To => AugustoMauch
2013-07-29 18:35egoitzModules => Core
2013-07-29 18:35egoitzOBNetwork customer => Yes
2013-07-29 18:35egoitzTriggers an Emergency Pack => No
2013-07-31 14:01alostaleTarget Version => 3.0MP27
2013-08-30 14:22AugustoMauchTarget Version3.0MP27 => 3.0MP28
2013-09-23 08:27alostaleTarget Version3.0MP28 => 3.0MP29
2013-10-06 16:09iperdomoFile Added: 1MProducts.jrxml
2013-10-06 16:09iperdomoFile Added: issue24430.diff
2013-10-06 16:11iperdomoNote Added: 0061556
2013-10-06 16:20iperdomoNote Added: 0061557
2013-10-06 16:33iperdomoNote Added: 0061558
2013-10-07 13:10iperdomoNote Added: 0061575
2013-10-16 11:28egoitzDescription Updatedbug_revision_view_page.php?rev_id=5112#r5112
2013-10-28 10:36alostaleAssigned ToAugustoMauch => eduardo_Argal
2013-10-28 16:51egoitzTarget Version3.0MP29 => 3.0MP30
2013-11-26 23:46jonalegriaesarteNote Added: 0062390
2013-11-26 23:46jonalegriaesarteTypedefect => design defect
2013-11-26 23:46jonalegriaesarteTarget Version3.0MP30 =>
2014-03-18 09:24SandrahuguetReview Assigned To => Sandrahuguet
2014-03-18 09:24SandrahuguetNote Added: 0065249
2014-03-18 09:24SandrahuguetStatusnew => closed
2014-03-18 09:24SandrahuguetResolutionopen => fixed
2015-08-28 12:53egoitzRelationship addedrelated to 0030694

Notes
(0061556)
iperdomo   
2013-10-06 16:11   
Some of the links in the proposed solutions are related to XML parsing when using XML documents as datasource. This is not the case for the ReportGeneralLedger
(0061557)
iperdomo   
2013-10-06 16:20   
I have created a simple Product list to check if the problem is when building the data for the report, or the report generation itself.

The proposed way for handling large reports is using "report virtualizers"
http://community.jaspersoft.com/wiki/support-large-reports [^]

With 1M products in the database and different JVM options I measured some parts of the reports generation.


## JVM OPTS: -Xms256M -Xmx4096M -XX:MaxPermSize=512M

(no virtualizer)
fillReport: 258370 ms
exportReportToPdfFile: 52614 ms
Memory: 2.4 GB allocated / 1.9 GB used
Total: 312284 ms (5.20 min)

Using virtualizer (maxSize = 2)
fillReport: 255363 ms
exportReportToPdfFile: 60546 ms
Memory: 2.0 GB allocated / 1.6 GB used
Total: 317217 ms (5.28 min)

Using virtualizer (maxSize = 4)
fillReport: 263196 ms
exportReportToPdfFile: 62446 ms
Memory: 2.0 GB allocated / 1.5 GB used
Total: 326966 ms (5.44 min)

Using virtualizer (maxSize = 6)
fillReport: 264471 ms
exportReportToPdfFile: 58343 ms
Memory: 2.2 GB allocated / 1.5 GB Used
Total: 324134 ms (5.40 min)

## JVM OPTS = -Xms256M -Xmx2048M -XX:MaxPermSize=256M

Using virtualizer (maxSize = 6)
fillReport: 259962 ms
exportReportToPdfFile: 58842 ms
Memory: 1.3 GB allocated / 892 MB used
Total: 320106 ms (5.40 min)


## JVM OPTS = -Xms128M -Xmx1024M -XX:MaxPermSize=256M

Using virtualizer (maxSize = 6)
fillReport: 268572 ms
exportReportToPdfFile: 62131 ms
Memory: 1018 MB allocated / 808 MB used
Total: 331975 ms (5.53 min)


Conclusion? It seems that using JRFileVirtualizer helps the process to use less memory without impacting too much in the overall generation time
(0061558)
iperdomo   
2013-10-06 16:33   
I would suggest that the large memory usage is also related to the way the report is built. By loading all the records in memory, and then calculating totals group.

As you can see just printing 1M products, takes ~6min.

As the issue description states, the problem to tackle is the 40min report generation.
(0061575)
iperdomo   
2013-10-07 13:10   
The process of generating data can be split per account and/or group, generating a pdf document. After the parts can be concatenated.

The class PrintController has code showing how to contatenate different PDFs documents.
(0062390)
jonalegriaesarte   
2013-11-26 23:46   
Moved to design defect, because we need to analyze in depth the behavior of the report
(0065249)
Sandrahuguet   
2014-03-18 09:24   
This issue has been fixed with the project xsql-scrollabe.
In these commits:
https://code.openbravo.com/erp/devel/pi/rev/3df74c8a30f3 [^]
https://code.openbravo.com/erp/devel/pi/rev/d537fb38c3c5 [^]
https://code.openbravo.com/erp/devel/pi/rev/89155988f108 [^]