Openbravo Issue Tracking System - Openbravo ERP |
| View Issue Details |
|
| ID | Project | Category | View Status | Date Submitted | Last Update |
| 0020658 | Openbravo ERP | A. Platform | public | 2012-05-31 18:40 | 2012-06-21 17:54 |
|
| Reporter | alostale | |
| Assigned To | alostale | |
| Priority | normal | Severity | major | Reproducibility | have not tried |
| Status | closed | Resolution | fixed | |
| Platform | | OS | 5 | OS Version | |
| Product Version | | |
| Target Version | 3.0MP13 | Fixed in Version | 3.0MP13 | |
| Merge Request Status | |
| Review Assigned To | |
| OBNetwork customer | No |
| Web browser | |
| Modules | Core |
| Support ticket | |
| Regression level | |
| Regression date | |
| Regression introduced in release | |
| Regression introduced by commit | |
| Triggers an Emergency Pack | No |
|
| Summary | 0020658: Heap space problem when exporting big csv files |
| Description | Trying to export a grid to csv can stall the whole application if number of records being exported is big enough.
The problem is memory is not correctly cleaned up, so each loaded record keeps part of it in memory till the process finishes, reaching a point where 100% CPU is used trying to collect the garbage though it is not possible.
|
| Steps To Reproduce | It hugely varies depending on the table being exported. Testing with 1.5GB of max heap space:
-200K Business Partners are exported in 1min (11MB) no memory issues
-130K Sales Invoices cannot be exported, after several minutes it exports ~45K (4.6MB) and 100%CPU |
| Proposed Solution | |
| Additional Information | |
| Tags | Performance |
| Relationships | |
| Attached Files |  issue-20658.export (2,171) 2012-06-01 08:55
https://issues.openbravo.com/file_download.php?file_id=5271&type=bug |
|
| Issue History |
| Date Modified | Username | Field | Change |
| 2012-05-31 18:40 | alostale | New Issue | |
| 2012-05-31 18:40 | alostale | Assigned To | => alostale |
| 2012-05-31 18:40 | alostale | Modules | => Core |
| 2012-05-31 18:40 | alostale | OBNetwork customer | => No |
| 2012-05-31 19:09 | alostale | Tag Attached: Performance | |
| 2012-05-31 19:24 | alostale | Steps to Reproduce Updated | bug_revision_view_page.php?rev_id=3515#r3515 |
| 2012-06-01 08:51 | alostale | Steps to Reproduce Updated | bug_revision_view_page.php?rev_id=3518#r3518 |
| 2012-06-01 08:55 | alostale | File Added: issue-20658.export | |
| 2012-06-01 08:55 | alostale | Note Added: 0049571 | |
| 2012-06-01 08:58 | alostale | Steps to Reproduce Updated | bug_revision_view_page.php?rev_id=3519#r3519 |
| 2012-06-01 09:04 | alostale | Note Added: 0049572 | |
| 2012-06-01 09:08 | alostale | Note Edited: 0049572 | bug_revision_view_page.php?bugnote_id=0049572#r3521 |
| 2012-06-04 11:27 | hgbot | Checkin | |
| 2012-06-04 11:27 | hgbot | Note Added: 0049600 | |
| 2012-06-04 11:27 | hgbot | Status | new => resolved |
| 2012-06-04 11:27 | hgbot | Resolution | open => fixed |
| 2012-06-04 11:27 | hgbot | Fixed in SCM revision | => http://code.openbravo.com/erp/devel/pi/rev/8466c9b3c506ffa81b0569106d700df5dcd68c62 [^] |
| 2012-06-21 17:54 | AugustoMauch | Note Added: 0049997 | |
| 2012-06-21 17:54 | AugustoMauch | Status | resolved => closed |
| 2012-06-21 17:54 | AugustoMauch | Fixed in Version | => 3.0MP13 |
|
Notes |
|
|
|
|
Attached patch since right now pi is closed to release 3.0MP12, fix will be included within 3.0MP13 |
|
|
|
(0049572)
|
|
alostale
|
2012-06-01 09:04
(edited on: 2012-06-01 09:08) |
|
Executed test plan:
*Before applying the patch, export 100K BPs. Takes ~1min.
*After applying the patch, export 100K BPs. Takes ~1min.
*Compare both files: they are identical.
*Export 130K invoices. It takes ~10min. to export 13MB. While exporting one CPU is consuming 100%, but memory is properly cleaned so no much garbage collection consumption and in case of more CPUs available, the application is still usable. Before applying the patch, this case after 20min and 45K invoices exported made the application unusable with 100%CPU in garbage collection because of high memory consumption.
|
|
|
|
(0049600)
|
|
hgbot
|
|
2012-06-04 11:27
|
|
Repository: erp/devel/pi
Changeset: 8466c9b3c506ffa81b0569106d700df5dcd68c62
Author: Asier Lostalé <asier.lostale <at> openbravo.com>
Date: Fri Jun 01 08:53:46 2012 +0200
URL: http://code.openbravo.com/erp/devel/pi/rev/8466c9b3c506ffa81b0569106d700df5dcd68c62 [^]
fixed bug 20658: Heap space problem when exporting big csv files
In fetch for csv exportation, changed evict with clear. Evict removes current
record from memory, but it doesn't remove its references, this causes unnecesary
memory consumption.
---
M modules/org.openbravo.service.json/src/org/openbravo/service/json/DefaultJsonDataService.java
---
|
|
|
|
|
|
Code reviewed and verified in pi@f06b2e72932e |
|