Anonymous | Login
Project:
RSS
  
News | My View | View Issues | Roadmap | Summary

View Issue DetailsJump to Notes ] Issue History ] Print ]
ID
0040747
TypeCategorySeverityReproducibilityDate SubmittedLast Update
defect[Openbravo ERP] A. Platformcriticalsometimes2019-05-02 13:432019-11-12 15:45
ReportermarvintmView Statuspublic 
Assigned Toalostale 
PriorityhighResolutionfixedFixed in Version3.0PR19Q3
StatusclosedFix in branchFixed in SCM revision01589a6ee69a
ProjectionnoneETAnoneTarget Version
OSAnyDatabaseAnyJava version
OS VersionDatabase versionAnt version
Product VersionSCM revision 
Review Assigned Tocaristu
Web browser
ModulesCore
Regression level
Regression date
Regression introduced in release
Regression introduced by commit
Triggers an Emergency PackNo
Summary

0040747: Unnecessary wait in Import Entry Manager is causing lag in the processing of import entries even if resources are available

DescriptionCurrently the main Import Entry Manager thread is moved to sleep for an amount of time that depends on how many import entries were processed, every time the pending import entries are checked.

This is causing the mechanism to not optimally use the resources. The reason seems to be that the wait time is too big, and this means that the executor threads all finish their current tasks, and then need for this time to be finished for the main Import Entry Manager thread to look for more import entries to assign to them.

The consequence for the user is that when there is a very high amount of activity in the system, Import Entry Manager initially picks a big amount of import entries to be processed. These import entries are then processed as fast as the system can, taking into account the configuration and hardware, but even if this finishes very fast, the whole system needs to wait for this arbitrary time to end before looking for more work, and in the meantime if the activity of the system is high, more import entries arrive, and the end result is that import entry queue can grow a lot depending on the request the system receives, while the system itself is spending most of the time idle.

Steps To ReproduceGenerate a big amount of import entry requests, and verify that the import entries get accumulated, while the system spends a significant amount of time idle without processing them.
Proposed SolutionThis wait time needs to be reconsidered, but at the same time the change that we do here needs to be properly validated to ensure that it doesn't affect the stability of the system.
TagsNo tags attached.
Attached Files

- Relationships Relation Graph ] Dependency Graph ]
related to feature request 0037853 closedalostale ImportEntryManager.reportsStatus only does long time avg, useless to see any short term changes 
related to defect 0041978 closedalostale Import Entry data types without sequence should be sumbitted in last place 

-  Notes
(0111518)
hgbot (developer)
2019-05-07 08:02

Repository: erp/devel/pi
Changeset: 01589a6ee69a5546a2c3c30068ef773b2c872101
Author: Asier Lostalé <asier.lostale <at> openbravo.com>
Date: Fri May 03 07:48:00 2019 +0200
URL: http://code.openbravo.com/erp/devel/pi/rev/01589a6ee69a5546a2c3c30068ef773b2c872101 [^]

fixes 40747: unnecessary wait in Import Entry Manager

  After handling import entries to be processed, ImportEntryManager thread sleeps
  in order to give time those entries to be processed before starting a new loop.

  The time was calculated based on the amount of import entries handled in last
  cycle (giving 300ms to each one) with a minimum of 2 secs. In cases with a high
  activity this time can be very high resulting in no new entries being processed
  because the import entry manager thread is sleeping.

  When doing this calculation, the number of threads processing import entries was
  not taken into account.

  Now this time is divided by this number of threads. Still this is just an inaccurate
  approximation. If this time is very small, the entries took in previous loop will
  be taken again in the next one(s) as they couldn't be processed yet so their status
  didn't change in database, causing unnecessary roundtrips to database. In any case,
  the consecuence of this would be limited to non required work being done as
  ImportEntryProcessor.ImportEntryProcessRunnable.addEntry method checks whether new
  import entries retrieved from DB are already being processed before adding them to
  the queue preventing in this way double processing.

---
M src/org/openbravo/service/importprocess/ImportEntryManager.java
---
(0111522)
hgbot (developer)
2019-05-07 08:37

Repository: erp/devel/pi
Changeset: 03190a39ef36a167a2e42e6af188d9c57bf49022
Author: Asier Lostalé <asier.lostale <at> openbravo.com>
Date: Tue May 07 08:37:36 2019 +0200
URL: http://code.openbravo.com/erp/devel/pi/rev/03190a39ef36a167a2e42e6af188d9c57bf49022 [^]

related to issue 40747: added comment to variable

---
M src/org/openbravo/service/importprocess/ImportEntryManager.java
---
(0111762)
hgbot (developer)
2019-05-17 09:51

Repository: erp/devel/pi
Changeset: 59e2bd0cc7fab99bd1362ecc17eea6978e77132c
Author: Asier Lostalé <asier.lostale <at> openbravo.com>
Date: Fri May 17 09:49:39 2019 +0200
URL: http://code.openbravo.com/erp/devel/pi/rev/59e2bd0cc7fab99bd1362ecc17eea6978e77132c [^]

related to issue 40747: improved comment

---
M src/org/openbravo/service/importprocess/ImportEntryManager.java
---
(0111763)
caristu (developer)
2019-05-17 09:52

Reviewed + tested
(0112011)
hudsonbot (developer)
2019-05-21 15:17

A changeset related to this issue has been promoted main and to the
Central Repository, after passing a series of tests.

Promotion changeset: https://code.openbravo.com/erp/devel/main/rev/9b8f37d9d85e [^]
Maturity status: Test
(0112043)
hudsonbot (developer)
2019-05-21 15:17

A changeset related to this issue has been promoted main and to the
Central Repository, after passing a series of tests.

Promotion changeset: https://code.openbravo.com/erp/devel/main/rev/9b8f37d9d85e [^]
Maturity status: Test

- Issue History
Date Modified Username Field Change
2019-05-02 13:43 marvintm New Issue
2019-05-02 13:43 marvintm Assigned To => platform
2019-05-02 13:43 marvintm Modules => Core
2019-05-02 13:43 marvintm Triggers an Emergency Pack => No
2019-05-03 13:44 alostale Relationship added related to 0037853
2019-05-03 13:44 alostale Status new => scheduled
2019-05-03 13:44 alostale Assigned To platform => alostale
2019-05-03 13:44 alostale Review Assigned To => caristu
2019-05-07 08:02 hgbot Checkin
2019-05-07 08:02 hgbot Note Added: 0111518
2019-05-07 08:02 hgbot Status scheduled => resolved
2019-05-07 08:02 hgbot Resolution open => fixed
2019-05-07 08:02 hgbot Fixed in SCM revision => http://code.openbravo.com/erp/devel/pi/rev/01589a6ee69a5546a2c3c30068ef773b2c872101 [^]
2019-05-07 08:37 hgbot Checkin
2019-05-07 08:37 hgbot Note Added: 0111522
2019-05-17 09:51 hgbot Checkin
2019-05-17 09:51 hgbot Note Added: 0111762
2019-05-17 09:52 caristu Note Added: 0111763
2019-05-17 09:52 caristu Status resolved => closed
2019-05-17 09:52 caristu Fixed in Version => 3.0PR19Q3
2019-05-21 15:17 hudsonbot Checkin
2019-05-21 15:17 hudsonbot Note Added: 0112011
2019-05-21 15:17 hudsonbot Checkin
2019-05-21 15:17 hudsonbot Note Added: 0112043
2019-10-10 10:43 alostale Relationship added related to 0041978
2019-11-12 15:45 alostale Severity major => critical


Copyright © 2000 - 2009 MantisBT Group
Powered by Mantis Bugtracker