Anonymous | Login
Project:
RSS
  
News | My View | View Issues | Roadmap | Summary

View Issue DetailsJump to Notes ] Issue History ] Print ]
ID
0037324
TypeCategorySeverityReproducibilityDate SubmittedLast Update
defect[Openbravo ERP] A. Platformmajorhave not tried2017-11-20 11:592018-01-03 12:39
ReporteralostaleView Statuspublic 
Assigned Toalostale 
PriorityurgentResolutionfixedFixed in Version3.0PR18Q1
StatusclosedFix in branchFixed in SCM revisionb817e5607d8b
ProjectionnoneETAnoneTarget Version
OSAnyDatabaseAnyJava version
OS VersionDatabase versionAnt version
Product VersionSCM revision 
Review Assigned Tocaristu
Web browser
ModulesCore
Regression level
Regression date
Regression introduced in release
Regression introduced by commit
Triggers an Emergency PackNo
Summary

0037324: slow login having many preferences

DescriptionIn a system with some thousands of preferences, depending on how they are distributed, login (both in back-office and POS) can last up to some seconds.

Two different cases detected with different configurations:

== Case 1: many visible preferences ==
Thousands of preferences visible from the client/org/role that is performing the login. Testing this case with 8K preferences defined at system level, login requests took up to 40s.

This case, even possible, it's a bit hypothetical, not that probable to occur in a real instance.

== Case 2: preferences defined in readable org, but not visible due to "visible at org" field ==
Having thousands of preferences defined on an organization that is in the parent tree of the one that is used for current login, login time takes up to extra 2s even those preferences are set as not visible through the "visible at org" field.

This case can occur for example in a real instance with many (in the steps emulating 500) organizations all of them children of a common parent. Preferences are defined in parent one, and visibility limited of each of the children.
Steps To ReproduceFollowing cases are reproducible in an instance with Openbravo for Retail installed.

1. case 1
1.1. Create 8K preferences at System level, visible from all roles:
  insert into ad_preference
    (ad_preference_id,  ad_client_id,      ad_org_id,
     createdby,         updatedby,         attribute,
     value,              ispropertylist)
    select get_uuid(),  '0',     '0',
           '0',         '0',     'visible-all-'||s, 
           'v:'||s,     'N'
      from generate_series(1,8000) s;

1.2. Login WebPOS
    -> It takes up to 40s to complete, note most of this time is spent at Preferneces.getPreferenceFromList (see Selection_230.png)

2. case 2
2.1. for each preference defined in WebPOS module, insert up to 500 instances, all of them with ad_org_id being in a common parent org (ie in The White Valley Group), but limiting visibility though visibleat_org_id to some no org not visible in login (ie. Central Mountains Store)

insert into ad_preference
            (ad_preference_id,  ad_client_id,      ad_org_id,         
             createdby,          updatedby,         property,         
             value,              ispropertylist,
             visibleat_org_id)
       slect get_uuid(), 
            '39363B0921BB4293B48383844325E84C', -- Client: The White Valley Group
            '67839EEFA49E44AC969BD60093FCC899', -- Org: The White Valley Group
            '0', '0',  value,
            'v:'||s, 'Y',
            '3B187EC130A549A7A9388F8060EF156D' -- visible at org: Central Mountains Store
       from ad_ref_list, generate_series(1,500) s
      where ad_module_id ='FF808181326CC34901326D53DBCF0018' 
        and ad_reference_id ='A26BA480E2014707B47257024C3CBFF7' -- all POS preferences     

2.2 login WebPOS with 'Demo user' on its default 'Vallblanca' organization, which cannot see 'Central Mountains Store'
  -> login request take up to 4s, in this case, time is spent in loading DAL objects in memory
Proposed SolutionAfter fixing both cases, time drops to less than 200ms for 1st case and less than 100ms for 2nd one.
TagsPerformance
Attached Filespng file icon Selection_230.png [^] (38,360 bytes) 2017-11-20 12:35


png file icon Selection_231.png [^] (73,305 bytes) 2017-11-20 12:50

- Relationships Relation Graph ] Dependency Graph ]
related to feature request 0035553 closedcaristu Improve scalability of the login process 
depends on defect 0037112 closedalostale incorrect HQL to retrieve preferences 
related to feature request 0037329 closedalostale make a more generic Utility.arrayListToString 

-  Notes
(0100637)
hgbot (developer)
2017-11-21 09:58

Repository: erp/devel/pi
Changeset: f422e1ebac8e738f9855d871d63d3727d4bd349d
Author: Asier Lostalé <asier.lostale <at> openbravo.com>
Date: Mon Nov 20 13:11:31 2017 +0100
URL: http://code.openbravo.com/erp/devel/pi/rev/f422e1ebac8e738f9855d871d63d3727d4bd349d [^]

fixed bug 37324: slow login having many preferences - case 1

  Retrieving all preferences for current role/org was slow having thousands of
  visible preferences because of how duplicate preferences with different visibility
  were checked. This check was performed by sequentially scanning a list of already
  saved preferences to verify whether current one was already in the list.

  It has now fixed by keeping a temporary map of preferences where the key defines
  if preference is the same or not, in this way it is not necessary to iterate
  over all the entries each time.

---
M src/org/openbravo/erpCommon/businessUtility/Preferences.java
---
(0100638)
hgbot (developer)
2017-11-21 09:58

Repository: erp/devel/pi
Changeset: b817e5607d8b45760927890f6f3e3558028c1c1c
Author: Asier Lostalé <asier.lostale <at> openbravo.com>
Date: Mon Nov 20 13:24:45 2017 +0100
URL: http://code.openbravo.com/erp/devel/pi/rev/b817e5607d8b45760927890f6f3e3558028c1c1c [^]

fixed bug 37324: slow login having many preferences - case 2

  When there were thousands of preferences with ad_org visible from current org
  but not visible through visible at org field, loading all preferences was slow.

  The problem was, all those preferences were loaded as DAL objects to filter
  out later in memory those that were not visible.

  This gets fixed by filtering out directly in the query so that no unneeded rows
  are retrived from DB to instantiate DAL objects that later will be discarded
  immediatelly.

---
M src/org/openbravo/erpCommon/businessUtility/Preferences.java
---
(0100641)
alostale (manager)
2017-11-21 10:07

Related changesets not detected automatically:

changeset: 33029:27c3e6dd6f54
user: Asier Lostalé <asier.lostale@openbravo.com>
date: Tue Nov 21 09:50:10 2017 +0100
summary: related to 37324: renamed PreferenceTest methods

changeset: 33030:f930ff85c698
tag: tip
user: Asier Lostalé <asier.lostale@openbravo.com>
date: Tue Nov 21 09:53:27 2017 +0100
summary: related to 37324: added test case for getAllPreferences
(0100808)
caristu (developer)
2017-11-29 11:06

Code reviewed + tested OK.
(0101444)
hudsonbot (developer)
2018-01-03 12:39

A changeset related to this issue has been promoted main and to the
Central Repository, after passing a series of tests.

Promotion changeset: https://code.openbravo.com/erp/devel/main/rev/c81e0d3cbab5 [^]
Maturity status: Test
(0101445)
hudsonbot (developer)
2018-01-03 12:39

A changeset related to this issue has been promoted main and to the
Central Repository, after passing a series of tests.

Promotion changeset: https://code.openbravo.com/erp/devel/main/rev/c81e0d3cbab5 [^]
Maturity status: Test

- Issue History
Date Modified Username Field Change
2017-11-20 11:59 alostale New Issue
2017-11-20 11:59 alostale Assigned To => platform
2017-11-20 11:59 alostale Modules => Core
2017-11-20 11:59 alostale Triggers an Emergency Pack => No
2017-11-20 12:00 alostale Steps to Reproduce Updated View Revisions
2017-11-20 12:08 alostale Steps to Reproduce Updated View Revisions
2017-11-20 12:10 alostale Steps to Reproduce Updated View Revisions
2017-11-20 12:10 alostale Tag Attached: Performance
2017-11-20 12:10 alostale Assigned To platform => alostale
2017-11-20 12:12 alostale Relationship added depends on 0037112
2017-11-20 12:13 alostale Relationship added related to 0035553
2017-11-20 12:35 alostale File Added: Selection_230.png
2017-11-20 12:37 alostale Steps to Reproduce Updated View Revisions
2017-11-20 12:50 alostale File Added: Selection_231.png
2017-11-20 12:51 alostale Steps to Reproduce Updated View Revisions
2017-11-20 13:00 alostale Review Assigned To => caristu
2017-11-20 13:00 alostale Priority normal => urgent
2017-11-20 13:00 alostale Description Updated View Revisions
2017-11-20 14:42 alostale Relationship added related to 0037329
2017-11-21 09:58 hgbot Checkin
2017-11-21 09:58 hgbot Note Added: 0100637
2017-11-21 09:58 hgbot Status new => resolved
2017-11-21 09:58 hgbot Resolution open => fixed
2017-11-21 09:58 hgbot Fixed in SCM revision => http://code.openbravo.com/erp/devel/pi/rev/f422e1ebac8e738f9855d871d63d3727d4bd349d [^]
2017-11-21 09:58 hgbot Checkin
2017-11-21 09:58 hgbot Note Added: 0100638
2017-11-21 09:58 hgbot Fixed in SCM revision http://code.openbravo.com/erp/devel/pi/rev/f422e1ebac8e738f9855d871d63d3727d4bd349d [^] => http://code.openbravo.com/erp/devel/pi/rev/b817e5607d8b45760927890f6f3e3558028c1c1c [^]
2017-11-21 10:07 alostale Note Added: 0100641
2017-11-21 10:28 alostale Proposed Solution updated
2017-11-29 11:06 caristu Note Added: 0100808
2017-11-29 11:06 caristu Status resolved => closed
2017-11-29 11:06 caristu Fixed in Version => 3.0PR18Q1
2018-01-03 12:39 hudsonbot Checkin
2018-01-03 12:39 hudsonbot Note Added: 0101444
2018-01-03 12:39 hudsonbot Checkin
2018-01-03 12:39 hudsonbot Note Added: 0101445


Copyright © 2000 - 2009 MantisBT Group
Powered by Mantis Bugtracker