0048418: import entries can be processed twice if their key is not deterministic

Description

If an ImportEntryProcessor can return different keys for the same import entry, ie. it returns a random number between 1 and 10 to distribute entries in 10 threads, and an entry is takes long to be processed since it was initially scheduled, it can be processed more than once.

Steps To Reproduce

Create an import entry which key is always different and takes long to execute and schedule it ensuring there is no other entries processing at the same time (to ensure no more time is awaited between cycle executions).
-> Check it gets processed 3 times.

This behavior can be emulated with Import Entry Load Simulator [1] and the following config:

** Import Entry config
  import.batch.size                     5000
  import.number.of.threads              11
  import.max.task.queue.size            1000
  import.wait.time                      600
  import.processing.capacity.per.second 330
** OBUIELS_Data1
  Num of Entries 1
  Period         100
  Cycles         1
  Delay          0
  Exec Time      5000
  # of Keys      1000

---

[1] https://gitlab.com/openbravo/tools/platform/org.openbravo.util.ieloadsim [^]

Tags

No tags attached.

Attached Files

Relationships [ Relation Graph ] [ Dependency Graph ]

related to	defect	0048427		closed	alostale	Openbravo ERP	high load of low priority import entries can cause higher priority entries not to be processed
related to	defect	0047219		closed	caristu	Modules	Use different runnables for EDL processes that should be executed in a specific thread

Notes
(0134374) hgbot (developer) 2022-01-19 08:24	Merge Request created: https://gitlab.com/openbravo/product/openbravo/-/merge_requests/503 [^]

Issue History
Date Modified	Username	Field	Change
2022-01-18 14:22	alostale	New Issue
2022-01-18 14:22	alostale	Assigned To	=> Triage Platform Conn
2022-01-18 14:22	alostale	Modules	=> Core
2022-01-18 14:22	alostale	Triggers an Emergency Pack	=> No
2022-01-19 08:24	hgbot	Note Added: 0134374
2022-01-19 13:06	alostale	Relationship added	related to 0048427
2022-04-04 14:17	rafaroda	Relationship added	related to 0047219