Openbravo Issue Tracking System - Openbravo ERP
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0052614Openbravo ERPA. Platformpublic2023-05-30 18:292023-10-26 18:06
Reportercberner 
Assigned Tocberner 
PrioritynormalSeveritymajorReproducibilityhave not tried
StatusclosedResolutionfixed 
PlatformOS5OS Version
Product Version 
Target VersionFixed in VersionPR23Q4 
Merge Request Status
Review Assigned To
OBNetwork customer
Web browser
ModulesCore
Support ticket
Regression level
Regression date
Regression introduced in release
Regression introduced by commit
Triggers an Emergency PackNo
Summary0052614: There should be a mechanism to have non-blocking async EDL Requests
DescriptionCurrently all async EDL requests block at least a Import Entry thread and a DB connection, to better use the resources, the generated IE should unblock the thread until the request or processing finishes, and when finished, it should mark the corresponding DB records with the proper status.

A functional and technical design have been prepared for this project:
- Functional design: https://docs.google.com/document/d/1J4pViMYbAqsCJwURGtG3zfub2pttz62n63EVj-sWgcE/edit# [^]
- Technical design: https://docs.google.com/document/d/11IStWu--gxJlyVio3rKT7mfMgM01WSYmn6nkSfajSNY/edit# [^]
Steps To Reproduce-
Proposed Solution
Additional Information
TagsNo tags attached.
Relationships
related to defect 0053816 closed caristu Modules EDL process is duplicating request lines 
causes defect 0053639 closed cberner Openbravo ERP EDL in initial or processing state indefinitely when role has no access to Import Entry table 
causes defect 0053742 closed cberner Openbravo ERP NullPointerException thrown in EDL in unhandled exception cases 
causes defect 0053774 closed caristu Openbravo ERP RO pool is not closed by IE threads 
causes defect 0053784 closed cberner Openbravo ERP failed non-blocking EDL requests remain in Initial status 
causes backport 0053749PR23Q4 closed cberner Openbravo ERP EDL in initial or processing state indefinitely when role has no access to Import Entry table 
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2023-05-30 18:29cbernerNew Issue
2023-05-30 18:29cbernerAssigned To => cberner
2023-05-30 18:29cbernerModules => Core
2023-05-30 18:29cbernerTriggers an Emergency Pack => No
2023-06-14 17:06hgbotNote Added: 0151165
2023-06-20 14:28hgbotNote Added: 0151378
2023-08-04 15:11hgbotNote Added: 0153250
2023-08-21 15:30hgbotResolutionopen => fixed
2023-08-21 15:30hgbotStatusnew => closed
2023-08-21 15:30hgbotFixed in Version => PR23Q4
2023-08-21 15:30hgbotNote Added: 0153739
2023-08-21 15:30hgbotNote Added: 0153740
2023-08-21 15:30hgbotNote Added: 0153741
2023-08-21 15:30hgbotNote Added: 0153742
2023-08-21 15:30hgbotNote Added: 0153743
2023-08-21 15:30hgbotNote Added: 0153744
2023-08-21 16:01hgbotNote Added: 0153745
2023-08-21 16:28hgbotNote Added: 0153750
2023-08-21 17:06hgbotNote Added: 0153751
2023-08-22 13:58hgbotNote Added: 0153791
2023-08-22 13:58hgbotNote Added: 0153792
2023-10-24 16:14cbernerRelationship addedcauses 0053639
2023-10-24 16:14cbernerRelationship addedcauses 0053742
2023-10-26 17:17rafarodaRelationship addedrelated to 0053774
2023-10-26 18:05caristuRelationship deletedrelated to 0053774
2023-10-26 18:06caristuRelationship addedcauses 0053774
2023-10-27 10:11alostaleRelationship addedcauses 0053784
2023-10-27 10:26AugustoMauchRelationship addedcauses 0053749
2023-11-10 09:11caristuRelationship addedrelated to 0053816
2023-11-10 09:13caristuRelationship deletedrelated to 0053816
2023-11-10 09:14caristuRelationship addedrelated to 0053816

Notes
(0151165)
hgbot   
2023-06-14 17:06   
Merge Request created: https://gitlab.com/openbravo/product/openbravo/-/merge_requests/912 [^]
(0151378)
hgbot   
2023-06-20 14:28   
Merge Request created: https://gitlab.com/openbravo/product/pmods/org.openbravo.externaldata.integration/-/merge_requests/27 [^]
(0153250)
hgbot   
2023-08-04 15:11   
Merge Request created: https://gitlab.com/openbravo/product/pmods/org.openbravo.api/-/merge_requests/169 [^]
(0153739)
hgbot   
2023-08-21 15:30   
Directly closing issue as related merge request is already approved.

Repository: https://gitlab.com/openbravo/product/openbravo [^]
Changeset: 03fa452e48c60b06e2f1a70f29a093bbf07004d0
Author: Cristian Berner <cristian.berner@openbravo.com>
Date: 21-08-2023 15:27:39
URL: https://gitlab.com/openbravo/product/openbravo/-/commit/03fa452e48c60b06e2f1a70f29a093bbf07004d0 [^]

Fixes FR-52614: Support non-blocking Import Entries and EDLs

Adds support for a new type of import entry and infrastructure for
external systems to handle non-blocking processing. The non-blocking
processing is performed through CompletableFuture's API, which allows
executing methods after the CompletableFuture has been resolved.

---
A src/org/openbravo/service/NonBlockingExecutorServiceProvider.java
A src/org/openbravo/service/importprocess/NonBlockingImportEntryProcessRunnable.java
M src-db/database/model/tables/C_IMPORT_ENTRY.xml
M src-db/database/model/tables/C_IMPORT_ENTRY_ARCHIVE.xml
M src-db/database/sourcedata/AD_COLUMN.xml
M src-db/database/sourcedata/AD_ELEMENT.xml
M src-db/database/sourcedata/AD_FIELD.xml
M src/org/openbravo/service/externalsystem/http/HttpExternalSystem.java
M src/org/openbravo/service/importprocess/ImportEntryBuilder.java
M src/org/openbravo/service/importprocess/ImportEntryManager.java
M src/org/openbravo/service/importprocess/ImportEntryProcessor.java
---
(0153740)
hgbot   
2023-08-21 15:30   
Repository: https://gitlab.com/openbravo/product/pmods/org.openbravo.externaldata.integration [^]
Changeset: 3814f42dd715096570c6d519a63e77fb9ebb2738
Author: Cristian Berner <cristian.berner@openbravo.com>
Date: 21-08-2023 15:22:39
URL: https://gitlab.com/openbravo/product/pmods/org.openbravo.externaldata.integration/-/commit/3814f42dd715096570c6d519a63e77fb9ebb2738 [^]

Related to FR-52614: Support non-blocking EDLs

Adds support for a new type of EDL, non-blocking, which allows not
awaiting the response of some EDLs, like for example external http
requests. This allows processing asynchronously while also freeing
resources, both DB and CPU wise.

---
A src-test/org/openbravo/externaldata/integration/process/AsynchronousProcessorTest.java
A src/org/openbravo/externaldata/integration/process/NonBlockingAfterBatch.java
A src/org/openbravo/externaldata/integration/process/NonBlockingBeforeBatch.java
M src/org/openbravo/externaldata/integration/process/AsynchronousProcessor.java
M src/org/openbravo/externaldata/integration/process/EDLImportEntryProcessor.java
M src/org/openbravo/externaldata/integration/process/ItemProcessor.java
M src/org/openbravo/externaldata/integration/process/ProcessRequest.java
---
(0153741)
hgbot   
2023-08-21 15:30   
Merge request merged: https://gitlab.com/openbravo/product/pmods/org.openbravo.externaldata.integration/-/merge_requests/27 [^]
(0153742)
hgbot   
2023-08-21 15:30   
Repository: https://gitlab.com/openbravo/product/pmods/org.openbravo.api [^]
Changeset: a91a4d01a195b12ea951ab1ea80f4edc6396b150
Author: Cristian Berner <cristian.berner@openbravo.com>
Date: 21-08-2023 15:25:52
URL: https://gitlab.com/openbravo/product/pmods/org.openbravo.api/-/commit/a91a4d01a195b12ea951ab1ea80f4edc6396b150 [^]

Related to FR-52614: Switch Push API to the new non-blocking EDL API

Push API was modified to use the non-blocking EDL API, allowing it to
perform external http requests without awaiting the response of each of
them.

---
M src/org/openbravo/api/edl/ApiEventItemProcessor.java
---
(0153743)
hgbot   
2023-08-21 15:30   
Merge request merged: https://gitlab.com/openbravo/product/pmods/org.openbravo.api/-/merge_requests/169 [^]
(0153744)
hgbot   
2023-08-21 15:30   
Merge request merged: https://gitlab.com/openbravo/product/openbravo/-/merge_requests/912 [^]
(0153745)
hgbot   
2023-08-21 16:01   
Repository: https://gitlab.com/openbravo/product/pmods/org.openbravo.externaldata.integration [^]
Changeset: 1ae6da586ab4a7c25fb0ebadfc50cb316e43625c
Author: Cristian Berner <cristian.berner@openbravo.com>
Date: 21-08-2023 15:59:53
URL: https://gitlab.com/openbravo/product/pmods/org.openbravo.externaldata.integration/-/commit/1ae6da586ab4a7c25fb0ebadfc50cb316e43625c [^]

Related to FR-52614: Fix eclipse warning on ItemProcessor

---
M src/org/openbravo/externaldata/integration/process/ItemProcessor.java
---
(0153750)
hgbot   
2023-08-21 16:28   
Repository: https://gitlab.com/openbravo/product/pmods/org.openbravo.api [^]
Changeset: 36cc31575a39bcb1906b2998779565dcab309303
Author: Cristian Berner <cristian.berner@openbravo.com>
Date: 21-08-2023 16:28:09
URL: https://gitlab.com/openbravo/product/pmods/org.openbravo.api/-/commit/36cc31575a39bcb1906b2998779565dcab309303 [^]

Revert "Related to FR-52614: Switch Push API to the new non-blocking EDL API"

This reverts commit a91a4d01a195b12ea951ab1ea80f4edc6396b150, reversing
changes made to aa47d8388ec87b666f8e9cbbe74fa504980d8d08.

This is reverted due to the APIEventsItemProcessorTest not passing in
green, it is missing some mocks and changes related with the
non-blocking API modification.

---
M src/org/openbravo/api/edl/ApiEventItemProcessor.java
---
(0153751)
hgbot   
2023-08-21 17:06   
Merge Request created: https://gitlab.com/openbravo/product/pmods/org.openbravo.api/-/merge_requests/178 [^]
(0153791)
hgbot   
2023-08-22 13:58   
Merge request merged: https://gitlab.com/openbravo/product/pmods/org.openbravo.api/-/merge_requests/178 [^]
(0153792)
hgbot   
2023-08-22 13:58   
Repository: https://gitlab.com/openbravo/product/pmods/org.openbravo.api [^]
Changeset: 0606f993d0671e3a2597df60ba1a5b9115f1e436
Author: Cristian Berner <cristian.berner@openbravo.com>
Date: 22-08-2023 11:58:52
URL: https://gitlab.com/openbravo/product/pmods/org.openbravo.api/-/commit/0606f993d0671e3a2597df60ba1a5b9115f1e436 [^]

Related to FR-52614: Switch Push API to the new non-blocking EDL API

This also fixes the ApiEventItemProcessorTest to properly mock
non-blocking EDL infrastructure and usage.

---
M src-test/org/openbravo/api/edl/ApiEventItemProcessorTest.java
M src/org/openbravo/api/edl/ApiEventItemProcessor.java
---