0047606: Missing timeout on Azure Blob Storage operations

Anonymous | Login

Project:

News | My View | View Issues | Roadmap | Summary

View Revisions: Issue #47606		[ All Revisions ] [ Back to Issue ]
Summary	0047606: Missing timeout on Azure Blob Storage operations

Revision	2021-09-01 16:15 by AugustoMauch
Description	In our use of the Azure JDK we are not defining timeouts on any of the upload/download/delete file operations. If there is any problem when working with that API and the server does not respond, the thread executing that code will get stack, and its database transaction will remain open. an import entry was blocked by this request for at least 4 hours : Relevant stack trace: Import Entry - 5" 0000884 daemon prio=5 os_prio=0 cpu=1105246.52ms elapsed=810940.05s tid=0x00007f27d404b000 nid=0x3720 waiting on condition [0x00007f271498b000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@11.0.11/Native Method) parking to wait for <0x000000009a1fab18> (a java.util.concurrent.CountDownLatch$Sync) at java.util.concurrent.locks.locksupport.park(java.base@11.0.11/locksupport.java:194) at java.util.concurrent.locks.abstractqueuedsynchronizer.parkandcheckinterrupt(java.base@11.0.11/abstractqueuedsynchronizer.java:885) at java.util.concurrent.locks.abstractqueuedsynchronizer.doacquiresharedinterruptibly(java.base@11.0.11/abstractqueuedsynchronizer.java:1039) at java.util.concurrent.locks.abstractqueuedsynchronizer.acquiresharedinterruptibly(java.base@11.0.11/abstractqueuedsynchronizer.java:1345) at java.util.concurrent.countdownlatch.await(java.base@11.0.11/countdownlatch.java:232) at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:81) at reactor.core.publisher.Mono.block(Mono.java:1494) at com.azure.storage.common.implementation.StorageImplUtils.blockWithOptionalTimeout(StorageImplUtils.java:94) at com.azure.storage.blob.BlobClient.uploadFromFile(BlobClient.java:182) at com.azure.storage.blob.BlobClient.uploadFromFile(BlobClient.java:153) at com.azure.storage.blob.BlobClient.uploadFromFile(BlobClient.java:129) at org.openbravo.service.integration.azure.storage.AzureStorageClient.putObject(AzureStorageClient.java:122) at org.openbravo.service.integration.azure.storage.AzureStorageClient.upload(AzureStorageClient.java:195) at org.openbravo.service.integration.azure.storage.attachments.AzureStorageAttachmentClient.upload(AzureStorageAttachmentClient.java:111) at org.openbravo.service.integration.azure.storage.attachments.AzureStorageAttachmentClient.upload(AzureStorageAttachmentClient.java:82) at org.openbravo.service.integration.azure.storage.attachments.AzureStorageAttachImplementation.uploadFile(AzureStorageAttachImplementation.java:45) at org.openbravo.service.integration.azure.storage.attachments.AzureStorageAttachImplementation$Proxy$_$$_WeldClientProxy.uploadFile(Unknown Source) at org.openbravo.client.application.attachment.AttachImplementationManager.upload(AttachImplementationManager.java:183) at com.openbravo.norauto.order.integration.processes.ProcessPendingItem.attachFileToOrder(ProcessPendingItem.java:118) at com.openbravo.norauto.order.integration.processes.ProcessPendingItem.processPendingItem(ProcessPendingItem.java:73) at com.openbravo.norauto.order.integration.integration.NOROrdSynchronization.saveRecord(NOROrdSynchronization.java:41) at org.openbravo.mobile.core.process.DataSynchronizationProcess.saveRecord(DataSynchronizationProcess.java:201) at org.openbravo.mobile.core.process.DataSynchronizationProcess.exec(DataSynchronizationProcess.java:156) at org.openbravo.mobile.core.process.DataSynchronizationProcess.exec(DataSynchronizationProcess.java:88) at org.openbravo.mobile.core.process.MobileImportEntryProcessorRunnable.processEntry(MobileImportEntryProcessorRunnable.java:54) at org.openbravo.service.importprocess.ImportEntryProcessor$ImportEntryProcessRunnable.doRunCycle(ImportEntryProcessor.java:373) at org.openbravo.service.importprocess.ImportEntryProcessor$ImportEntryProcessRunnable.run(ImportEntryProcessor.java:294) at java.util.concurrent.executors$runnableadapter.call(java.base@11.0.11/executors.java:515) at java.util.concurrent.futuretask.run(java.base@11.0.11/futuretask.java:264) at java.util.concurrent.threadpoolexecutor.runworker(java.base@11.0.11/threadpoolexecutor.java:1128) at java.util.concurrent.threadpoolexecutor$worker.run(java.base@11.0.11/threadpoolexecutor.java:628) at java.lang.thread.run(java.base@11.0.11/thread.java:829)

Revision	2021-08-30 11:31 by AugustoMauch
Description	an import entry was blocked by this request for at least 4 hours : here is the call associated stack : select attachment0_.C_Attachment_Method_ID as C_Attach1_165_0, attachment0_.AD_Client_ID as AD_Clien2_165_0, attachment0_.AD_Org_ID as AD_Org_I3_165_0, attachment0_.Isactive as Isactive4_165_0, attachment0_.Created as Created5_165_0, attachment0_.Createdby as Createdb6_165_0, attachment0_.Updated as Updated7_165_0, attachment0_.Updatedby as Updatedb8_165_0, attachment0_.AD_Module_ID as AD_Modul9_165_0, attachment0_.Name as Name10_165_0, attachment0_.Value as Value11_165_0 from C_Attachment_Method attachment0 where attachment0_.C_Attachment_Method_ID=$1 Import Entry - 5" 0000884 daemon prio=5 os_prio=0 cpu=1105246.52ms elapsed=810940.05s tid=0x00007f27d404b000 nid=0x3720 waiting on condition [0x00007f271498b000] java.lang.Thread.State: WAITING (parking) at jdk.internal.misc.Unsafe.park(java.base@11.0.11/Native Method) parking to wait for <0x000000009a1fab18> (a java.util.concurrent.CountDownLatch$Sync) at java.util.concurrent.locks.locksupport.park(java.base@11.0.11/locksupport.java:194) at java.util.concurrent.locks.abstractqueuedsynchronizer.parkandcheckinterrupt(java.base@11.0.11/abstractqueuedsynchronizer.java:885) at java.util.concurrent.locks.abstractqueuedsynchronizer.doacquiresharedinterruptibly(java.base@11.0.11/abstractqueuedsynchronizer.java:1039) at java.util.concurrent.locks.abstractqueuedsynchronizer.acquiresharedinterruptibly(java.base@11.0.11/abstractqueuedsynchronizer.java:1345) at java.util.concurrent.countdownlatch.await(java.base@11.0.11/countdownlatch.java:232) at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:81) at reactor.core.publisher.Mono.block(Mono.java:1494) at com.azure.storage.common.implementation.StorageImplUtils.blockWithOptionalTimeout(StorageImplUtils.java:94) at com.azure.storage.blob.BlobClient.uploadFromFile(BlobClient.java:182) at com.azure.storage.blob.BlobClient.uploadFromFile(BlobClient.java:153) at com.azure.storage.blob.BlobClient.uploadFromFile(BlobClient.java:129) at org.openbravo.service.integration.azure.storage.AzureStorageClient.putObject(AzureStorageClient.java:122) at org.openbravo.service.integration.azure.storage.AzureStorageClient.upload(AzureStorageClient.java:195) at org.openbravo.service.integration.azure.storage.attachments.AzureStorageAttachmentClient.upload(AzureStorageAttachmentClient.java:111) at org.openbravo.service.integration.azure.storage.attachments.AzureStorageAttachmentClient.upload(AzureStorageAttachmentClient.java:82) at org.openbravo.service.integration.azure.storage.attachments.AzureStorageAttachImplementation.uploadFile(AzureStorageAttachImplementation.java:45) at org.openbravo.service.integration.azure.storage.attachments.AzureStorageAttachImplementation$Proxy$_$$_WeldClientProxy.uploadFile(Unknown Source) at org.openbravo.client.application.attachment.AttachImplementationManager.upload(AttachImplementationManager.java:183) at com.openbravo.norauto.order.integration.processes.ProcessPendingItem.attachFileToOrder(ProcessPendingItem.java:118) at com.openbravo.norauto.order.integration.processes.ProcessPendingItem.processPendingItem(ProcessPendingItem.java:73) at com.openbravo.norauto.order.integration.integration.NOROrdSynchronization.saveRecord(NOROrdSynchronization.java:41) at org.openbravo.mobile.core.process.DataSynchronizationProcess.saveRecord(DataSynchronizationProcess.java:201) at org.openbravo.mobile.core.process.DataSynchronizationProcess.exec(DataSynchronizationProcess.java:156) at org.openbravo.mobile.core.process.DataSynchronizationProcess.exec(DataSynchronizationProcess.java:88) at org.openbravo.mobile.core.process.MobileImportEntryProcessorRunnable.processEntry(MobileImportEntryProcessorRunnable.java:54) at org.openbravo.service.importprocess.ImportEntryProcessor$ImportEntryProcessRunnable.doRunCycle(ImportEntryProcessor.java:373) at org.openbravo.service.importprocess.ImportEntryProcessor$ImportEntryProcessRunnable.run(ImportEntryProcessor.java:294) at java.util.concurrent.executors$runnableadapter.call(java.base@11.0.11/executors.java:515) at java.util.concurrent.futuretask.run(java.base@11.0.11/futuretask.java:264) at java.util.concurrent.threadpoolexecutor.runworker(java.base@11.0.11/threadpoolexecutor.java:1128) at java.util.concurrent.threadpoolexecutor$worker.run(java.base@11.0.11/threadpoolexecutor.java:628) at java.lang.thread.run(java.base@11.0.11/thread.java:829)