Async operations on magento : how to speed this up ?

Hello ! I’m currently using a script to add some products / information through the bulk api (asnync operations).

It’s working pretty well but it’s honestly slow . I’m using a magento 2.4.2 on Kubernetes and I have a cron pod that is always running “php bin/magento queue:consumers:start async.operations.all”, in order to consume the queues content as quick as possible.

The addition of tasks in the queues goes really fast but. currently, I can process (get the item in the queue and add this to Magento) something like 2 products in 10 seconds ! too slow…

I’m using a rabbitmq ( bitnami/rabbitmq:3.10.8-debian-11-r0) pod that manages the queues. The monitoring of this rabbitmq (and the cron one) instance doesn’t seem to indicate that the CPU / RAM is too weak.

Do you have any clue / idea that could help to improve the performances ? Have you ever faced something like this ?

Thanks for now 🙂

submitted by /u/No-Appointment-3446
[link] [comments]