Opening a channel inside a consumer interface callback times out waiting for continuation

[sergeyshaykhullin]

Problem

Creating a new channel in an asynchronous consumer creates new channel on server, but throws timeout exception on client

Details

I tried to make a pipeline from multiple queues and ran into a problem that a basic consumer is working, but an asynchronous one is constantly falling with a timeout. It turned out to understand that the problem is in creating a new channel using same connection inside async consumer callback, it hangs when creating. But if you create a channel before the start consuming and pass inside consumer using closure, it works

How to reproduce

Send some message to queue1, when in queue1 async consumer try to create new channel (or try lazy initialization/lazy cache - need connection.CreateModel() call) and when send using created channel to somewhere else

Example: https://github.com/phema-team/Phema.RabbitMQ/tree/master/examples/Phema.RabbitMQ.RawClient

Client version 6.0.0-pre3, 5.1.0
Server version 3.7.17
Exception stack trace

System.TimeoutException: The operation has timed out.
   at RabbitMQ.Util.BlockingCell`1.WaitForValue(TimeSpan timeout)
   at RabbitMQ.Client.Impl.SimpleBlockingRpcContinuation.GetReply(TimeSpan timeout)
   at RabbitMQ.Client.Impl.ModelBase.ModelRpc(MethodBase method, ContentHeaderBase header, Byte[] body)
   at RabbitMQ.Client.Framing.Impl.Model._Private_ChannelOpen(String outOfBand)
   at RabbitMQ.Client.Framing.Impl.AutorecoveringConnection.CreateNonRecoveringModel()
   at RabbitMQ.Client.Framing.Impl.AutorecoveringConnection.CreateModel()
   at Phema.RabbitMQ.RawClient.Program.<>c__DisplayClass0_0.<<Main>b__0>d.MoveNext() in C:\Users\Sergey\GitHub\Phema\Phema.RabbitMQ\examples\Phema.RabbitMQ.RawClie
nt\Program.cs:line 34

[michaelklishin]

"Async consumer" is not very specific. Do you mean a method such as Consumer.HandleDelivery? If so it's a design limitation that Java client addressed years ago.

[sergeyshaykhullin]

@michaelklishin Callback raise in HandleBasicDeliver, but I double check with async and sync consumers

• Async will create new channel, but throw timeout exception https://gist.github.com/sergeyshaykhullin/e70f8d30c3ddd3ce364c35f5d80ace55
• Sync will not and process message as expected https://gist.github.com/sergeyshaykhullin/0eba5a3e02093b3a7a862449d043e52e

Both use HandleBasicDeliver

[michaelklishin]

This is a side effect of the design choice around consumer dispatch operations, which is different in this client compared to Java or Ruby. I cannot yet say if it is solvable without giving up all the gains of the new design but it's hardly a novel scenario. You have two possible workarounds:

• Avoid opening channels in consumer operation callbacks
• Do it in a delayed manner (task or similar), so that the continuation mechanism is not deadlocked by the channel.open method execution

[kjnilsson]

Maybe we need Async versions of all RPC interactions?

[fizch]

That would be much appreciated!!!

…

On Fri, Sep 6, 2019 at 3:25 AM Karl Nilsson ***@***.***> wrote: Maybe we need Async versions of all RPC interactions? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#650?email_source=notifications&email_token=ABBXNAPU27LMCLEBQJZFKQ3QIIHXHA5CNFSM4ISTLY32YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6CEOAI#issuecomment-528762625>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABBXNANE2XKTNYKP7KJZQWDQIIHXHANCNFSM4ISTLY3Q> .

[stebet]

Maybe we need Async versions of all RPC interactions?

That's something I'd be very much open and willing to submit a PR for. I already have a PR ready that moves all async/bakcground work to the ThreadPool, so this shouldn't be all that hard to implement. Is there an issue logged for this request?

[lukebakken]

Is there an issue logged for this request?

Not at the moment, and I wouldn't worry about an issue for it. If the work could be a follow-on to #687 that would be ideal (to make review easier).

[lukebakken]

Moving this to version 7.0.0 because -

• A workaround exists - Opening a channel inside a consumer interface callback times out waiting for continuation #650 (comment)
• A real fix is a large change to this library ("async versions of all RPC interactions") Opening a channel inside a consumer interface callback times out waiting for continuation #650 (comment) and Opening a channel inside a consumer interface callback times out waiting for continuation #650 (comment)