web stats
QUEUED messages from the destination_Connector are not processing out fast - Mirth Community

Go Back   Mirth Community > Mirth Connect > Support

Reply
 
Thread Tools Display Modes
  #1  
Old 08-04-2018, 08:02 AM
sadarsh sadarsh is offline
What's HL7?
 
Join Date: Jun 2017
Posts: 3
sadarsh is on a distinguished road
Exclamation QUEUED messages from the destination_Connector are not processing out fast

Hi All,

We have a situation where QUEUED messages from the destination_Connector are not processing out fast on one of our channels. We disabled all the transformations on this channel and connector making it a pass through channel so that all the QUEUED messages are processed out. Initially we had this channel feeding Mirth Results Appliance. Each message processing out of this particular destination connector is sent at one message per minute. We have a queue of 70,000 message to drain by Monday. Any help would be greatly appreciated.

Back Ground: This channel sends out results to Mirth Results through the "Mirth Results Sender" in built connector type plugin. We had a JavaScript that was QUERYING MR database for provider enrichment. We had a failover 2 days back in the cluster and neither of our primary nor secondary was stable. We had to de cluster the nodes and and make our secondary node as primary so that it starts delivering the live data, we have the actual primary node which has QUEUED messages on it isolated so that it can drain the QUEUE fast. But this doesn't seem to be working.


We keep seeing the below error in the logs, not sure if it has any impact on the issue.


[2018-08-04 10:59:22,915] ERROR (com.mirth.connect.donkey.server.channel.Channel:1 814): An error occurred in channel destinations_MirthResults (09da16df-a68a-45f8-a452-05bf2a93959c) while processing message ID 4045603 from the source queue
com.mirth.connect.donkey.server.data.DonkeyDaoExce ption: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "d_mm7_pkey" Detail: Key (message_id, id)=(4045603, 1) already exists.
at com.mirth.connect.donkey.server.data.jdbc.JdbcDao. insertConnectorMessage(JdbcDao.java:686)
at com.mirth.connect.donkey.server.data.buffered.Buff eredDao.executeTasks(BufferedDao.java:109)
at com.mirth.connect.donkey.server.data.buffered.Buff eredDao.commit(BufferedDao.java:85)
at com.mirth.connect.donkey.server.data.buffered.Buff eredDao.commit(BufferedDao.java:67)
at com.mirth.connect.donkey.server.channel.Channel.pr ocess(Channel.java:1680)
at com.mirth.connect.donkey.server.channel.Channel.pr ocessSourceQueue(Channel.java:1811)
at com.mirth.connect.donkey.server.channel.Channel.ru n(Channel.java:1797)
at java.lang.Thread.run(Thread.java:745)Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "d_mm7_pkey" Detail: Key (message_id, id)=(4045603, 1) already exists.
at org.postgresql.core.v3.QueryExecutorImpl.receiveEr rorResponse(QueryExecutorImpl.java:2455)
at org.postgresql.core.v3.QueryExecutorImpl.processRe sults(QueryExecutorImpl.java:2155)
at org.postgresql.core.v3.QueryExecutorImpl.execute(Q ueryExecutorImpl.java:288)
at org.postgresql.jdbc.PgStatement.executeInternal(Pg Statement.java:430)
at org.postgresql.jdbc.PgStatement.execute(PgStatemen t.java:356)
at org.postgresql.jdbc.PgPreparedStatement.executeWit hFlags(PgPreparedStatement.java:168)
at org.postgresql.jdbc.PgPreparedStatement.executeUpd ate(PgPreparedStatement.java:135)
at com.mirth.connect.donkey.server.data.jdbc.JdbcDao. insertConnectorMessage(JdbcDao.java:673)
... 7 more
Thanks,
Sam


MC Version: 3.5
Reply With Quote
  #2  
Old 08-07-2018, 08:26 AM
narupley's Avatar
narupley narupley is online now
Mirth Employee
 
Join Date: Oct 2010
Posts: 7,054
narupley is on a distinguished road
Default

Have you removed/deleted any messages? Usually that error is expected when you've manually removed anything: MIRTH-3541

In the message browser, enable the Send Date and Response Date columns. Look at the elapsed time for your destination. How does it compare to the rate of inbound received messages? My guess is that the external system isn't responding quickly enough to keep up with your channel.

You can enable multiple queue threads to increase throughput on the destination queue side. However it's possible that the downstream external system might not be able to keep up with that volume. Order is also not guaranteed in that case, though you can still preserve message order among certain messages (like those with the same patient ID) by using the Thread Assignment Variable.
__________________
Step 1: JAVA CACHE...DID YOU CLEAR ...wait, ding dong the witch is dead?

Nicholas Rupley
Work: 949-237-6069
Always include what Mirth Connect version you're working with. Also include (if applicable) the code you're using and full stacktraces for errors (use CODE tags). Posting your entire channel is helpful as well; make sure to scrub any PHI/passwords first.


- How do I foo?
- You just bar.
Reply With Quote
  #3  
Old 08-26-2018, 10:54 AM
sadarsh sadarsh is offline
What's HL7?
 
Join Date: Jun 2017
Posts: 3
sadarsh is on a distinguished road
Default

We have the QUEUE threads count set to 10 on the destination connector and we have Thread Assignment Variable on "Patient DOB", however there was a stuck message from a particular source that is causing the queue to build up. We had to manually delete the stuck message to keep the messages flowing, can we enable the "Rotate Queue" option as we now have the Thread Assignment Variable in place to guarantee the order of the messages that go outbound ?
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -8. The time now is 04:46 AM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2018, vBulletin Solutions, Inc.
Mirth Corporation