web stats
Deadlock Errors - Mirth Community

Go Back   Mirth Community > Mirth Connect > Support

Reply
 
Thread Tools Display Modes
  #1  
Old 06-13-2012, 03:25 PM
GregD GregD is offline
Mirth Guru
 
Join Date: May 2010
Posts: 187
GregD is on a distinguished road
Default Deadlock Errors

I received 4 of these deadlock errors in the past week. None before that. Is there a reason this has suddenly started happening. I this case, I looked in the database in table MESSAGE and there is a message with ID ab33e66c-57c8-4a5f-9e8c-c21da0d800a8 which has a DATE_CREATED value of 4 seconds before this error is logged.

Should I be concerned?

Greg
Quote:
[2012-06-13 12:59:53,694] ERROR (com.mirth.connect.server.controllers.DefaultMessa geObjectController:248): could not log message: id=ab33e66c-57c8-4a5f-9e8c-c21da0d800a8
com.ibatis.common.jdbc.exception.NestedSQLExceptio n: --- The error occurred in sqlserver/sqlserver-message.xml. --- The error occurred while applying a parameter map. --- Check the Message.update-message-param. --- Check the statement (update failed). --- Cause: java.sql.SQLException: Transaction (Process ID 112) was deadlocked on lock | communication buffer resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
at com.ibatis.sqlmap.engine.mapping.statement.MappedS tatement.executeUpdate(MappedStatement.java:107)
at com.ibatis.sqlmap.engine.impl.SqlMapExecutorDelega te.update(SqlMapExecutorDelegate.java:457)
at com.ibatis.sqlmap.engine.impl.SqlMapSessionImpl.up date(SqlMapSessionImpl.java:90)
at com.ibatis.sqlmap.engine.impl.SqlMapClientImpl.upd ate(SqlMapClientImpl.java:66)
at com.mirth.connect.server.controllers.DefaultMessag eObjectController.writeMessageToDatabase(DefaultMe ssageObjectController.java:241)
at com.mirth.connect.server.controllers.DefaultMessag eObjectController.updateMessage(DefaultMessageObje ctController.java:185)
at com.mirth.connect.server.controllers.DefaultMessag eObjectController.setStatus(DefaultMessageObjectCo ntroller.java:894)
at com.mirth.connect.server.controllers.DefaultMessag eObjectController.setSuccess(DefaultMessageObjectC ontroller.java:818)
at com.mirth.connect.connectors.mllp.MllpMessageDispa tcher.manageResponseAck(MllpMessageDispatcher.java :358)
at com.mirth.connect.connectors.mllp.MllpMessageDispa tcher.sendPayload(MllpMessageDispatcher.java:293)
at org.mule.providers.QueueEnabledConnector$QueueWork er.run(QueueEnabledConnector.java:218)
at java.lang.Thread.run(Unknown Source)Caused by: java.sql.SQLException: Transaction (Process ID 112) was deadlocked on lock | communication buffer resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
at net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnos tic(SQLDiagnostic.java:365)
at net.sourceforge.jtds.jdbc.TdsCore.tdsErrorToken(Td sCore.java:2781)
at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCor e.java:2224)
at net.sourceforge.jtds.jdbc.TdsCore.getMoreResults(T dsCore.java:628)
at net.sourceforge.jtds.jdbc.JtdsStatement.processRes ults(JtdsStatement.java:525)
at net.sourceforge.jtds.jdbc.JtdsStatement.executeSQL (JtdsStatement.java:487)
at net.sourceforge.jtds.jdbc.JtdsPreparedStatement.ex ecute(JtdsPreparedStatement.java:475)
at org.apache.commons.dbcp.DelegatingPreparedStatemen t.execute(DelegatingPreparedStatement.java:172)
at org.apache.commons.dbcp.DelegatingPreparedStatemen t.execute(DelegatingPreparedStatement.java:172)
at com.ibatis.sqlmap.engine.execution.SqlExecutor.exe cuteUpdate(SqlExecutor.java:80)
at com.ibatis.sqlmap.engine.mapping.statement.MappedS tatement.sqlExecuteUpdate(MappedStatement.java:216 )
at com.ibatis.sqlmap.engine.mapping.statement.MappedS tatement.executeUpdate(MappedStatement.java:94)
... 11 more
Reply With Quote
  #2  
Old 07-25-2012, 10:47 AM
GregD GregD is offline
Mirth Guru
 
Join Date: May 2010
Posts: 187
GregD is on a distinguished road
Default

I am getting 2 to 5 of these a day. I understand what a dead-lock error is, but I'm not sure why I am getting them. The message says, "could not log message: id=ab33e66c-57c8-4a5f-9e8c-c21da0d800a8". When I run the query below, the message is always in the database

SELECT * FROM MESSAGE WHERE (ID = 'ab33e66c-57c8-4a5f-9e8c-c21da0d800a8')

So what exactly is this telling me. If it says it can not log message id, is it referring to the ID column in table MESSAGE or is it not being logged someplace else. The DATE_CREATED column in table MESSAGE always corresponds to the date and time of the error.

Greg
Reply With Quote
  #3  
Old 07-25-2012, 11:06 AM
dereke's Avatar
dereke dereke is offline
Mirth Employee
 
Join Date: Mar 2012
Posts: 18
dereke is on a distinguished road
Send a message via AIM to dereke
Default

Which version of Mirth Connect are you on? I know more recent versions have rowlock/nolock statements in the sqlserver-message.xml file to help with this issue.

The SQL thread count configuration might need to be increased to avoid the deadlocks. I don't know which version of SQL Server you're on, but I think SQL Server 2008 tries to auto-allocate the thread count. There are times when this needs to be manually adjusted.
Reply With Quote
  #4  
Old 07-25-2012, 01:25 PM
GregD GregD is offline
Mirth Guru
 
Join Date: May 2010
Posts: 187
GregD is on a distinguished road
Default

Mirth - 2.1.1.5490
SQL Server - 2005

The errors are not associated with any one channel or destination. I get two different kinds of errors, both related to "com.ibatis.common.jdbc.exception.NestedSQLExcepti on". One is "Could not log message" and the other is "Error updating message".
Reply With Quote
  #5  
Old 07-26-2012, 05:09 AM
dereke's Avatar
dereke dereke is offline
Mirth Employee
 
Join Date: Mar 2012
Posts: 18
dereke is on a distinguished road
Send a message via AIM to dereke
Default

Do you get these at any specific time of day, or randomly? When is your pruning set to occur?
Reply With Quote
  #6  
Old 07-26-2012, 08:26 AM
GregD GregD is offline
Mirth Guru
 
Join Date: May 2010
Posts: 187
GregD is on a distinguished road
Default

As I said, normally there are 3 to 5 a day so I don't notice the time. The last 2 days have been particularly bad. I received close 40 to 50 of these errors between 8:30AM and 4:30PM one day and 9:30AM to 5:30PM the next. Then another at 11:30 PM on Monday. None today, so far.

We don't prune any channels. We made the decision to hold on to all messages. Our volume is relatively low. We receive about 14,000 messages a day, with about half of those coming from a single ADT feed. Another 1000 of those were ACKs routed from a few channels to a channel.

Greg
Reply With Quote
  #7  
Old 08-23-2012, 01:26 PM
GregD GregD is offline
Mirth Guru
 
Join Date: May 2010
Posts: 187
GregD is on a distinguished road
Default

I think I figured out what was causing this, I'll let the people of Mirth determine if this is a bug, or just a poorly designed set of channels by me.

I had one channel with a single source connector which was a channel reader and a single destination to which was MLLP over a VPN. I

had three other channels with source connectors of varying types receiving data of varying types. All three of these channels formatted incoming data to a standard ADT message and then routed them to the channel reader source connector of the first channel. This seems to be what was causing all of the SQL Server errors.

I changed so that the 3 channels receiving data wrote to a single folder instead of to the channel reader on the first channel. I changed the first channel from a channel reader to a file reader and pointed it to the folder which the other 3 channels wrote to. The SQL Server errors stopped immediately.

Greg
Reply With Quote
  #8  
Old 08-23-2012, 08:51 PM
dereke's Avatar
dereke dereke is offline
Mirth Employee
 
Join Date: Mar 2012
Posts: 18
dereke is on a distinguished road
Send a message via AIM to dereke
Default

Hey Greg,

Thanks for the update on this and letting us know your solution!
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 12:52 PM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2014, vBulletin Solutions, Inc.
Mirth Corporation