web stats
What happens when Mirth Connect loses the connection with its internal database? - Mirth Community

Go Back   Mirth Community > Mirth Connect > Support

Reply
 
Thread Tools Display Modes
  #1  
Old 05-15-2017, 12:34 AM
Jurjan Jurjan is offline
OBX.1 Kenobi
 
Join Date: Feb 2011
Posts: 37
Jurjan is on a distinguished road
Default What happens when Mirth Connect loses the connection with its internal database?

Dear people,
about one and a half month ago one of our clients finally listened to us and replaced their embedded derby database with a MS SQL (2016 / 11.0.5058.0) database.
That database is located on a different server in their serverpark, although we had indicated that perhaps a local version would be better for performance.
(side question: is this true?).

Since they started using this database they have recurring problems with Mirth Connect (3.2.1.750) 'hanging' i.e. doing nothing.
There are no errors in the dashboard, just channels doing nothing.
not reading, not writing, just nothing.
The Mirth machine itself is not doing much (processor load is low).

In the logging (mirth.log) we see this:
ERROR 2017-05-14 20:14:38,983 [Timer-46] com.mirth.connect.donkey.server.channel.Destinatio nChain: Error processing destination Verplaats file als verwerking is gelukt for channel 0f99951c-3386-4749-92bd-622ce35cbe13.
com.mirth.connect.donkey.server.data.DonkeyDaoExce ption: java.sql.SQLException: Invalid state, the Connection object is closed.
at com.mirth.connect.donkey.server.data.jdbc.JdbcDao. close(JdbcDao.java:1983)
at com.mirth.connect.donkey.server.data.buffered.Buff eredDao.executeTasks(BufferedDao.java:140)


more details in the attached log excerpt (Mirth_log_excerpt.txt)

And yes, because there are no alerts etc. to warn the admins it took them 90 minutes to react (and restart the mirth machine)!

The strange situation is that restarting the Mirth machine solves the problem (for the time being).

The destination first mentioned in the logging (Verplaats file als verwerking is gelukt) doesn't even use our user database, one of the reasons we have a feeling it's the internal Mirth database that's acting up.


Is there ANY way to find out in logging or something is that's the case?
In other words: how can we find out what the problem is here?

Any help / hints are greatly appreciated.
Thanks
Jurjan
Attached Files
File Type: txt Mirth_log_excerpt.txt (17.4 KB, 5 views)
Reply With Quote
  #2  
Old 05-15-2017, 08:21 AM
brentm brentm is offline
Mirth Employee
 
Join Date: Jan 2012
Posts: 85
brentm is on a distinguished road
Default

Locating the database on the same server as Mirth Connect would certainly help performance while processing messages.

You can enable more log output around database access by adding this to the end of the log4j.properties file under the /conf folder in the Mirth Connect installation:

Code:
log4j.logger.com.mirth.connect.donkey.server.data.jdbc = DEBUG
This will produce a lot of output in your logs, so be aware of that.

Also, updating to the latest version of Mirth Connect (3.5.0) could help. There was a performance issue that was addressed in version 3.4.0 that could be related to what you observed: http://www.mirthcorp.com/community/i...wse/MIRTH-3447
Reply With Quote
  #3  
Old 08-16-2017, 01:57 AM
Jurjan Jurjan is offline
OBX.1 Kenobi
 
Join Date: Feb 2011
Posts: 37
Jurjan is on a distinguished road
Default

OOOOPS,
I'm sorry, I just noticed your answer.

(seems the mail notifying me of your reply never got to me).

I will forward your suggestions to my customer.

Thanks
Reply With Quote
  #4  
Old 09-29-2017, 04:39 AM
Jurjan Jurjan is offline
OBX.1 Kenobi
 
Join Date: Feb 2011
Posts: 37
Jurjan is on a distinguished road
Default

We have had a new occurrence of this problem at our clients location.

The logs are full of this:
DEBUG 2017-09-26 23:19:17,227 [qtp173832146-32 - /channelstatus] org.eclipse.jetty.io.nio.ssl: [Session-1, SSL_NULL_WITH_NULL_NULL] SslConnection@2bc4c9c2 SSL NOT_HANDSHAKING i/o/u=0/0/0 ishut=false oshut=false {AsyncHttpConnection@4dc92409,g=HttpGenerator{s=2, h=0,b=0,c=-1},p=HttpParser{s=0,l=12,c=44},r=108138} NOT_HANDSHAKING filled=0/0 flushed=0/0


DEBUG 2017-09-26 23:59:18,831 [qtp173832146-34 - /channelstatus] org.eclipse.jetty.io.nio.ssl: [Session-1, SSL_NULL_WITH_NULL_NULL] SslConnection@2bc4c9c2 SSL NOT_HANDSHAKING i/o/u=0/0/0 ishut=false oshut=false {AsyncHttpConnection@4dc92409,g=HttpGenerator{s=2, h=0,b=0,c=-1},p=HttpParser{s=0,l=12,c=44},r=108498} NOT_HANDSHAKING filled=0/0 flushed=1030/0
DEBUG 2017-09-26 23:59:18,831 [qtp173832146-34 - /channelstatus] org.eclipse.jetty.io.nio.ssl: [Session-1, SSL_NULL_WITH_NULL_NULL] SslConnection@2bc4c9c2 SSL NOT_HANDSHAKING i/o/u=0/0/0 ishut=false oshut=false {AsyncHttpConnection@4dc92409,g=HttpGenerator{s=2, h=0,b=0,c=-1},p=HttpParser{s=0,l=12,c=44},r=108498} NOT_HANDSHAKING filled=0/0 flushed=0/0
DEBUG 2017-09-26 23:59:18,831 [qtp173832146-34 - /channelstatus] org.eclipse.jetty.io.nio.ssl: [Session-1, SSL_NULL_WITH_NULL_NULL] SslConnection@2bc4c9c2 SSL NOT_HANDSHAKING i/o/u=0/0/0 ishut=false oshut=false {AsyncHttpConnection@4dc92409,g=HttpGenerator{s=2, h=0,b=1026,c=-1},p=HttpParser{s=0,l=12,c=44},r=108498} NOT_HANDSHAKING filled=0/0 flushed=0/0
DEBUG 2017-09-26 23:59:18,831 [qtp173832146-34 - /channelstatus] org.eclipse.jetty.io.nio.ssl: [Session-1, SSL_NULL_WITH_NULL_NULL] wrap OK NOT_HANDSHAKING consumed=1026 produced=1055



Any idea what the problem could be?
And, perhaps more importantly: what would the solution (direction) be?

Thanks!
Reply With Quote
  #5  
Old 10-03-2017, 04:32 AM
Jurjan Jurjan is offline
OBX.1 Kenobi
 
Join Date: Feb 2011
Posts: 37
Jurjan is on a distinguished road
Default

It happended again.
some extra logging info:


See attached log extract!


Please help/advise
Attached Files
File Type: txt log_part.txt (76.2 KB, 5 views)
Reply With Quote
  #6  
Old 10-03-2017, 08:17 AM
brentm brentm is offline
Mirth Employee
 
Join Date: Jan 2012
Posts: 85
brentm is on a distinguished road
Default

Mirth Connect maintains an pool of connections to its internal database. It seems that its failing to detect when connections to the SQL Server database are closed and new connections need to be established.

Mirth Connect ships with two connection pool libraries, "HikariCP" and "DBCP". HikariCP is the default, but you can switch to DBCP by adding this to your mirth.properties file:

Code:
database.pool = DBCP
I'd recommend giving that a try to see if it fixes this problem.
Reply With Quote
  #7  
Old 10-05-2017, 03:08 AM
Jurjan Jurjan is offline
OBX.1 Kenobi
 
Join Date: Feb 2011
Posts: 37
Jurjan is on a distinguished road
Default

Brent,
thanks!

I'll forward your suggestion to the client (and their application support).

I'll let you know if this helps or not (hopefully the former!)
Reply With Quote
  #8  
Old 10-06-2017, 11:52 AM
satheeskumar satheeskumar is offline
OBX.1 Kenobi
 
Join Date: Aug 2015
Location: UK
Posts: 46
satheeskumar is on a distinguished road
Default

com.mirth.connect.donkey.server.data.DonkeyDaoExce ption: java.sql.SQLException: Invalid state, the Connection object is closed.

I am not sure how mirhtconnect maintains the connections in the connection pool. I faced the same problem in JBoss server when there is a glitch in the network connection / server between jboss server and SQL. the solution was to configure the connection pool (*-DS.xml) with more information such as validate the connection everytime fetched from the pool & flush the pool if there are not valid.

hope this might give some idea.
__________________
Mirth Certified
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -8. The time now is 01:41 PM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
Mirth Corporation