web stats
Mirth Performance Issue - Mirth Community

Go Back   Mirth Community > Mirth Connect > Support

Reply
 
Thread Tools Display Modes
  #1  
Old 09-04-2014, 08:10 AM
kunalgupta15 kunalgupta15 is offline
Mirth Newb
 
Join Date: Dec 2013
Posts: 7
kunalgupta15 is on a distinguished road
Default Mirth Performance Issue

1. Too many indexes are killing the database performance.

MESSAGE table have the following indexes:

KEY `MESSAGE_INDEX1` (`CHANNEL_ID`,`DATE_CREATED`),
KEY `MESSAGE_INDEX2` (`CHANNEL_ID`,`DATE_CREATED`,`CONNECTOR_NAME`),
KEY `MESSAGE_INDEX3` (`CHANNEL_ID`,`DATE_CREATED`,`RAW_DATA_PROTOCOL`),
KEY `MESSAGE_INDEX4` (`CHANNEL_ID`,`DATE_CREATED`,`SOURCE`),
KEY `MESSAGE_INDEX5` (`CHANNEL_ID`,`DATE_CREATED`,`STATUS`),
KEY `MESSAGE_INDEX6` (`CHANNEL_ID`,`DATE_CREATED`,`TYPE`),
KEY `MESSAGE_INDEX7` (`CORRELATION_ID`,`CONNECTOR_NAME`),
KEY `MESSAGE_INDEX8` (`ATTACHMENT`),

After dropping 7 indexes( 1,2,3,4,5,7,8), I/O performance improved from 75% to 35%.

Question : Do we need all the indexes? if Yes, why.


2. Mirth is using insert which is causing performance issue for too may inserts.

Question : Can you use Load data infile which provides 20x times faster performance for data load?
Reply With Quote
  #2  
Old 09-04-2014, 08:51 AM
narupley's Avatar
narupley narupley is online now
Mirth Employee
 
Join Date: Oct 2010
Posts: 7,119
narupley is on a distinguished road
Default

Those indexes have been consolidated in 3.x, so I would recommend upgrading.
__________________
Step 1: JAVA CACHE...DID YOU CLEAR ...wait, ding dong the witch is dead?

Nicholas Rupley
Work: 949-237-6069
Always include what Mirth Connect version you're working with. Also include (if applicable) the code you're using and full stacktraces for errors (use CODE tags). Posting your entire channel is helpful as well; make sure to scrub any PHI/passwords first.


- How do I foo?
- You just bar.
Reply With Quote
  #3  
Old 09-29-2014, 01:31 PM
kunalgupta15 kunalgupta15 is offline
Mirth Newb
 
Join Date: Dec 2013
Posts: 7
kunalgupta15 is on a distinguished road
Default

Mirth 3.0.3 Issue :

Most of the tables have 2-3 indexes and it is slowing down the write performance. We are getting 3MB/sec write, but our Disk(SSD drive) can handle 295MB/sec write. Can we drop some of the indexes to improve the write performance?

If we change the innodb_flush_log_at_trx_commit=2 ( write after 1 sec) in MySql, it improves the write performance by 40%. But we can't do this, because we don't want to lose any committed data.

Please let us know how to improve the write performance in Mysql database for Mirth 3.0.3 Our I/O is 99% on disk( SSD drive) for Mysql server and can't handle the write.
Reply With Quote
  #4  
Old 09-29-2014, 01:33 PM
narupley's Avatar
narupley narupley is online now
Mirth Employee
 
Join Date: Oct 2010
Posts: 7,119
narupley is on a distinguished road
Default

Quote:
Originally Posted by kunalgupta15 View Post
Mirth 3.0.3 Issue :

Most of the tables have 2-3 indexes and it is slowing down the write performance. We are getting 3MB/sec write, but our Disk(SSD drive) can handle 295MB/sec write. Can we drop some of the indexes to improve the write performance?

If we change the innodb_flush_log_at_trx_commit=2 ( write after 1 sec) in MySql, it improves the write performance by 40%. But we can't do this, because we don't want to lose any committed data.

Please let us know how to improve the write performance in Mysql database for Mirth 3.0.3 Our I/O is 99% on disk( SSD drive) for Mysql server and can't handle the write.
You can turn channel storage down to Raw. You won't be able to use destination queuing, but you will be able to use source queuing. At the very least the source raw content will be stored, so if an error occurs you can still reprocess the message.
__________________
Step 1: JAVA CACHE...DID YOU CLEAR ...wait, ding dong the witch is dead?

Nicholas Rupley
Work: 949-237-6069
Always include what Mirth Connect version you're working with. Also include (if applicable) the code you're using and full stacktraces for errors (use CODE tags). Posting your entire channel is helpful as well; make sure to scrub any PHI/passwords first.


- How do I foo?
- You just bar.
Reply With Quote
  #5  
Old 10-02-2014, 10:14 AM
kunalgupta15 kunalgupta15 is offline
Mirth Newb
 
Join Date: Dec 2013
Posts: 7
kunalgupta15 is on a distinguished road
Default

But then we wont be able to see transformed messages on mirth. Can you suggest something else?

What is the recommended configuration of My SQL machine for production?
Reply With Quote
  #6  
Old 10-02-2014, 12:22 PM
narupley's Avatar
narupley narupley is online now
Mirth Employee
 
Join Date: Oct 2010
Posts: 7,119
narupley is on a distinguished road
Default

Quote:
Originally Posted by kunalgupta15 View Post
But then we wont be able to see transformed messages on mirth. Can you suggest something else?

What is the recommended configuration of My SQL machine for production?
If the bottleneck is your database and the physical machine you're writing to, there's not really anything Mirth Connect can do about it. We have MC deployed in a variety of very high-volume production systems with no disk I/O problems (at least when SSD drives are used). The recommended approach is to upgrade your storage hardware to handle the additional load. Mirth Connect is only the software, obviously it's still going to be dependent on the physical machine you deploy it on.
__________________
Step 1: JAVA CACHE...DID YOU CLEAR ...wait, ding dong the witch is dead?

Nicholas Rupley
Work: 949-237-6069
Always include what Mirth Connect version you're working with. Also include (if applicable) the code you're using and full stacktraces for errors (use CODE tags). Posting your entire channel is helpful as well; make sure to scrub any PHI/passwords first.


- How do I foo?
- You just bar.
Reply With Quote
Reply

Tags
index, performance

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -8. The time now is 07:12 PM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2019, vBulletin Solutions, Inc.
Mirth Corporation