我的服务器应用程序通过单独的线程与
MySQL建立多个连接每个连接都会触发SELECT查询并获取结果,然后应用程序会将其返回给其连接的用户.
我正在使用InnoDB.令我惊讶的是,我发现如果我增加与MySQL的连接数量,查询性能会下降并且结果获取时间也会增加,这是非常奇怪的.下面是一张表格.
当我在MySQL表中有3333条记录并且基于给定的随机参数的SELECT查询从中获取大约450条记录时,会生成此数据.每条记录大约有10个字段,所有字段一起包含1.2 KB的数据. (因此,单个SELECT查询总共提取1.2 * 450 = 540 KB数据)
╔═══════════╦═══════════════╦══════════════╗ ║ Number of ║Query execution║ Result fetch ║ ║connections║ time range ║ time range ║ ║ to MySQL ║ (in seconds) ║ (in seconds) ║ ╠═══════════╬═══════════════╬══════════════╣ ║ 1 ║ 0.02 to 0.06 ║ 0.03 to 0.18 ║ ║ 7 ║ 0.23 to 0.64 ║ 0.54 to 0.74 ║ ║ 17 ║ 0.32 to 1.71 ║ 0.53 to 1.18 ║ ║ 37 ║ 0.37 to 2.01 ║ 0.70 to 1.70 ║ ║ 117 ║ 1.13 to 3.29 ║ 2.48 to 3.25 ║ ╚═══════════╩═══════════════╩══════════════╝
我在这里不明白的是,为什么MySQL增加连接数量会花费更多时间?特别是当没有对表进行更新时,MySQL应该在单独的线程中处理来自每个连接的SELECT请求.从而并发处理查询.因此,理想情况下,性能和提取不应该显着降低.
我不介意与数据库建立单一连接,但问题是我的服务器性能会因此而显着降低.成千上万的用户(连接到我的服务器)将不得不等待轮到他们的单个线程.
在经历了一些关于SO的相关问题后,我尝试增加
innodb_buffer_pool_size为1 GB,没有运气.
这是我的所有InnoDB参数:
innodb_adaptive_flushing ON
innodb_adaptive_flushing_lwm 10
innodb_adaptive_hash_index ON
innodb_adaptive_max_sleep_delay 150000
innodb_additional_mem_pool_size 2097152
innodb_api_bk_commit_interval 5
innodb_api_disable_rowlock OFF
innodb_api_enable_binlog OFF
innodb_api_enable_mdl OFF
innodb_api_trx_level 0
innodb_autoextend_increment 64
innodb_autoinc_lock_mode 1
innodb_buffer_pool_dump_at_shutdown OFF
innodb_buffer_pool_dump_now OFF
innodb_buffer_pool_filename ib_buffer_pool
innodb_buffer_pool_instances 8
innodb_buffer_pool_load_abort OFF
innodb_buffer_pool_load_at_startup OFF
innodb_buffer_pool_load_now OFF
innodb_buffer_pool_size 1073741824
innodb_change_buffer_max_size 25
innodb_change_buffering all
innodb_checksum_algorithm crc32
innodb_checksums ON
innodb_cmp_per_index_enabled OFF
innodb_commit_concurrency 0
innodb_compression_failure_threshold_pct 5
innodb_compression_level 6
innodb_compression_pad_pct_max 50
innodb_concurrency_tickets 5000
innodb_data_file_path ibdata1:12M:autoextend
innodb_data_home_dir
innodb_disable_sort_file_cache OFF
innodb_doublewrite ON
innodb_fast_shutdown 1
innodb_file_format Antelope
innodb_file_format_check ON
innodb_file_format_max Antelope
innodb_file_per_table ON
innodb_flush_log_at_timeout 1
innodb_flush_log_at_trx_commit 2
innodb_flush_method normal
innodb_flush_neighbors 1
innodb_flushing_avg_loops 30
innodb_force_load_corrupted OFF
innodb_force_recovery 0
innodb_ft_aux_table
innodb_ft_cache_size 8000000
innodb_ft_enable_diag_print OFF
innodb_ft_enable_stopword ON
innodb_ft_max_token_size 84
innodb_ft_min_token_size 3
innodb_ft_num_word_optimize 2000
innodb_ft_result_cache_limit 2000000000
innodb_ft_server_stopword_table
innodb_ft_sort_pll_degree 2
innodb_ft_total_cache_size 640000000
innodb_ft_user_stopword_table
innodb_io_capacity 200
innodb_io_capacity_max 2000
innodb_large_prefix OFF
innodb_lock_wait_timeout 50
innodb_locks_unsafe_for_binlog OFF
innodb_log_buffer_size 268435456
innodb_log_compressed_pages ON
innodb_log_file_size 262144000
innodb_log_files_in_group 2
innodb_log_group_home_dir .\
innodb_lru_scan_depth 1024
innodb_max_dirty_pages_pct 75
innodb_max_dirty_pages_pct_lwm 0
innodb_max_purge_lag 0
innodb_max_purge_lag_delay 0
innodb_mirrored_log_groups 1
innodb_monitor_disable
innodb_monitor_enable
innodb_monitor_reset
innodb_monitor_reset_all
innodb_old_blocks_pct 37
innodb_old_blocks_time 1000
innodb_online_alter_log_max_size 134217728
innodb_open_files 300
innodb_optimize_fulltext_only OFF
innodb_page_size 16384
innodb_print_all_deadlocks OFF
innodb_purge_batch_size 300
innodb_purge_threads 1
innodb_random_read_ahead OFF
innodb_read_ahead_threshold 56
innodb_read_io_threads 64
innodb_read_only OFF
innodb_replication_delay 0
innodb_rollback_on_timeout OFF
innodb_rollback_segments 128
innodb_sort_buffer_size 1048576
innodb_spin_wait_delay 6
innodb_stats_auto_recalc ON
innodb_stats_method nulls_equal
innodb_stats_on_metadata OFF
innodb_stats_persistent ON
innodb_stats_persistent_sample_pages 20
innodb_stats_sample_pages 8
innodb_stats_transient_sample_pages 8
innodb_status_output OFF
innodb_status_output_locks OFF
innodb_strict_mode OFF
innodb_support_xa ON
innodb_sync_array_size 1
innodb_sync_spin_loops 30
innodb_table_locks ON
innodb_thread_concurrency 8
innodb_thread_sleep_delay 0
innodb_undo_directory .
innodb_undo_logs 128
innodb_undo_tablespaces 0
innodb_use_native_aio OFF
innodb_use_sys_malloc ON
innodb_version 5.6.28
innodb_write_io_threads 16
有人可以请光吗?这很长时间困扰着我.
(注意:我没有在这个问题中提到实际查询,因为查询很复杂,这个问题不是关于那个查询.但是当查询相同时,它会增加连接的性能下降)
更新1
这是我的表的SHOW CREATE TABLE输出:
CREATE TABLE `profiles` (
`SRNO` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`HANDLE_FIRST` int(10) unsigned NOT NULL,
`HANDLE_SECOND` bigint(20) unsigned NOT NULL,
`USERID` binary(16) NOT NULL,
`UNIQUESTRING` char(10) NOT NULL,
`CLIENT_VERSION` smallint(5) unsigned NOT NULL,
`ISCONNECTED` bit(1) NOT NULL,
`ISPROFILEPRESENT` bit(1) NOT NULL,
`USERNAME` varchar(32) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
`GENDER` tinyint(1) DEFAULT NULL,
`DND` bit(1) DEFAULT NULL,
`STATUS` varchar(128) CHARACTER SET utf8 COLLATE utf8_unicode_ci DEFAULT NULL,
`PROFILE_URL` varchar(128) DEFAULT NULL,
PRIMARY KEY (`SRNO`),
UNIQUE KEY `USERID` (`USERID`),
KEY `USERID_INDEX` (`USERID`),
KEY `UNIQUESTRING_INDEX` (`UNIQUESTRING`),
KEY `ISCONNECTED_INDEX` (`ISCONNECTED`),
KEY `ISPROFILEPRESENT_INDEX` (`ISPROFILEPRESENT`)
) ENGINE=InnoDB AUTO_INCREMENT=9250 DEFAULT CHARSET=utf8
CREATE TABLE `blockers` (
`BLOCKER_PROFILE_SRNO` bigint(20) unsigned NOT NULL,
`BLOCKED_PROFILE_SRNO` bigint(20) unsigned NOT NULL,
UNIQUE KEY `BLOCKER_PROFILE_SRNO` (`BLOCKER_PROFILE_SRNO`,`BLOCKED_PROFILE_SRNO`),
KEY `BLOCKER_PROFILE_SRNO_INDEX` (`BLOCKER_PROFILE_SRNO`),
KEY `BLOCKED_PROFILE_SRNO_INDEX` (`BLOCKED_PROFILE_SRNO`),
CONSTRAINT `fk_BlockedIndex` FOREIGN KEY (`BLOCKED_PROFILE_SRNO`) REFERENCES `profiles` (`SRNO`),
CONSTRAINT `fk_BlockerIndex` FOREIGN KEY (`BLOCKER_PROFILE_SRNO`) REFERENCES `profiles` (`SRNO`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
这是我正在运行的查询:
select prfls.*
from profiles as prfls
left outer join blockers as blkr1 on blkr1.blocker_profile_srno = prfls.srno
and blkr1.blocked_profile_srno = 6443
left outer join blockers as blkr2 on blkr2.blocker_profile_srno = 6443
and blkr2.blocked_profile_srno = prfls.srno
where blkr1.blocker_profile_srno is null
and blkr2.blocker_profile_srno is null
and ( (prfls.uniquestring like 'phk5600dcc%')
or (prfls.uniquestring like 'phk5600dcf%')
)
and prfls.isconnected=1
and prfls.isprofilepresent=1
limit 450
此查询本质上是一个准备好的语句,其中blocked_profile_srno,blocker_profile_srno和uniquestring参数不断更改每个查询.但是,blocked_profile_srno和blocker_profile_srno始终保持相等(在上面的查询中,它们的值为6443).表格阻止程序是空白的(我将它用于将来使用,但目前它没有数据)
当117个连接同时运行查询时,输出SHOW GLOBAL STATUS LIKE’Threads_running’;大部分时间都是1.然而它有时会达到27.同时,输出SHOW GLOBAL STATUS LIKE’Max_used_connections’;是130
更新2
我可以从Rick James回答下面的回答,优化查询减少了查询执行时间范围.该时间范围仍然随着连接数量的增加而增加但在可接受的范围内.这就是我接受答案的原因.
最佳答案 可能每个连接都在对配置文件进行全表扫描.我们试着避免这种情况.当有数十个查询命中同一个表时,会有锁导致InnoDB“绊倒自己”.这些计划中的任何一个都将加速查询并减少触摸的行数(因此减少锁定).使用建议的“复合”索引将加快查询速度.但OR妨碍了.我看到两个技巧仍然有一个索引看uniquestring,但避免一些或所有的OR.
( (prfls.uniquestring like 'phk5600dcc%')
or (prfls.uniquestring like 'phk5600dcf%')
)
或者很难优化.
添加这个:
INDEX(isconnected, isprofilepresent, uniquestring)
然后…
计划A:
prfls.uniquestring like 'phk5600dc%' AND -- note common prefix
( (prfls.uniquestring like 'phk5600dcc%')
or (prfls.uniquestring like 'phk5600dcf%')
)
这假设您可以构造该公共前缀.
B计划(转入OR进入UNION):
( SELECT ...
WHERE prfls.uniquestring like 'phk5600dcc%' AND ...
LIMIT 450 )
UNION ALL -- ? You may want DISTINCT, if there could be dups
( SELECT ...
WHERE prfls.uniquestring like 'phk5600dcf%' AND ... -- the only diff
LIMIT 450 )
LIMIT 450 -- yes, again
计划A(如果可行的话)利用了似乎是常见的起始值. B计划无论如何都可以,但可能会慢一点,尽管仍然比原来快很多.
其他说明……
标志(你有两个)的索引几乎从未使用过. EXPLAIN SELECT …可能会显示两者都没有使用过.请为需要讨论的任何SELECT提供EXPLAIN.
UNIQUE KEY是一个KEY,因此不需要USERID上的冗余索引.
限制450 – 你想要450?如果没有ORDER BY,则允许查询给你任何450.(当然,也许这很好.)(而ORDER BY可能会减慢查询速度.)
我的建议不会“解决”问题,但他们应该在减速变得明显之前增加连接数.