Жил да был себе один mysql-сервер с версией 5.5.13. В один прекрасный солнечный весенний день возникла необходимость обновить его до версии 5.6.10. Остановил mysql, удалил старые rpm-ки, поставил новые. Но вот после команды "/sbin/service mysql start" ждал меня следующий облом в error.log-е:
130319 11:10:54 mysqld_safe Number of processes running now: 0 130319 11:10:54 mysqld_safe mysqld restarted 2013-03-19 11:10:57 9319 [Note] Plugin 'FEDERATED' is disabled. 2013-03-19 11:10:57 9319 [Note] InnoDB: The InnoDB memory heap is disabled 2013-03-19 11:10:57 9319 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2013-03-19 11:10:57 9319 [Note] InnoDB: Compressed tables use zlib 1.2.3 2013-03-19 11:10:57 9319 [Note] InnoDB: CPU does not support crc32 instructions 2013-03-19 11:10:57 9319 [Note] InnoDB: Using Linux native AIO 2013-03-19 11:10:57 9319 [Note] InnoDB: Initializing buffer pool, size = 150.0G 2013-03-19 11:11:11 9319 [Note] InnoDB: Completed initialization of buffer pool 2013-03-19 11:11:11 9319 [Note] InnoDB: Highest supported file format is Barracuda. 2013-03-19 11:11:14 9319 [Note] InnoDB: Log scan progressed past the checkpoint lsn 810321211776 2013-03-19 11:11:14 9319 [Note] InnoDB: Database was not shutdown normally! 2013-03-19 11:11:14 9319 [Note] InnoDB: Starting crash recovery. 2013-03-19 11:11:14 9319 [Note] InnoDB: Reading tablespace information from the .ibd files... 2013-03-19 11:11:14 9319 [Note] InnoDB: Restoring possible half-written data pages 2013-03-19 11:11:14 9319 [Note] InnoDB: from the doublewrite buffer... InnoDB: Doing recovery: scanned up to log sequence number 810321211796 InnoDB: Last MySQL binlog file position 0 864691679, file name ./serv4-bin.000427 2013-03-19 11:11:20 9319 [Note] InnoDB: 128 rollback segment(s) are active. 2013-03-19 11:11:20 9319 [Note] InnoDB: Waiting for purge to start 2013-03-19 11:11:20 9319 [Note] InnoDB: 1.2.10 started; log sequence number 810321211796 2013-03-19 11:11:20 9319 [Note] Recovering after a crash using serv4-bin 2013-03-19 11:11:20 9319 [Note] Starting crash recovery... 2013-03-19 11:11:20 9319 [Note] Crash recovery finished. 2013-03-19 11:11:20 9319 [Note] Server hostname (bind-address): '0.0.0.0'; port: 3309 2013-03-19 11:11:20 9319 [Note] - '0.0.0.0' resolves to '0.0.0.0'; 2013-03-19 11:11:20 9319 [Note] Server socket created on IP: '0.0.0.0'. 2013-03-19 11:11:20 9319 [Warning] Info table is not ready to be used. Table 'mysql.slave_master_info' cannot be opened. 2013-03-19 11:11:20 9319 [Warning] Info table is not ready to be used. Table 'mysql.slave_relay_log_info' cannot be opened. 2013-03-19 11:11:20 9319 [ERROR] Column count of mysql.events_waits_current is wrong. Expected 19, found 16. Created with MySQL 50513, now running 50610. Please use mysql_upgrade to fix this error. 2013-03-19 11:11:20 9319 [Warning] Slave SQL: If a crash happens this configuration does not guarantee that the relay log info will be consistent, Error_code: 0 2013-03-19 11:11:20 9319 [ERROR] Column count of mysql.events_waits_history is wrong. Expected 19, found 16. Created with MySQL 50513, now running 50610. Please use mysql_upgrade to fix this error. 2013-03-19 11:11:20 9319 [ERROR] Column count of mysql.events_waits_history_long is wrong. Expected 19, found 16. Created with MySQL 50513, now running 50610. Please use mysql_upgrade to fix this error. 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'events_waits_summary_by_host_by_event_name' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Incorrect definition of table performance_schema.events_waits_summary_by_thread_by_event_name: expected column 'THREAD_ID' at position 0 to have type bigint(20), found type int(11). 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'events_waits_summary_by_user_by_event_name' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'events_waits_summary_by_account_by_event_name' has the wrong structure 11:11:20 UTC - mysqld got signal 11 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. key_buffer_size=33554432 read_buffer_size=2097152 max_used_connections=0 max_threads=10000 thread_count=2 connection_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 61607924 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x2ad3330edb90 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 43cd0930 thread_stack 0x30000 2013-03-19 11:11:20 9319 [ERROR] Column count of mysql.file_summary_by_event_name is wrong. Expected 23, found 5. Created with MySQL 50513, now running 50610. Please use mysql_upgrade to fix this error. 2013-03-19 11:11:20 9319 [ERROR] Column count of mysql.file_summary_by_instance is wrong. Expected 25, found 6. Created with MySQL 50513, now running 50610. Please use mysql_upgrade to fix this error. 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'host_cache' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Incorrect definition of table performance_schema.mutex_instances: expected column 'LOCKED_BY_THREAD_ID' at position 2 to have type bigint(20), found type int(11). /usr/sbin/mysqld(my_print_stacktrace+0x35)[0x8ff785] 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'objects_summary_global_by_type' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Incorrect definition of table performance_schema.rwlock_instances: expected column 'WRITE_LOCKED_BY_THREAD_ID' at position 2 to have type bigint(20), found type int(11). 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'setup_actors' has the wrong structure /usr/sbin/mysqld(handle_fatal_signal+0x3e8)[0x66e9b8] /lib64/libpthread.so.0[0x304880eb10] 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'setup_objects' has the wrong structure /usr/sbin/mysqld[0x694995] 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'table_io_waits_summary_by_index_usage' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'table_io_waits_summary_by_table' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'table_lock_waits_summary_by_table' has the wrong structure /usr/sbin/mysqld(_Z11mysql_grantP3THDPKcR4ListI11st_lex_userEmbb+0x525)[0x6987c5] 2013-03-19 11:11:20 9319 [ERROR] Column count of mysql.threads is wrong. Expected 14, found 3. Created with MySQL 50513, now running 50610. Please use mysql_upgrade to fix this error. 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'events_stages_current' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'events_stages_history' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'events_stages_history_long' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'events_stages_summary_by_thread_by_event_name' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'events_stages_summary_by_account_by_event_name' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'events_stages_summary_by_user_by_event_name' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'events_stages_summary_by_host_by_event_name' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'events_stages_summary_global_by_event_name' has the wrong structure /usr/sbin/mysqld(_Z21mysql_execute_commandP3THD+0x5a85)[0x6f0545] 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'events_statements_current' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'events_statements_history' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'events_statements_history_long' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'events_statements_summary_by_thread_by_event_name' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'events_statements_summary_by_account_by_event_name' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'events_statements_summary_by_user_by_event_name' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'events_statements_summary_by_host_by_event_name' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'events_statements_summary_global_by_event_name' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'events_statements_summary_by_digest' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'users' has the wrong structure /usr/sbin/mysqld(_Z11mysql_parseP3THDPcjP12Parser_state+0x318)[0x6f1858] 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'accounts' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'hosts' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'socket_instances' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'socket_summary_by_instance' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'socket_summary_by_event_name' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'session_connect_attrs' has the wrong structure 2013-03-19 11:11:20 9319 [ERROR] Native table 'performance_schema'.'session_account_connect_attrs' has the wrong structure /usr/sbin/mysqld(_ZN15Query_log_event14do_apply_eventEPK14Relay_log_infoPKcj+0x60b)[0x89676b] /usr/sbin/mysqld(_ZN9Log_event11apply_eventEP14Relay_log_info+0x74)[0x8907a4] /usr/sbin/mysqld(_Z26apply_event_and_update_posPP9Log_eventP3THDP14Relay_log_info+0x18a)[0x8d33fa] /usr/sbin/mysqld[0x8d3f68] /usr/sbin/mysqld(handle_slave_sql+0xd14)[0x8d5314] /usr/sbin/mysqld(pfs_spawn_thread+0x13b)[0x9790cb] /lib64/libpthread.so.0[0x304880673d] /lib64/libc.so.6(clone+0x6d)[0x30480d3f6d] Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (2ad3330f87b5): is an invalid pointer Connection ID (thread ID): 2 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash.
И так по кругу – mysqld пытается взлететь, у него не получается, он в конвульсиях пишет в лог всё вышепроцитированное, дохнет, mysqld_safe пытается перезапустить его заново... Поскольку основная ругань была на базу performance_schema, то я попробовал запустить mysql вот так:
/usr/sbin/mysqld --defaults-file=/etc/my4.cnf --basedir=/usr --datadir=/var/lib/mysql4 --plugin-dir=/usr/lib64/mysql/plugin --user=mysql --log-error=/var/lib/mysql4/error.log --open-files-limit=65536 --pid-file=/var/lib/mysql4/mysql-4.pid --socket=/var/lib/mysql4/mysql.sock --port=3309 --skip-performance-schema --read-only --skip-networking --skip-slave-start
Параметров так много, потому что на сервере крутится несколько экземпляров mysql на разных портах, каждый со своими базами. Ключевым параметром здесь является --skip-performance-schema. После этого mysql таки успешно взлетел и у меня появилась возможность выполнить команду
mysql_upgrade --socket=/var/lib/mysql4/mysql.sock
которая починила все ошибки в таблицах, связанные с изменением их структуры в новой версии. Далее тушим аккуратно этот экземпляр:
mysqladmin shutdown --socket=/var/lib/mysql4/mysql.sock
И запускаем его уже штатным образом:
/etc/init.d/mysql4 start
И наслаждаемся в полной мере новыми фичами версии 5.6 :)
Тоже нарвался, спасибо за инфу, помогло.