文章目录
一、查询日志
说明:
查询日志记录了数据库客户端所有的语句,二进制日志不包含查询数据的SQL。
参数说明:
3306[youngboy]>select @@general_log;
+---------------+
| @@general_log |
+---------------+
| 0 |
+---------------+
1 row in set (0.00 sec)
general_log=1/0 是否开启querylog 默认0关闭
3306[youngboy]>select @@general_log_file;
+-------------------------------------+
| @@general_log_file |
+-------------------------------------+
| /opt/mysql-data/mysql/localhost.log |
+-------------------------------------+
1 row in set (0.00 sec)
general_log_file=/opt/mysql-data/mysql/localhost.log 路径
3306[youngboy]>select @@log_output;
+--------------+
| @@log_output |
+--------------+
| FILE |
+--------------+
1 row in set (0.00 sec)
value值可以是TABLE、FILE、NONE
TABLE:表示日志输出到表
FILE:表示日志输出到文件
NONE:表示日志不写到表也不写到文件
如果设置输出到表则:
querylog:记录在mysql库的general_log表中
slowlog:记录在mysql库的slow_log表中
默认输出到文件,性能更高,因为表会比文件占到更多的资源。
注:querylog在数据库中用的不多,会记录所有数据库的操作,需要经常访问的数据库,这个日志对数据库的性能影响会比较大,所以一般是关闭的。
二、binlog(二进制日志) *****
binlog作用及参数
binlog记录了所有的DDL/DML语句,不包括DQL语句,是用event形式存放的,在备份恢复和主从环境中很重要。
vim /etc/my.cnf
server_id=6 ----->5.6中,单机可以不需要此参数
log_bin=/opt/mysql-data/binlog/mysql-bin
binlog_format=row
sync_binlog=1
max_binlog_size=1024000000 日志文件到达1个G自动滚动日志到下一个
重启数据库生效
systemctl restart mysqld
注:MySQL数据库默认不开启二进制日志
相关命令:
开关及路径:
3306[youngboy]>select @@log_bin; 查看二进制日志是否开启
+-----------+
| @@log_bin |
+-----------+
| 1 |
+-----------+
1 row in set (0.00 sec)
3306[(none)]>select @@log_bin_basename; 查看二进制日志文件路径
+----------------------------------+
| @@log_bin_basename |
+----------------------------------+
| /opt/mysql-data/binlog/mysql-bin |
+----------------------------------+
1 row in set (0.01 sec)
服务ID:
3306[(none)]>select @@server_id; 查看ID号,主要用在主从
+-------------+
| @@server_id |
+-------------+
| 6 |
+-------------+
1 row in set (0.00 sec)
二进制日志格式:
3306[(none)]>select @@binlog_format;
+-----------------+
| @@binlog_format |
+-----------------+
| ROW |
+-----------------+
1 row in set (0.00 sec)
双一标准的另一个参数:
3306[(none)]>select @@sync_binlog; 同步二进制日志
+---------------+
| @@sync_binlog |
+---------------+
| 1 |
+---------------+
1 row in set (0.00 sec)
3306[youngboy]>show binary logs; 查看有几个二进制文件
+------------------+-----------+
| Log_name | File_size |
+------------------+-----------+
| mysql-bin.000007 | 89629964 |
| mysql-bin.000008 | 509 |
+------------------+-----------+
2 rows in set (0.00 sec)
3306[youngboy]>flush logs; 刷新一个新的binlog日志
Query OK, 0 rows affected (0.02 sec)
日志滚动重启mysql也会自动刷新一个,达到设置日志的大小也会自动滚动一个。
3306[(none)]>show master status ; 查看正在使用的binlog日志
+------------------+----------+--------------+------------------+---------------------------------------------------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+---------------------------------------------------------------------------------------+
| mysql-bin.000008 | 234 | | | 0a99e3f9-3a95-11ea-87df-000c29b6e0ec:1-12,
979de5b8-3c01-11ea-8252-000c29b6e0ec:1-326 |
+------------------+----------+--------------+------------------+---------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
二进制日志格式参数说明(binlog_format):
statement(SBR):原封不动的记录用户操作的DML语句
ROW(RBR):记录用户进行SQL操作后每一行的记录,也就是数据行的变化(日志量比statement大很多,因为需要记录行的变化,大表数据量会非常大)
select查询语句不记录
mixed(MBR):以上两种模式的混合,用两种模式的优点,避开他们的缺点。
事件(event)
事件的组成
1、事件在二进制中的开始标识
2、事件的内容
3、事件在二进制中的结束标识
4、事件的示例以及参数说明
3306[youngboy]>show binlog events in 'mysql-bin.000008';
+------------------+-----+----------------+-----------+-------------+---------------------------------------------------------------------------------------+
| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |
+------------------+-----+----------------+-----------+-------------+---------------------------------------------------------------------------------------+
| mysql-bin.000008 | 4 | Format_desc | 6 | 123 | Server ver: 5.7.20-log, Binlog ver: 4 |
| mysql-bin.000008 | 123 | Previous_gtids | 6 | 234 | 0a99e3f9-3a95-11ea-87df-000c29b6e0ec:1-12,
979de5b8-3c01-11ea-8252-000c29b6e0ec:1-326 |
| mysql-bin.000008 | 234 | Gtid | 6 | 299 | SET @@SESSION.GTID_NEXT= '979de5b8-3c01-11ea-8252-000c29b6e0ec:327' |
| mysql-bin.000008 | 299 | Query | 6 | 375 | BEGIN |
| mysql-bin.000008 | 375 | Table_map | 6 | 427 | table_id: 267 (youngboy.t1) |
| mysql-bin.000008 | 427 | Write_rows | 6 | 478 | table_id: 267 flags: STMT_END_F |
| mysql-bin.000008 | 478 | Xid | 6 | 509 | COMMIT /* xid=1506670 */ |
+------------------+-----+----------------+-----------+-------------+---------------------------------------------------------------------------------------+
7 rows in set (0.00 sec)
如果一个binlog中日志太多还可以做限制,显示几条信息:
3306[youngboy]>show binlog events in 'mysql-bin.000008' limit 6;
+------------------+-----+----------------+-----------+-------------+---------------------------------------------------------------------------------------+
| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |
+------------------+-----+----------------+-----------+-------------+---------------------------------------------------------------------------------------+
| mysql-bin.000008 | 4 | Format_desc | 6 | 123 | Server ver: 5.7.20-log, Binlog ver: 4 |
| mysql-bin.000008 | 123 | Previous_gtids | 6 | 234 | 0a99e3f9-3a95-11ea-87df-000c29b6e0ec:1-12,
979de5b8-3c01-11ea-8252-000c29b6e0ec:1-326 |
| mysql-bin.000008 | 234 | Gtid | 6 | 299 | SET @@SESSION.GTID_NEXT= '979de5b8-3c01-11ea-8252-000c29b6e0ec:327' |
| mysql-bin.000008 | 299 | Query | 6 | 375 | BEGIN |
| mysql-bin.000008 | 375 | Table_map | 6 | 427 | table_id: 267 (youngboy.t1) |
| mysql-bin.000008 | 427 | Write_rows | 6 | 478 | table_id: 267 flags: STMT_END_F |
+------------------+-----+----------------+-----------+-------------+---------------------------------------------------------------------------------------+
6 rows in set (0.00 sec)
Log_name:binlog文件名
Pos:事件开始的position
Event_type:事件类型
Format_desc:MySQL识别binlog必要信息,没有这些信息识别不了
Server_id:mysql服务号,在主从中主要对应哪台机器
End_log_pos:事件结束的位置号
Info:事件内容,二进制一般看不懂,只知道做了什么操作
注:上一个结束的pos号就是下一个开始的pos号,不需要加1。
如果不知道格式,还可以使用help show binlog events;来查看events的使用
通过过滤的方式查看到二进制日志里面insert的操作
[root@localhost binlog]# mysql -e "show binlog events in 'mysql-bin.000007'" | grep insert
mysql-bin.000007 737771 Query 6 738457 use `mysql`; CREATE TABLE `ndb_binlog_index` (\n `Position` bigint(20) unsigned NOT NULL,\n `File` varchar(255) NOT NULL,\n `epoch` bigint(20) unsigned NOT NULL,\n `inserts` int(10) unsigned NOT NULL,\n `updates` int(10) unsigned NOT NULL,\n `deletes` int(10) unsigned NOT NULL,\n `schemaops` int(10) unsigned NOT NULL,\n `orig_server_id` int(10) unsigned NOT NULL,\n `orig_epoch` bigint(20) unsigned NOT NULL,\n `gci` int(10) unsigned NOT NULL,\n `next_position` bigint(20) unsigned NOT NULL,\n `next_file` varchar(255) NOT NULL,\n PRIMARY KEY (`epoch`,`orig_server_id`,`orig_epoch`)\n) ENGINE=MyISAM DEFAULT CHARSET=latin1
事件的说明
1、事件是二进制日志中最小的记录单元
2、相对于DDL、DCL来说,一条语句就是一个event
3、相对于DML来说已提交的事务中begin和commit在内的每一条语句都是一个事件
例:
begin; 第一个事件
insert into xxx values(xxx,xxx); 第二个事件
insert into xxx values(xxx,xxx); ...(可以包括N个事件,看事务语句有多少)
commit; 最后一个事件
binlog使用
binlog文件内容查看
日志说明:
1、从BEGIN到COMMIT就是一个日志的完整binlog日志
2、日志中binlog中绿色的两行就是ROW模式下的日志,加密数据,需要解密才能查看
3、详细日志中的“@1=8”可以看出来是插入了一行数据到第8行
直接查看binlog日志信息:
[root@localhost binlog]# mysqlbinlog mysql-bin.000008
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at 4
#200224 10:39:10 server id 6 end_log_pos 123 CRC32 0xeb936dbc Start: binlog v 4, server v 5.7.20-log created 200224 10:39:10
BINLOG '
TjdTXg8GAAAAdwAAAHsAAAAAAAQANS43LjIwLWxvZwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAEzgNAAgAEgAEBAQEEgAAXwAEGggAAAAICAgCAAAACgoKKioAEjQA
Abxtk+s=
'/*!*/;
# at 123
#200224 10:39:10 server id 6 end_log_pos 234 CRC32 0x51fdd69a Previous-GTIDs
# 0a99e3f9-3a95-11ea-87df-000c29b6e0ec:1-12,
# 979de5b8-3c01-11ea-8252-000c29b6e0ec:1-326
# at 234
#200224 10:45:37 server id 6 end_log_pos 299 CRC32 0xb51fc4a2 GTID last_committed=0 sequence_number=1 rbr_only=yes
/*!50718 SET TRANSACTION ISOLATION LEVEL READ COMMITTED*//*!*/;
SET @@SESSION.GTID_NEXT= '979de5b8-3c01-11ea-8252-000c29b6e0ec:327'/*!*/;
# at 299
#200224 10:45:28 server id 6 end_log_pos 375 CRC32 0xfc0d9d46 Query thread_id=53 exec_time=0 error_code=0
SET TIMESTAMP=1582512328/*!*/;
SET @@session.pseudo_thread_id=53/*!*/;
SET @@session.foreign_key_checks=1, @@session.sql_auto_is_null=0, @@session.unique_checks=1, @@session.autocommit=1/*!*/;
SET @@session.sql_mode=1436549152/*!*/;
SET @@session.auto_increment_increment=1, @@session.auto_increment_offset=1/*!*/;
/*!\C utf8 *//*!*/;
SET @@session.character_set_client=33,@@session.collation_connection=33,@@session.collation_server=45/*!*/;
SET @@session.lc_time_names=0/*!*/;
SET @@session.collation_database=DEFAULT/*!*/;
BEGIN
/*!*/;
# at 375
#200224 10:45:28 server id 6 end_log_pos 427 CRC32 0xe113aadb Table_map: `youngboy`.`t1` mapped to number 267
# at 427
#200224 10:45:28 server id 6 end_log_pos 478 CRC32 0x49664d8c Write_rows: table id 267 flags: STMT_END_F
BINLOG '
yDhTXhMGAAAANAAAAKsBAAAAAAsBAAAAAAEACHlvdW5nYm95AAJ0MQACAw8CUAAD26oT4Q==
yDhTXh4GAAAAMwAAAN4BAAAAAAsBAAAAAAEAAgAC//wIAAAACmhkZmVuZzIzNDWMTWZJ
'/*!*/;
# at 478
#200224 10:45:37 server id 6 end_log_pos 509 CRC32 0x3681e7dd Xid = 1506670
COMMIT/*!*/;
---------------------------------------------------------------------------------------------------------------
查看更详细的日志信息:
[root@localhost binlog]# mysqlbinlog --base64-output=decode-rows -vv /data/binlog/mysql-bin.000009
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
mysqlbinlog: File '/data/binlog/mysql-bin.000009' not found (Errcode: 2 - No such file or directory)
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
DELIMITER ;
# End of log file
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
[root@localhost binlog]# mysqlbinlog --base64-output=decode-rows -vvv mysql-bin.000008
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at 4
#200224 10:39:10 server id 6 end_log_pos 123 CRC32 0xeb936dbc Start: binlog v 4, server v 5.7.20-log created 200224 10:39:10
# at 123
#200224 10:39:10 server id 6 end_log_pos 234 CRC32 0x51fdd69a Previous-GTIDs
# 0a99e3f9-3a95-11ea-87df-000c29b6e0ec:1-12,
# 979de5b8-3c01-11ea-8252-000c29b6e0ec:1-326
# at 234
#200224 10:45:37 server id 6 end_log_pos 299 CRC32 0xb51fc4a2 GTID last_committed=0 sequence_number=1 rbr_only=yes
/*!50718 SET TRANSACTION ISOLATION LEVEL READ COMMITTED*//*!*/;
SET @@SESSION.GTID_NEXT= '979de5b8-3c01-11ea-8252-000c29b6e0ec:327'/*!*/;
# at 299
#200224 10:45:28 server id 6 end_log_pos 375 CRC32 0xfc0d9d46 Query thread_id=53 exec_time=0 error_code=0
SET TIMESTAMP=1582512328/*!*/;
SET @@session.pseudo_thread_id=53/*!*/;
SET @@session.foreign_key_checks=1, @@session.sql_auto_is_null=0, @@session.unique_checks=1, @@session.autocommit=1/*!*/;
SET @@session.sql_mode=1436549152/*!*/;
SET @@session.auto_increment_increment=1, @@session.auto_increment_offset=1/*!*/;
/*!\C utf8 *//*!*/;
SET @@session.character_set_client=33,@@session.collation_connection=33,@@session.collation_server=45/*!*/;
SET @@session.lc_time_names=0/*!*/;
SET @@session.collation_database=DEFAULT/*!*/;
BEGIN
/*!*/;
# at 375
#200224 10:45:28 server id 6 end_log_pos 427 CRC32 0xe113aadb Table_map: `youngboy`.`t1` mapped to number 267
# at 427
#200224 10:45:28 server id 6 end_log_pos 478 CRC32 0x49664d8c Write_rows: table id 267 flags: STMT_END_F
### INSERT INTO `youngboy`.`t1`
### SET
### @1=8 /* INT meta=0 nullable=1 is_null=0 */
### @2='hdfeng2345' /* VARSTRING(80) meta=80 nullable=1 is_null=0 */
# at 478
#200224 10:45:37 server id 6 end_log_pos 509 CRC32 0x3681e7dd Xid = 1506670
COMMIT/*!*/;
---------------------------------------------------------------------------------------------------------------
mysqlbinlog -d world mysql-bin.000008 查看在某个库下的二进制日志
mysqlbinlog --start-datetime='2020-02-24 8:00:00' --stop-datetime='2020-02-24 9:00:00' mysql-bin.000008 按时间进行查看
日志的截取
–start-position=xxx --stop-position=xxx
下列是截取的内容
[root@localhost binlog]# mysqlbinlog --start-position=299 --stop-position=478 mysql-bin.000008
# at 299
#200224 10:45:28 server id 6 end_log_pos 375 CRC32 0xfc0d9d46 Query thread_id=53 exec_time=0 error_code=0
SET TIMESTAMP=1582512328/*!*/;
SET @@session.pseudo_thread_id=53/*!*/;
SET @@session.foreign_key_checks=1, @@session.sql_auto_is_null=0, @@session.unique_checks=1, @@session.autocommit=1/*!*/;
SET @@session.sql_mode=1436549152/*!*/;
SET @@session.auto_increment_increment=1, @@session.auto_increment_offset=1/*!*/;
/*!\C utf8 *//*!*/;
SET @@session.character_set_client=33,@@session.collation_connection=33,@@session.collation_server=45/*!*/;
SET @@session.lc_time_names=0/*!*/;
SET @@session.collation_database=DEFAULT/*!*/;
BEGIN
/*!*/;
# at 375
#200224 10:45:28 server id 6 end_log_pos 427 CRC32 0xe113aadb Table_map: `youngboy`.`t1` mapped to number 267
# at 427
#200224 10:45:28 server id 6 end_log_pos 478 CRC32 0x49664d8c Write_rows: table id 267 flags: STMT_END_F
BINLOG '
yDhTXhMGAAAANAAAAKsBAAAAAAsBAAAAAAEACHlvdW5nYm95AAJ0MQACAw8CUAAD26oT4Q==
yDhTXh4GAAAAMwAAAN4BAAAAAAsBAAAAAAEAAgAC//wIAAAACmhkZmVuZzIzNDWMTWZJ
'/*!*/;
ROLLBACK /* added by mysqlbinlog */ /*!*/;
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
DELIMITER ;
# End of log file
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
---------------------------------------------------------------------------------------------------------------
[root@localhost binlog]# mysqlbinlog --start-position=299 --stop-position=478 mysql-bin.000008 > /tmp/2020-02-24binlog.sql
将截取的binlog输出到sql文件,进行恢复使用
日志的删除
单机模式下:
方法1:
reset master; 直接删除所有日志从000001开始
查看当前数据库下binlog日志文件:
3306[(none)]>system ls -ltrh /opt/mysql-data/binlog
总用量 16K
-rw-r-----. 1 mysql mysql 1.7K 2月 24 17:15 mysql-bin.000001
-rw-r-----. 1 mysql mysql 257 2月 24 17:15 mysql-bin.000002
-rw-r-----. 1 mysql mysql 120 2月 24 17:16 mysql-bin.index
-rw-r-----. 1 mysql mysql 447 2月 24 17:38 mysql-bin.000003
注:单机模式下可以这样进行删除,如果是主从模式下会导致主从数据不一致等一系列问题。
方法2:
purge master logs to 'mysql-bin.000002';
删除000002之前所有的binlog日志
3306[(none)]>system ls -ltrh /opt/mysql-data/binlog
总用量 8.0K
-rw-r-----. 1 mysql mysql 447 2月 24 17:38 mysql-bin.000003
-rw-r-----. 1 mysql mysql 40 2月 24 17:49 mysql-bin.index
删除了000003之前所有的日志
方法3:
用日期的方式删除,删除17:51分之前的所有日志
3306[(none)]>system ls -ltrh /opt/mysql-data/binlog
总用量 44K
-rw-r-----. 1 mysql mysql 494 2月 24 17:50 mysql-bin.000003
-rw-r-----. 1 mysql mysql 281 2月 24 17:50 mysql-bin.000004
-rw-r-----. 1 mysql mysql 281 2月 24 17:50 mysql-bin.000005
-rw-r-----. 1 mysql mysql 281 2月 24 17:50 mysql-bin.000006
-rw-r-----. 1 mysql mysql 281 2月 24 17:50 mysql-bin.000007
-rw-r-----. 1 mysql mysql 281 2月 24 17:50 mysql-bin.000008
-rw-r-----. 1 mysql mysql 281 2月 24 17:50 mysql-bin.000009
-rw-r-----. 1 mysql mysql 281 2月 24 17:50 mysql-bin.000010
-rw-r-----. 1 mysql mysql 281 2月 24 17:51 mysql-bin.000011
-rw-r-----. 1 mysql mysql 234 2月 24 17:51 mysql-bin.000012
-rw-r-----. 1 mysql mysql 400 2月 24 17:51 mysql-bin.index
3306[(none)]>purge master logs before '2020-02-24 17:51:00';
Query OK, 0 rows affected (0.02 sec)
3306[(none)]>system ls -ltrh /opt/mysql-data/binlog
总用量 12K
-rw-r-----. 1 mysql mysql 281 2月 24 17:51 mysql-bin.000011
-rw-r-----. 1 mysql mysql 234 2月 24 17:51 mysql-bin.000012
-rw-r-----. 1 mysql mysql 80 2月 24 18:42 mysql-bin.index
方法4:
使用设置参数的方法去定期的让MySQL自己去删除日志
3306[(none)]>select @@expire_logs_days;
+--------------------+
| @@expire_logs_days |
+--------------------+
| 15 |
+--------------------+
1 row in set (0.00 sec)
可以设置成:
set global expire_logs_days=8;设置全局模式的清理时间为8天
也可以在数据库配置文件中加入:
vim /etc/my.cnf
expire_logs_days=8;
binlog模拟日志丢失进行恢复(pos方式)
步骤:
1、创建测试库,以及数据录入,删除库的操作
3306[(none)]>create database binlogtest charset utf8mb4;
Query OK, 1 row affected (0.00 sec)
3306[(none)]>show binlog events in 'mysql-bin.000010';
+------------------+-----+----------------+-----------+-------------+---------------------------------------------------------------------------------------+
| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |
+------------------+-----+----------------+-----------+-------------+---------------------------------------------------------------------------------------+
| mysql-bin.000010 | 4 | Format_desc | 6 | 123 | Server ver: 5.7.20-log, Binlog ver: 4 |
| mysql-bin.000010 | 123 | Previous_gtids | 6 | 234 | 0a99e3f9-3a95-11ea-87df-000c29b6e0ec:1-12,
979de5b8-3c01-11ea-8252-000c29b6e0ec:1-327 |
| mysql-bin.000010 | 234 | Gtid | 6 | 299 | SET @@SESSION.GTID_NEXT= '979de5b8-3c01-11ea-8252-000c29b6e0ec:328' |
| mysql-bin.000010 | 299 | Query | 6 | 427 | create database binlogtest charset utf8mb4 |
+------------------+-----+----------------+-----------+-------------+---------------------------------------------------------------------------------------+
4 rows in set (0.00 sec)
3306[(none)]>use binlogtest;
3306[binlogtest]>create table binlog1(id int,sname varchar(20));
Query OK, 0 rows affected (0.04 sec)
3306[binlogtest]>insert into binlog1 values(1,'sfsdfwr'),(2,'hdfeng1'),(3,'ljwfesf'),(4,'cxwsdf'),(5,'hdfeng2'),(6,'wet3465sdf');
Query OK, 6 rows affected (0.00 sec)
Records: 6 Duplicates: 0 Warnings: 0
3306[binlogtest]>drop database binlogtest;
Query OK, 1 row affected (0.05 sec)
2、库被删了,需要进行恢复,得先去找数据的开始点和删除表之前的结束点
3306[(none)]>show master status ;
+------------------+----------+--------------+------------------+---------------------------------------------------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+---------------------------------------------------------------------------------------+
| mysql-bin.000010 | 1147 | | | 0a99e3f9-3a95-11ea-87df-000c29b6e0ec:1-12,
979de5b8-3c01-11ea-8252-000c29b6e0ec:1-331 |
+------------------+----------+--------------+------------------+---------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
3306[(none)]>show binlog events in 'mysql-bin.000010';
+------------------+------+----------------+-----------+-------------+---------------------------------------------------------------------------------------+
| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |
+------------------+------+----------------+-----------+-------------+---------------------------------------------------------------------------------------+
| mysql-bin.000010 | 4 | Format_desc | 6 | 123 | Server ver: 5.7.20-log, Binlog ver: 4 |
| mysql-bin.000010 | 123 | Previous_gtids | 6 | 234 | 0a99e3f9-3a95-11ea-87df-000c29b6e0ec:1-12,
979de5b8-3c01-11ea-8252-000c29b6e0ec:1-327 |
| mysql-bin.000010 | 234 | Gtid | 6 | 299 | SET @@SESSION.GTID_NEXT= '979de5b8-3c01-11ea-8252-000c29b6e0ec:328' |
| mysql-bin.000010 | 299 | Query | 6 | 427 | create database binlogtest charset utf8mb4 |
| mysql-bin.000010 | 427 | Gtid | 6 | 492 | SET @@SESSION.GTID_NEXT= '979de5b8-3c01-11ea-8252-000c29b6e0ec:329' |
| mysql-bin.000010 | 492 | Query | 6 | 624 | use `binlogtest`; create table binlog1(id int,sname varchar(20)) |
| mysql-bin.000010 | 624 | Gtid | 6 | 689 | SET @@SESSION.GTID_NEXT= '979de5b8-3c01-11ea-8252-000c29b6e0ec:330' |
| mysql-bin.000010 | 689 | Query | 6 | 767 | BEGIN |
| mysql-bin.000010 | 767 | Table_map | 6 | 826 | table_id: 298 (binlogtest.binlog1) |
| mysql-bin.000010 | 826 | Write_rows | 6 | 941 | table_id: 298 flags: STMT_END_F |
| mysql-bin.000010 | 941 | Xid | 6 | 972 | COMMIT /* xid=1506711 */ |
| mysql-bin.000010 | 972 | Gtid | 6 | 1037 | SET @@SESSION.GTID_NEXT= '979de5b8-3c01-11ea-8252-000c29b6e0ec:331' |
| mysql-bin.000010 | 1037 | Query | 6 | 1147 | drop database binlogtest |
+------------------+------+----------------+-----------+-------------+---------------------------------------------------------------------------------------+
13 rows in set (0.00 sec)
发现是从299-1037为止。
3、找到二进制日志以后,进行二进制日志的截取,并且导出
[root@localhost binlog]# mysqlbinlog --start-position=299 --stop-position=1037 mysql-bin.000010 > /tmp/binlogtest.sql
[root@localhost tmp]# ll
-rw-r--r--. 1 root root 3145 2月 24 14:27 binlogtest.sql
4、截取成功以后,因为二进制日志会记录在数据库里面的操作,所以恢复之前需要先把二进制日志给关掉。
3306[(none)]>set sql_log_bin=0;
Query OK, 0 rows affected (0.00 sec)
5、进行数据的恢复,再打开二进制日志,进行正常数据的日志记录。
3306[(none)]>source /tmp/binlogtest.sql;
Query OK, 0 rows affected (0.01 sec)
Query OK, 0 rows affected (0.00 sec)
Charset changed
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected, 1 warning (0.00 sec)
Database changed
Query OK, 0 rows affected (0.00 sec)
Query OK, 0 rows affected (0.03 sec)
执行成功以后看看库是否恢复
3306[binlogtest]>show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| binlogtest |
| day1 |
| gtid |
| mysql |
| performance_schema |
| sys |
| test |
| world |
| youngboy |
+--------------------+
10 rows in set (0.00 sec)
3306[binlogtest]>select * from binlog1;
+------+------------+
| id | sname |
+------+------------+
| 1 | sfsdfwr |
| 2 | hdfeng1 |
| 3 | ljwfesf |
| 4 | cxwsdf |
| 5 | hdfeng2 |
| 6 | wet3465sdf |
+------+------------+
6 rows in set (0.00 sec)
数据恢复成功!
开启binlog:
3306[binlogtest]>set sql_log_bin=1;
Query OK, 0 rows affected (0.00 sec)
3306[binlogtest]>select @@sql_log_bin;
+---------------+
| @@sql_log_bin |
+---------------+
| 1 |
+---------------+
1 row in set (0.00 sec)
binlog(GTID方式)
说明:GTID(Global Transaction ID)全局事务编号,记录在日志中的每一个事务都会有一个这样的编号,并且这个编号是在数据库全局唯一的一个编号。

版本说明:
5.6开始这个新的特性,不开启没有这个功能
5.7对于GTID的功能进行了加强,不开也自动生成,关闭比较麻烦,需要3个步骤
相对于pos方式的优点:pos如果遇到截取的内容是分段的,就要分次进行截取,而GTID则不用,直接截取相应的gtid号即可截出对应日志,而且还方便恢复。
GTID参数说明
参数介绍:
vim /etc/my.cnf
gtid-mode=on GTID模式开启
enforce-gtid-consistency=true 强制GTID一致性
GTID的使用
1、查看GTID号,下列Executed_Gtid_Set中就是GTID号码,冒号前的是source ID,后面的1-12,1-335就是事务号。
3306[gtid]>show master status ;
+------------------+----------+--------------+------------------+---------------------------------------------------------------------------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+---------------------------------------------------------------------------------------+
| mysql-bin.000016 | 843 | | | 0a99e3f9-3a95-11ea-87df-000c29b6e0ec:1-12,
979de5b8-3c01-11ea-8252-000c29b6e0ec:1-335 |
+------------------+----------+--------------+------------------+---------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
2、DDL/DML/DCL语句都会被记录到GTID内,每一个操作都会有一个全局事务号,如果是DML事务,从begin到commit会生产一个事务号。
3306[gtid]>show binlog events in 'mysql-bin.000016';
+------------------+-----+----------------+-----------+-------------+------------------------------------------------------------------------------------------------------------------------------------------------+
| Log_name | Pos | Event_type | Server_id | End_log_pos | Info |
+------------------+-----+----------------+-----------+-------------+------------------------------------------------------------------------------------------------------------------------------------------------+
| mysql-bin.000016 | 4 | Format_desc | 7 | 123 | Server ver: 5.7.20-log, Binlog ver: 4 |
| mysql-bin.000016 | 123 | Previous_gtids | 7 | 234 | 0a99e3f9-3a95-11ea-87df-000c29b6e0ec:1-12,
979de5b8-3c01-11ea-8252-000c29b6e0ec:1-332 |
| mysql-bin.000016 | 234 | Gtid | 7 | 299 | SET @@SESSION.GTID_NEXT= '979de5b8-3c01-11ea-8252-000c29b6e0ec:333' |
| mysql-bin.000016 | 299 | Query | 7 | 409 | create database gtid charset utf8mb4 |
| mysql-bin.000016 | 409 | Gtid | 7 | 474 | SET @@SESSION.GTID_NEXT= '979de5b8-3c01-11ea-8252-000c29b6e0ec:334' |
| mysql-bin.000016 | 474 | Query | 7 | 571 | use `gtid`; create table t(id int ) |
| mysql-bin.000016 | 571 | Gtid | 7 | 636 | SET @@SESSION.GTID_NEXT= '979de5b8-3c01-11ea-8252-000c29b6e0ec:335' |
| mysql-bin.000016 | 636 | Query | 7 | 708 | BEGIN |
| mysql-bin.000016 | 708 | Table_map | 7 | 752 | table_id: 230 (gtid.t) |
| mysql-bin.000016 | 752 | Write_rows | 7 | 812 | table_id: 230 flags: STMT_END_F |
| mysql-bin.000016 | 812 | Xid | 7 | 843 | COMMIT /* xid=82 */ |
| mysql-bin.000016 | 843 | Gtid | 7 | 908 | SET @@SESSION.GTID_NEXT= '979de5b8-3c01-11ea-8252-000c29b6e0ec:336' |
| mysql-bin.000016 | 908 | Query | 7 | 1129 | use `gtid`; GRANT SELECT ON *.* TO 'hdfeng'@'localhost' IDENTIFIED WITH 'mysql_native_password' AS '*23AE809DDACAF96AF0FD78ED04B6A265E05AA257' |
+------------------+-----+----------------+-----------+-------------+------------------------------------------------------------------------------------------------------------------------------------------------+
13 rows in set (0.00 sec)
3306[gtid]>drop database gtid;
Query OK, 1 row affected (0.02 sec)
模拟数据库被删除的过程
使用GTID查看binlog以及GTID恢复日志
查看及截取命令:
--include-gtids 值为需要恢复的gtid事务号范围
--exclude-gtids 值为需要路过的gtid事务号范围
当需要跳过多个号码的时候可以用逗号格开
mysqlbinlog --include-gtids='979de5b8-3c01-11ea-8252-000c29b6e0ec:333-336' --exclude-gtids='979de5b8-3c01-11ea-8252-000c29b6e0ec:xxx' /opt/mysql-data/binlog/mysql-bin.000016
因为中间没有其它事务,所以不需要跳过事务ID
[root@localhost ~]# mysqlbinlog --include-gtids='979de5b8-3c01-11ea-8252-000c29b6e0ec:333-336' /opt/mysql-data/binlog/mysql-bin.000016
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at 4
#200224 18:44:10 server id 7 end_log_pos 123 CRC32 0x8dc43445 Start: binlog v 4, server v 5.7.20-log created 200224 18:44:10
# Warning: this binlog is either in use or was not closed properly.
BINLOG '
+qhTXg8HAAAAdwAAAHsAAAABAAQANS43LjIwLWxvZwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAEzgNAAgAEgAEBAQEEgAAXwAEGggAAAAICAgCAAAACgoKKioAEjQA
AUU0xI0=
'/*!*/;
# at 123
#200224 18:44:10 server id 7 end_log_pos 234 CRC32 0x2bd7914a Previous-GTIDs
# 0a99e3f9-3a95-11ea-87df-000c29b6e0ec:1-12,
# 979de5b8-3c01-11ea-8252-000c29b6e0ec:1-332
# at 234
#200225 10:29:32 server id 7 end_log_pos 299 CRC32 0xa6a3bfb1 GTID last_committed=0 sequence_number=1 rbr_only=no
SET @@SESSION.GTID_NEXT= '979de5b8-3c01-11ea-8252-000c29b6e0ec:333'/*!*/;
# at 299
#200225 10:29:32 server id 7 end_log_pos 409 CRC32 0x418618ab Query thread_id=7 exec_time=0 error_code=0
SET TIMESTAMP=1582597772/*!*/;
SET @@session.pseudo_thread_id=7/*!*/;
SET @@session.foreign_key_checks=1, @@session.sql_auto_is_null=0, @@session.unique_checks=1, @@session.autocommit=1/*!*/;
SET @@session.sql_mode=1436549152/*!*/;
SET @@session.auto_increment_increment=1, @@session.auto_increment_offset=1/*!*/;
/*!\C utf8 *//*!*/;
SET @@session.character_set_client=33,@@session.collation_connection=33,@@session.collation_server=45/*!*/;
SET @@session.lc_time_names=0/*!*/;
SET @@session.collation_database=DEFAULT/*!*/;
create database gtid charset utf8mb4
/*!*/;
# at 409
#200225 10:34:52 server id 7 end_log_pos 474 CRC32 0xee989acd GTID last_committed=1 sequence_number=2 rbr_only=no
SET @@SESSION.GTID_NEXT= '979de5b8-3c01-11ea-8252-000c29b6e0ec:334'/*!*/;
# at 474
#200225 10:34:52 server id 7 end_log_pos 571 CRC32 0x70133894 Query thread_id=7 exec_time=1 error_code=0
use `gtid`/*!*/;
SET TIMESTAMP=1582598092/*!*/;
create table t(id int )
/*!*/;
# at 571
#200225 10:36:57 server id 7 end_log_pos 636 CRC32 0x1b423a86 GTID last_committed=2 sequence_number=3 rbr_only=yes
/*!50718 SET TRANSACTION ISOLATION LEVEL READ COMMITTED*//*!*/;
SET @@SESSION.GTID_NEXT= '979de5b8-3c01-11ea-8252-000c29b6e0ec:335'/*!*/;
# at 636
#200225 10:36:54 server id 7 end_log_pos 708 CRC32 0xee84dc06 Query thread_id=7 exec_time=0 error_code=0
SET TIMESTAMP=1582598214/*!*/;
BEGIN
/*!*/;
# at 708
#200225 10:36:54 server id 7 end_log_pos 752 CRC32 0x97603303 Table_map: `gtid`.`t` mapped to number 230
# at 752
#200225 10:36:54 server id 7 end_log_pos 812 CRC32 0x5d3de002 Write_rows: table id 230 flags: STMT_END_F
BINLOG '
RohUXhMHAAAALAAAAPACAAAAAOYAAAAAAAEABGd0aWQAAXQAAQMAAQMzYJc=
RohUXh4HAAAAPAAAACwDAAAAAOYAAAAAAAEAAgAB//4BAAAA/gIAAAD+AwAAAP4EAAAA/gUAAAAC
4D1d
'/*!*/;
# at 812
#200225 10:36:57 server id 7 end_log_pos 843 CRC32 0x2804d8c9 Xid = 82
COMMIT/*!*/;
# at 843
#200225 10:44:24 server id 7 end_log_pos 908 CRC32 0x54ab549a GTID last_committed=3 sequence_number=4 rbr_only=no
SET @@SESSION.GTID_NEXT= '979de5b8-3c01-11ea-8252-000c29b6e0ec:336'/*!*/;
# at 908
#200225 10:44:24 server id 7 end_log_pos 1129 CRC32 0xda23c80a Query thread_id=7 exec_time=0 error_code=0
SET TIMESTAMP=1582598664/*!*/;
GRANT SELECT ON *.* TO 'hdfeng'@'localhost' IDENTIFIED WITH 'mysql_native_password' AS '*23AE809DDACAF96AF0FD78ED04B6A265E05AA257'
/*!*/;
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
DELIMITER ;
# End of log file
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;
---------------------------------------------------------------------------------------------------------------
进行日志导出恢复:
[root@localhost ~]# mysqlbinlog --include-gtids='979de5b8-3c01-11ea-8252-000c29b6e0ec:333-336' /opt/mysql-data/binlog/mysql-bin.000016 >/tmp/gtid.sql
---------------------------------------------------------------------------------------------------------------
3306[(none)]>set sql_log_bin=0;
Query OK, 0 rows affected (0.00 sec)
3306[(none)]> source /tmp/gtid1.sql;
ERROR 1049 (42000): Unknown database 'gtid'
Query OK, 0 rows affected (0.00 sec)
ERROR 1046 (3D000): No database selected
Query OK, 0 rows affected (0.00 sec)
发现报错,这是为啥?这个是因为GTID有幂等性的原因。
GTID的幂等性
什么又叫GTID的幂等性呢?这个就是开启了GTID之后,恢复binlog的时候,MySQL的重复GTID不读取的操作。数据库一旦发现有相同的GTID号,就会直接跳过不执行,这也是为了保证主从复制一致性。
需要加参数:
--skip-gtids 跳过重复的GTID号
[root@localhost ~]# mysqlbinlog --skip-gtids --include-gtids='979de5b8-3c01-11ea-8252-000c29b6e0ec:333-336' /opt/mysql-data/binlog/mysql-bin.000016 >/tmp/gtid1.sql
加入以后再进行恢复就不会报错了
3306[(none)]>source /tmp/gtid1.sql;
可以查看与原来截取出来的日志进行对比:
[root@hdfeng ~]# vimdiff /tmp/gtid.sql /tmp/gtid1.sql
本文深入解析MySQL中的查询日志和二进制日志(binlog),包括日志的作用、配置参数、事件类型及其组成,以及如何使用binlog进行数据恢复,特别关注GTID模式下的日志管理和恢复策略。
&spm=1001.2101.3001.5002&articleId=104445591&d=1&t=3&u=b158fa185cf44dba9132c752b31c0291)
617

被折叠的 条评论
为什么被折叠?



