Looking at the source it was clear we could make mk-table-checksum run the select min(col), max(col) from table on the read-only slave and use the values to checksum the master.

It was subtle code changes in function:
get_range_statistics adding

my $cxn_string_dc = “DBI:mysql:;host=slavehost;port=3306;mysql_read_default_group=client”;
my $user = ‘user’;
my $pass = ‘password’;
my $dbh_slave = DBI->connect($cxn_string_dc, $user, $pass);

and changing $dbh connection to $dbh_slave in the rest of the function where the min,max values were being calculated.

Good luck and enjoy a much less intrusive checksumming on your masters.

Because sharing is caring

A while back I needed to create an archive slave database from a half a terabyte myisam master and had space restrictions. I could not dump the db, load it, then drop keys (archive doesn’t support keys apart from a primary key on one column as of 5.1), alter engine etc (would take even longer than it took either way). So an ugly single liner came to mind and worked nicely too.

mysqldump -uuser -ppassword -h127.0.0.1 -P3306 dbname --master-data=1 | sed 's/ENGINE=MyISAM/ENGINE=archive/g' | grep -v '^ UNIQUE KEY' | grep -v '^ KEY' | perl -p0777i -e 's/,\n^\)/\n\)/mg' | mysql -uuser -ppassword -h127.0.0.1 -P3307 dbname

So what is it doing?
Broken down:
mysqldump -uuser -ppassword -h127.0.0.1 -P3306 dbname --master-data=1 –> extract the database with the master position
sed 's/ENGINE=MyISAM/ENGINE=archive/g' –> change myisam with archive so as to spare an alter table later
grep -v '^ UNIQUE KEY' –> remove unique keys
grep -v '^ KEY' –> remove other indexes
perl -p0777i -e 's/,\n^\)/\n\)/mg' –> after removing the indexes, any commas before the closing bracket need to be removed
mysql -uuser -ppassword -h127.0.0.1 -P3307 dbname –> inserting it directly into the slave

Let me know if you have any other ideas, again, the above is definitely not the neatest way, but it does work.

Because Sharing Is Caring

Scenario part 1 : Migrating a couple thousand stored procedures from database technology X to mysql
Scenario part 2 : Legacy system where the people who wrote it left a long time ago
Scenario part 3 : Developers sure can get real creative and invent all kinds of ways to get data (eg: having a stored proc which formulates a big query using concat after going through a bunch of conditions (fair enough), BUT the parts making up the different queries are stored in a table on a database rather than within the stored proc itself) … talk about KIS – Keep it simple!!
Scenario part 4 : This stored proc references 18 tables, 4 views, and another two stored procedures on 5 databases

Now close your eyes and try to imagine that for a few seconds, nah kidding don’t want you to hurt yourself.

I wonder, who’s gonna cover my health insurance if i go crazy? :)

mysql 02:55:47 DEV > call storedprocblahbla(‘I’,'am’,'going’,'crazy’);
ERROR 1052 (23000): Column ‘state_of_mind’ in field list is ambiguous

Sure thats REALLY REALLY helpful thanks :), you know what, lets just mysqldump -d -R -B db1 db2 db3 db4 db5 > /wherever/you/like and grep for column ‘state_of_mind’.

I love a challenge but facing migration of so many stored procs I’d really love to have something “GOOD” to debug them with.

The solutions I found were:
1. Illatis eclipse plugin for $40
works on linux, mac and windows but does show me a lot of:
“An error has occurred when activating this view
com/illatis/parser/lib/k”
2. mydebugger for $50
works on linux and mac via wine
3. dbForge for MySQL for $50
works only on windows due to .net framework (kinda crap because I `hate` using the former)

any other ideas are very welcome, especially opensource ones but I can’t find anything.

I can always try setting up debugging using GDB but would rather try to avoid that.

Say you have the following date: MAR 16 2008 09:12:51:893AM
SELECT DATE_FORMAT(STR_TO_DATE('MAR 16 2008 09:12:51:893AM','%M %d %Y %h:%i:%s:%f%p'),'%Y%m%d%k%i%s.%f'); --> 2008031691251.893000


What if its PM rather than AM
SELECT DATE_FORMAT(STR_TO_DATE('MAR 16 2008 09:12:51:893PM','%M %d %Y %h:%i:%s:%f%p'),'%Y%m%d%k%i%s.%f'); --> 20080316211251.893000


Ok so this is just simple string manipulation where:
%M is the month name
%d is day number
%Y is the year
%h is the hour
%i is the minute
%s is the second
%f is the microsecond
%p is the period: ante or post meridiem
In the DATE_FORMAT part we se a %k which is in 24hr format in order to loose the period.

A more detailed list is found here

Here is a demo:

mysql Tue Jun 15 12:32:37 2010 > CREATE TABLE test.abc(a DECIMAL(17,3)) ENGINE=MYISAM;
Query OK, 0 rows affected (0.03 sec)

mysql Tue Jun 15 12:32:45 2010 > INSERT INTO abc VALUES ( DATE_FORMAT(STR_TO_DATE('MAR 16 2008 09:12:51:893PM','%M %d %Y %h:%i:%s:%f%p'),'%Y%m%d%k%i%s.%f') );
Query OK, 1 row affected (0.01 sec)

mysql Tue Jun 15 12:32:51 2010 > SELECT * FROM abc;
+--------------------+
| a                  |
+--------------------+
| 20080316211251.893 |
+--------------------+
1 row in set (0.00 sec)

mysql Tue Jun 15 12:32:56 2010 > SELECT TIMESTAMP(a) FROM abc;
+----------------------------+
| TIMESTAMP(a)               |
+----------------------------+
| 2008-03-16 21:12:51.893000 |
+----------------------------+
1 row in set (0.00 sec)

The tool is downloadable from there and anyone can use it for free in accordance to GPLv2.

I wanted to throw out tutorial about how to install it and use it (Note this tutorial is for version securich version 0.1.2):

Steps:
1. Download it,
2. Install it,
3. Create a role named 'role1' having privileges: select insert update
4. Check roles,
5. Check role privileges,
6. Create a first user john@machine.domain.com (granting privileges on a whole database employees apart from one table),
7. Create a second user paul@10.0.0.2 (granting privileges on all tables in world having word Country in them),
8. Create a third user peter@localhost (granting privileges on the database test),
9. Check user privileges for (paul),
10. Update role created above and see changes (add delete to role 1),
11. Update password (for paul) and see changes,
12. Clone user paul to judas,
13. Check user privileges
14. Check user,
15. Rename user judas to james,
16. Revoke privileges from third user disconnecting any existing connections from that user (useful if a security breach is suspected or if you are a security paranoid thus wanting to make sure the person you are blocking out won't have any more access as from that point onwards).

1. Go to www.securich.com downloads page and download the install script
2. Untar the install script and run it using ./securich_install.sh and it'll install everything automatically

dcassar@ubuntu:~/Desktop$ ./securich_install.sh
Enter version number: 0.1.1
Which kind of installation would you like to do?
1. Install from file on disk
2. Download and install (recommended)
Enter choice (default 2):

Installation starting
–2009-06-19 16:27:56– http://www.securich.com/downloads/securich.0.1.1.tar.gz
Resolving www.securich.com… 64.202.163.10
Connecting to www.securich.com|64.202.163.10|:80… connected.
HTTP request sent, awaiting response… 200 OK
Length: 29217 (29K) [application/x-tar]
Saving to: `securich.0.1.1.tar.gz'

100%[=====================================================================================================>] 29,217 64.7K/s in 0.4s

2009-06-19 16:27:59 (64.7 KB/s) – 'securich.0.1.1.tar.gz' saved [29217/29217]

Enter mysql root Password (default ):
Enter mysql Hostname/IP (default 127.0.0.1): localhost
Enter mysql Port (default 3306): 3306
Installation complete

3. #log into mysql
use securich;
call create_update_role('role1','select');
call create_update_role('role1','insert');
call create_update_role('role1','update');
4. call check_roles();
5. call check_role_privileges('role1');
6. call grant_privileges('john' , 'machine.domain.com' , 'employees' , '' , 'alltables' , 'role1' , 'john@domain.com');
call revoke_privileges('john' , 'machine.domain.com' , 'employees' , 'salaries' , 'table' , 'role1' , 'N');
7. call grant_privileges('paul' , '10.0.0.2' , 'world' , '^Country' , 'regexp' , 'role1' , 'paul@domain.com');
8. call grant_privileges('peter' , 'localhost' , 'test' , '' , 'all' , 'role1' , 'peter@domain.com');
9. call check_full_user_entries('paul');
10. call create_update_role('role1','delete');
call check_full_user_entries('paul');
11. call set_password('paul' , '10.0.0.2' , 'password123');
12. call clone_user('paul' , '10.0.0.2' , 'judas' , '10.0.0.2' , 'judas@domain.com');
13. call check_full_user_entries('judas');
14. call check_user_privileges('judas' , '10.0.0.2' , 'world' , 'role1');
15. call rename_user('judas' , 'james' , 'james@domain.com');
16. call create_update_role('role2','execute');
17. call grant_privileges('peter' , 'localhost' , 'securich' , 'my_privileges' , 'storedprocedure' , 'role2' , 'peter@domain.com');

18. #connect to mysql using thirduser peter in another session
show databases;
use securich;
show tables;
call my_privileges('test');
show processlist;

19. call revoke_privileges('peter' , 'localhost' , 'test' , '' , '' , 'role1' , 'Y');

20. #as user peter again from 2nd open instance run
show processlist;

dcassar@ubuntu:~/Desktop$ ./securich_install.sh
Enter version number: 0.1.1
Which kind of installation would you like to do?
1. Install from file on disk
2. Download and install (recommended)
Enter choice (default 2):

Installation starting
–2009-06-19 16:27:56– http://www.securich.com/downloads/securich.0.1.1.tar.gz
Resolving www.securich.com… 64.202.163.10
Connecting to www.securich.com|64.202.163.10|:80… connected.
HTTP request sent, awaiting response… 200 OK
Length: 29217 (29K) [application/x-tar]
Saving to: `securich.0.1.1.tar.gz'

100%[=====================================================================================================>] 29,217 64.7K/s in 0.4s

2009-06-19 16:27:59 (64.7 KB/s) – `securich.0.1.1.tar.gz' saved [29217/29217]

Enter mysql root Password (default ):
Enter mysql Hostname/IP (default 127.0.0.1): localhost
Enter mysql Port (default 3306): 3306
Installation complete
dcassar@ubuntu:~/Desktop$ mysql -u root -p -h 127.0.0.1 -P 3306 Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 429
Server version: 5.1.33 MySQL Community Server (GPL)

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> use securich;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
mysql> call create_update_role('role1','select');
Query OK, 0 rows affected, 5 warnings (0.03 sec)

mysql> call create_update_role('role1','insert');
Query OK, 0 rows affected (0.04 sec)

mysql> call create_update_role('role1','update');
Query OK, 0 rows affected (0.04 sec)

mysql> call check_roles();
+—-+——-+
| ID | ROLE |
+—-+——-+
| 1 | read |
| 2 | write |
| 3 | role1 |
+—-+——-+
3 rows in set (0.00 sec)

Query OK, 0 rows affected (0.00 sec)

mysql> call check_role_privileges('role1');
+———–+
| PRIVILEGE |
+———–+
| INSERT |
| SELECT |
| UPDATE |
+———–+
3 rows in set (0.00 sec)

Query OK, 0 rows affected (0.00 sec)

mysql> call grant_privileges('john' , 'machine.domain.com' , 'employees' , '' , 'alltables' , 'role1' , 'john@domain.com');
+——————————————————————————————–+
| USER_PASSWORD |
+——————————————————————————————–+
| Password for user — john — contactable at — john@domain.com — is — bfcbd8234d9eb44 — |
+——————————————————————————————–+
1 row in set (0.16 sec)

Query OK, 0 rows affected, 1 warning (0.16 sec)

mysql> call revoke_privileges('john' , 'machine.domain.com' , 'employees' , 'salaries' , 'table' , 'role1' , 'N');
Query OK, 0 rows affected (0.09 sec)

mysql> call grant_privileges('paul' , '10.0.0.2' , 'world' , '^Country' , 'regexp' , 'role1' , 'paul@domain.com');
+——————————————————————————————–+
| USER_PASSWORD |
+——————————————————————————————–+
| Password for user — paul — contactable at — paul@domain.com — is — bc4ab08785e1be6 — |
+——————————————————————————————–+
1 row in set (0.06 sec)

Query OK, 0 rows affected, 1 warning (0.06 sec)

mysql> call grant_privileges('peter' , 'localhost' , 'test' , '' , 'all' , 'role1' , 'peter@domain.com');
+——————————————————————————————-+
| USER_PASSWORD |
+——————————————————————————————-+
| Password for user — peter — contactable at — peter@domain.com — is — 7b3b4746d04b — |
+——————————————————————————————-+
1 row in set (0.04 sec)

Query OK, 0 rows affected (0.04 sec)

mysql> call check_full_user_entries('paul');
+———-+———-+————–+—————–+——-+———–+——-+
| USERNAME | HOSTNAME | DATABASENAME | TABLENAME | ROLE | PRIVILEGE | STATE |
+———-+———-+————–+—————–+——-+———–+——-+
| paul | 10.0.0.2 | world | Country | role1 | INSERT | A |
| paul | 10.0.0.2 | world | Country | role1 | SELECT | A |
| paul | 10.0.0.2 | world | Country | role1 | UPDATE | A |
| paul | 10.0.0.2 | world | CountryLanguage | role1 | INSERT | A |
| paul | 10.0.0.2 | world | CountryLanguage | role1 | SELECT | A |
| paul | 10.0.0.2 | world | CountryLanguage | role1 | UPDATE | A |
+———-+———-+————–+—————–+——-+———–+——-+
6 rows in set (0.01 sec)

Query OK, 0 rows affected, 4 warnings (0.01 sec)

mysql> call create_update_role('role1','delete');
Query OK, 0 rows affected (0.09 sec)

mysql> call check_full_user_entries('paul');
+———-+———-+————–+—————–+——-+———–+——-+
| USERNAME | HOSTNAME | DATABASENAME | TABLENAME | ROLE | PRIVILEGE | STATE |
+———-+———-+————–+—————–+——-+———–+——-+
| paul | 10.0.0.2 | world | Country | role1 | DELETE | A |
| paul | 10.0.0.2 | world | Country | role1 | INSERT | A |
| paul | 10.0.0.2 | world | Country | role1 | SELECT | A |
| paul | 10.0.0.2 | world | Country | role1 | UPDATE | A |
| paul | 10.0.0.2 | world | CountryLanguage | role1 | DELETE | A |
| paul | 10.0.0.2 | world | CountryLanguage | role1 | INSERT | A |
| paul | 10.0.0.2 | world | CountryLanguage | role1 | SELECT | A |
| paul | 10.0.0.2 | world | CountryLanguage | role1 | UPDATE | A |
+———-+———-+————–+—————–+——-+———–+——-+
8 rows in set (0.00 sec)

Query OK, 0 rows affected (0.00 sec)

mysql> call set_password('paul' , '10.0.0.2' , 'password123');
Query OK, 1 row affected (0.02 sec)

mysql> call clone_user('paul' , '10.0.0.2' , 'judas' , '10.0.0.2' , 'judas@domain.com');
+———————————————————————————————-+
| USER_PASSWORD |
+———————————————————————————————-+
| Password for user — judas — contactable at — judas@domain.com — is — 70d5b79d80fab04 — |
+———————————————————————————————-+
1 row in set (0.01 sec)

Query OK, 0 rows affected, 1 warning (0.10 sec)

mysql> call check_full_user_entries('judas');
+———-+———-+————–+—————–+——-+———–+——-+
| USERNAME | HOSTNAME | DATABASENAME | TABLENAME | ROLE | PRIVILEGE | STATE |
+———-+———-+————–+—————–+——-+———–+——-+
| judas | 10.0.0.2 | world | Country | role1 | DELETE | A |
| judas | 10.0.0.2 | world | Country | role1 | INSERT | A |
| judas | 10.0.0.2 | world | Country | role1 | SELECT | A |
| judas | 10.0.0.2 | world | Country | role1 | UPDATE | A |
| judas | 10.0.0.2 | world | CountryLanguage | role1 | DELETE | A |
| judas | 10.0.0.2 | world | CountryLanguage | role1 | INSERT | A |
| judas | 10.0.0.2 | world | CountryLanguage | role1 | SELECT | A |
| judas | 10.0.0.2 | world | CountryLanguage | role1 | UPDATE | A |
+———-+———-+————–+—————–+——-+———–+——-+
8 rows in set (0.00 sec)

Query OK, 0 rows affected (0.00 sec)

mysql> call check_user_privileges('judas' , '10.0.0.2' , 'world' , 'role1');
+———–+
| PRIVILEGE |
+———–+
| DELETE |
| INSERT |
| SELECT |
| UPDATE |
+———–+
4 rows in set (0.00 sec)

Query OK, 0 rows affected (0.00 sec)

mysql> call rename_user('judas' , 'james' , 'james@domain.com');
+———————————————————————————————-+
| USER_PASSWORD |
+———————————————————————————————-+
| Password for user — james — contactable at — james@domain.com — is — 85c2fc100d83884 — |
+———————————————————————————————-+
1 row in set (0.02 sec)

Query OK, 0 rows affected, 1 warning (0.11 sec)

mysql> call create_update_role('role2','execute');
Query OK, 0 rows affected (0.09 sec)

mysql> call grant_privileges('peter' , 'localhost' , 'securich' , 'my_privileges' , 'storedprocedure' , 'role2' , 'peter@domain.com');
Query OK, 0 rows affected (0.08 sec)

mysql> call revoke_privileges('peter' , 'localhost' , 'test' , '' , '' , 'role1' , 'Y');
Query OK, 0 rows affected (0.15 sec)

mysql>

dcassar@ubuntu:~/Desktop$ mysql -u peter -p7b3b4746d04b -h 127.0.0.1 -P 3306
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 437
Server version: 5.1.33 MySQL Community Server (GPL)

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> show processlist;
+—–+——-+—————–+——+———+——+——-+——————+
| Id | User | Host | db | Command | Time | State | Info |
+—–+——-+—————–+——+———+——+——-+——————+
| 437 | peter | localhost:49022 | NULL | Query | 0 | NULL | show processlist |
+—–+——-+—————–+——+———+——+——-+——————+
1 row in set (0.00 sec)

mysql> show databases;
+——————–+
| Database |
+——————–+
| information_schema |
| securich |
| test |
+——————–+
3 rows in set (0.00 sec)

mysql> use securich;
Database changed
mysql> show tables;
Empty set (0.00 sec)

**** Note that the only privileges peter has on securich is on the stored procedure 'my_privileges' and definitely no tables

mysql> call my_privileges('test');
+———–+
| PRIVILEGE |
+———–+
| DELETE |
| INSERT |
| SELECT |
| UPDATE |
+———–+
4 rows in set (0.00 sec)

Query OK, 0 rows affected (0.00 sec)

mysql> show processlist;
+—–+——-+—————–+——+———+——+——-+——————+
| Id | User | Host | db | Command | Time | State | Info |
+—–+——-+—————–+——+———+——+——-+——————+
| 437 | peter | localhost:49022 | NULL | Query | 0 | NULL | show processlist |
+—–+——-+—————–+——+———+——+——-+——————+
1 row in set (0.00 sec)

****** – In the meantime the dba revoked rights with terminate live connections from peter@localhost

mysql> show processlist;
ERROR 2006 (HY000): MySQL server has gone away
No connection. Trying to reconnect…
ERROR 1045 (28000): Access denied for user 'peter'@'localhost' (using password: YES)
ERROR:
Can't connect to the server

mysql>

I truly hope you enjoyed this run through. I excuse myself it's a tad too long, but I wished to illustrate some of the cool features of this package.

Cheers,
Darren

PS don't forget to check out Securich HERE