Thursday, December 2, 2010

MySQL Database Optimization

This is specially tested with Snort IDS Database

Select your database and then issue the command below, where 'mytablename' is the name of the table you want to query
show table status like 'mytablename'\G
You can omit the "like 'mytablename'" part and then it will show this information for all tables.
However if you have a lot of tables and there's only one or two you want to examine then it's better to specify the particular table.
You can end you query with either ; or \G. I prefer \G for this particular query because it shows each column from the resultset on a new line,
whereas ; will show the columns across the screen. This is OK for a resultset with only a few columns with only a
small amount of information in each one, but it's not so good for this query.

The result from the above will look something like so:

*************************** 1. row ***************************
Name: mytablename
Engine: MyISAM
Version: 10
Row_format: Dynamic
Rows: 2444
Avg_row_length: 7536
Data_length: 564614700
Max_data_length: 281474976710655
Index_length: 7218176
Data_free: 546194608
Auto_increment: 1187455
Create_time: 2008-03-19 10:33:13
Update_time: 2008-09-02 22:18:15
Check_time: 2008-08-27 23:07:48
Collation: latin1_swedish_ci
Checksum: NULL
Create_options: pack_keys=0
Comment:
***************************************************************

The values that are important for working out if the table is non optimal is the "Data_free" value.
If this is high, as in the above example where 564614700 bytes are free (538MB),
then the table has a lot of space not being used and should be optimized.
To optimize the table, issue the following command, where "mytablename" is the name of the MySQL table to optimise:

optimize table mytablename;

After doing this (it may take a few seconds dpending on the size of the table, free space etc) and running "show table status" again, the result should look much better:
*************************** 1. row ***************************
Name: tblmailqueue
Engine: MyISAM
Version: 10
Row_format: Dynamic
Rows: 6145
Avg_row_length: 7505
Data_length: 46119636
Max_data_length: 281474976710655
Index_length: 296960
Data_free: 0
Auto_increment: 1191156
Create_time: 2008-03-19 10:33:13
Update_time: 2008-09-02 22:24:58
Check_time: 2008-09-02 22:21:32
Collation: latin1_swedish_ci
Checksum: NULL
Create_options: pack_keys=0
Comment:
1 row in set (0.00 sec)
***************************************************************
In the above example we can see the "Data_free" value is now zero so the table is nicely optimised.

You can do something like this:
SELECT concat("OPTIMIZE TABLE ", table_schema,".",table_name,";") FROM tables WHERE DATA_FREE > 0 INTO OUTFILE '/tmp/optimize.sql'; SOURCE '/tmp/optimize.sql';

SQL SERVER DATABASE MAINTENANACE - BEST PRACTICES BY MICROSOFT

Data-file (MDF and LDF) Maintenance

Transaction Log grows unexpectedly or becomes full on a SQL Server >> http://support.microsoft.com/?id=317375
How to move SQL Server databases to a new location http://support.microsoft.com/kb/224071
How to move databases between computers that are running SQL Server http://support.microsoft.com/kb/314546

DATABASE MaintenanceDatabase Maintenance >> http://technet.microsoft.com/en-us/magazine/2008.08.database.aspx
How to Minimize Deadlocks in SQL Server >> http://msdn.microsoft.com/en-us/library/ms191242(SQL.90).aspx
How to Minimize Blocking in SQL Server >> http://technet.microsoft.com/en-us/magazine/2008.04.blocking.aspx

SQL SERVER TRANSACTION LOG MAINTENANACE - BEST PRACTICES

A transaction log grows unexpectedly or becomes full on a computer that is running SQL Server http://support.microsoft.com/kb/317375
How to stop the transaction log of a SQL Server database from growing unexpectedly http://support.microsoft.com/kb/873235
Shrinking a Database http://msdn.microsoft.com/en-us/library/aa933076(SQL.80).aspx

MS SQL Database Full Recovery from the TRANSACTION LOGS ( Tested with MsSQL - 2005, 2008 )

MsSQL Database full Recovery from transaction logs

Spl.Note :-
I am assuming that your Database recovery mode is set to FULL and not the SIMPLE.
You have a database which is set to FULL recovery and u have a full Database backup on say Dec 2009 after
that u have regularly backed up its transactional logs and suddenly your database crashed or by any chance it is being altered.,

in this scenario u need to get the changes made to your database on a previous day.

in this conditions you must be having database complete backup which is taken on Dec 2009 or any latest full database backup and transactional logs till today, in our need we assume that the database restore requirement is till yesterday and not today as i discussed above.

Now its time to start the Restore Database operations.

Step:-1
RESTORE DATABASE YOUR_DATABASE_NAME
FROM DISK = 'D:\YOUR_DATABASE_PATH\YOUR_FULL_DATABASE_BACKUP.bak'
WITH NORECOVERY

The above command will results in following output

Processed 184 pages for database 'YOUR_FULL_DATABASE_BACKUP', file 'YOUR_FULL_DATABASE_BACKUP' on file 1.
Processed 6 pages for database 'YOUR_FULL_DATABASE_BACKUP', file 'YOUR_FULL_DATABASE_BACKUP_log' on file 1.
RESTORE DATABASE successfully processed 190 pages in 0.205 seconds (7.233 MB/sec).

Once you restore the full backup using the NORECOVERY option, you can begin applying the transaction log backups or the differential backup as given below.

Step:-2

RESTORE LOG YOUR_DATABASE_NAME
FROM DISK = 'D:\YOUR_DATABASE_PATH\Transaction_Logs_1.trn' WITH NORECOVERY

The above command will results in following output
Processed 0 pages for database 'YOUR_DATABASE_NAME', file 'YOUR_DATABASE_NAME' on file 1.
Processed 7 pages for database 'YOUR_DATABASE_NAME', file 'YOUR_DATABASE_NAME_log' on file 1.
RESTORE LOG successfully processed 7 pages in 0.017 seconds (2.900 MB/sec).

Step:-3
RESTORE LOG YOUR_DATABASE_NAME
FROM DISK = 'D:\YOUR_DATABASE_PATH\Transaction_Logs_2.trn' WITH NORECOVERY

The above command will results in following output
Processed 0 pages for database 'YOUR_DATABASE_NAME', file 'YOUR_DATABASE_NAME' on file 1.
Processed 3 pages for database 'YOUR_DATABASE_NAME', file 'YOUR_DATABASE_NAME_log' on file 1.
RESTORE LOG successfully processed 3 pages in 0.018 seconds (1.247 MB/sec).

Step:-4

i m assuming Transaction_Logs_3.trn is the file which we want to get the database data till yesterday,

one more file Transaction_Logs_4.trn is there but our requirement is till file Transaction_Logs_3.trn only

RESTORE LOG YOUR_DATABASE_NAME
FROM DISK = 'D:\YOUR_DATABASE_PATH\Transaction_Logs_3.trn' WITH RECOVERY

The above command will results in following output

Processed 0 pages for database 'YOUR_DATABASE_NAME', file 'YOUR_DATABASE_NAME' on file 1.
Processed 2 pages for database 'YOUR_DATABASE_NAME', file 'YOUR_DATABASE_NAME_log' on file 1.

RESTORE LOG successfully processed 2 pages in 0.064 seconds (0.190 MB/sec).

after the successfull operations of the above step the database is now ready for use,

till the time it is in RECOVERYMODE the same will not be available for use.

In the example above, we restore the database to the end of the 2nd last transaction log.

If we want to recover our database to a specific point in time before the end of that transaction log,
then we must use the STOPAT option.

The script below restores the fourth transaction logs in the log sequence to 3:00 AM - time just before the database gets currupted.

RESTORE LOG YOUR_DATABASE_NAME
FROM DISK = 'D:\YOUR_DATABASE_PATH\Transaction_Logs_4.trn'
WITH STOPAT = N'8/30/2010 3:00:00 AM', RECOVERY