Tuesday, January 6, 2009

Dumping Table Structure and Data

mysqldump

Utility to dump a database or a collection of database for backup or for transferring the data to another SQL server (not necessarily a MySQL server). The dump will contain SQL statements to create the table and/or populate the table.

If you are doing a backup on the server, you should consider using the mysqlhotcopy instead. See section 4.8.6 mysqlhotcopy, Copying MySQL Databases and Tables.

shell> mysqldump [OPTIONS] database [tables]
OR mysqldump [OPTIONS] --databases [OPTIONS] DB1 [DB2 DB3...]
OR mysqldump [OPTIONS] --all-databases [OPTIONS]

If you don't give any tables or use the --databases or --all-databases, the whole database(s) will be dumped.

You can get a list of the options your version of mysqldump supports by executing mysqldump --help.

Note that if you run mysqldump without --quick or --opt, mysqldump will load the whole result set into memory before dumping the result. This will probably be a problem if you are dumping a big database.

Note that if you are using a new copy of the mysqldump program and you are going to do a dump that will be read into a very old MySQL server, you should not use the --opt or -e options.

mysqldump supports the following options:

--add-locks
Add LOCK TABLES before and UNLOCK TABLE after each table dump. (To get faster inserts into MySQL.)
--add-drop-table
Add a drop table before each create statement.
-A, --all-databases
Dump all the databases. This will be same as --databases with all databases selected.
-a, --all
Include all MySQL-specific create options.
--allow-keywords
Allow creation of column names that are keywords. This works by prefixing each column name with the table name.
-c, --complete-insert
Use complete insert statements (with column names).
-C, --compress
Compress all information between the client and the server if both support compression.
-B, --databases
To dump several databases. Note the difference in usage. In this case no tables are given. All name arguments are regarded as database names. USE db_name; will be included in the output before each new database.
--delayed
Insert rows with the INSERT DELAYED command.
-e, --extended-insert
Use the new multiline INSERT syntax. (Gives more compact and faster inserts statements.)
-#, --debug[=option_string]
Trace usage of the program (for debugging).
--help
Display a help message and exit.
--fields-terminated-by=...

--fields-enclosed-by=...

--fields-optionally-enclosed-by=...

--fields-escaped-by=...

--lines-terminated-by=...
These options are used with the -T option and have the same meaning as the corresponding clauses for LOAD DATA INFILE. See section 6.4.9 LOAD DATA INFILE Syntax.
-F, --flush-logs
Flush log file in the MySQL server before starting the dump.
-f, --force,
Continue even if we get a SQL error during a table dump.
-h, --host=..
Dump data from the MySQL server on the named host. The default host is localhost.
-l, --lock-tables.
Lock all tables before starting the dump. The tables are locked with READ LOCAL to allow concurrent inserts in the case of MyISAM tables. Please note that when dumping multiple databases, --lock-tables will lock tables for each database separately. So using this option will not guarantee your tables will be logically consistent between databases. Tables in different databases may be dumped in completely different states.
-K, --disable-keys
/*!40000 ALTER TABLE tb_name DISABLE KEYS */; and /*!40000 ALTER TABLE tb_name ENABLE KEYS */; will be put in the output. This will make loading the data into a MySQL 4.0 server faster as the indexes are created after all data are inserted.
-n, --no-create-db
CREATE DATABASE /*!32312 IF NOT EXISTS*/ db_name; will not be put in the output. The above line will be added otherwise, if a --databases or --all-databases option was given.
-t, --no-create-info
Don't write table creation information (the CREATE TABLE statement).
-d, --no-data
Don't write any row information for the table. This is very useful if you just want to get a dump of the structure for a table!
--opt
Same as --quick --add-drop-table --add-locks --extended-insert --lock-tables. Should give you the fastest possible dump for reading into a MySQL server.
-pyour_pass, --password[=your_pass]
The password to use when connecting to the server. If you specify no `=your_pass' part, mysqldump you will be prompted for a password.
-P, --port=...
Port number to use for TCP/IP connections.
--protocol=(TCP | SOCKET | PIPE | MEMORY)
To specify the connect protocol to use. New in MySQL 4.1.
-q, --quick
Don't buffer query, dump directly to stdout. Uses mysql_use_result() to do this.
-Q, --quote-names
Quote table and column names within ``' characters.
-r, --result-file=...
Direct output to a given file. This option should be used in MSDOS, because it prevents new line `\n' from being converted to `\n\r' (new line + carriage return).
--single-transaction
This option issues a BEGIN SQL command before dumping data from server. It is mostly useful with InnoDB tables and READ_COMMITTED transaction isolation level, as in this mode it will dump the consistent state of the database at the time then BEGIN was issued without blocking any applications. When using this option you should keep in mind that only transactional tables will be dumped in a consistent state, e.g., any MyISAM or HEAP tables dumped while using this option may still change state. The --single-transaction option was added in version 4.0.2. This option is mutually exclusive with the --lock-tables option as LOCK TABLES already commits a previous transaction internally.
-S /path/to/socket, --socket=/path/to/socket
The socket file to use when connecting to localhost (which is the default host).
--tables
Overrides option --databases (-B).
-T, --tab=path-to-some-directory
Creates a table_name.sql file, that contains the SQL CREATE commands, and a table_name.txt file, that contains the data, for each give table. The format of the `.txt' file is made according to the --fields-xxx and --lines--xxx options. Note: This option only works if mysqldump is run on the same machine as the mysqld daemon, and the user/group that mysqld is running as (normally user mysql, group mysql) needs to have permission to create/write a file at the location you specify.
-u user_name, --user=user_name
The MySQL user name to use when connecting to the server. The default value is your Unix login name.
-O var=option, --set-variable var=option
Set the value of a variable. The possible variables are listed below. Please note that --set-variable is deprecated since MySQL 4.0, just use --var=option on its own.
-v, --verbose
Verbose mode. Print out more information on what the program does.
-V, --version
Print version information and exit.
-w, --where='where-condition'
Dump only selected records. Note that quotes are mandatory:
"--where=user='jimf'" "-wuserid>1" "-wuserid<1"
-X, --xml
Dumps a database as well formed XML
-x, --first-slave
Locks all tables across all databases.
--master-data
Like --first-slave, but also prints some CHANGE MASTER TO commands which will later make your slave start from the right position in the master's binlogs, if you have set up your slave using this SQL dump of the master.
-O net_buffer_length=#, where # <>
When creating multi-row-insert statements (as with option --extended-insert or --opt), mysqldump will create rows up to net_buffer_length length. If you increase this variable, you should also ensure that the max_allowed_packet variable in the MySQL server is bigger than the net_buffer_length.

The most normal use of mysqldump is probably for making a backup of whole databases. See section 4.4.1 Database Backups.

mysqldump --opt database > backup-file.sql

You can read this back into MySQL with:

mysql database < backup-file.sql

or

mysql -e "source /patch-to-backup/backup-file.sql" database

However, it's also very useful to populate another MySQL server with information from a database:

mysqldump --opt database | mysql --host=remote-host -C database

It is possible to dump several databases with one command:

mysqldump --databases database1 [database2 ...] > my_databases.sql

If all the databases are wanted, one can use:

mysqldump --all-databases > all_databases.sql

Selectively dumping data with mysqldump

mysqldump is a command line tool for outputting table structures and data and can be used for backups etc. By default mysqldump will dump all data from a table, but it is possible to select which data to be exported with mysqldump. This post looks at how to do this.

The examples in this post have a table called "mytable" in a database called "test". mytable has three columns: mytable_id, category_id and name, and we will be selectively exporting data that matches a specific category_id.

Using mysqldump to dump all data from the table would look like this, subsituting [username] for your username (the -t flag suppresses the table creation sql from the dump):

mysqldump -t -u [username] -p test mytable

The output from my example table looks like this, once we remove all the extra SQL commands (I've added linebreaks to make it more legible):

INSERT INTO `mytable` VALUES
(1,1,'Lorem ipsum dolor sit amet'),
(2,1,'Ut purus est'),
(3,2,'Leo sed condimentum semper'),
(4,2,'Donec velit neque'),
(5,3,'Maecenas ullamcorper');

If we only wanted to dump data from mytable in category_id 1, we would do this:

mysqldump -t -u [username] -p test mytable --where=category_id=1

which would output this:

INSERT INTO `mytable` VALUES
(1,1,'Lorem ipsum dolor sit amet'),
(2,1,'Ut purus est');

You can also abbreviate --where as -w like so:

mysqldump -t -u [username] -p test mytable -wcategory_id=1

If you need to have spaces in the where query or other special shell characters (such as > and <) then you need to put quotes around the where clause like so:

mysqldump -t -u [username] -p test mytable --where="category_id = 1"
OR
mysqldump -t -u [username] -p test mytable -w"category_id = 1"

You can also use the --where flag to selectively dump data from more than one table, but obviously the columns specified in the where clause need to be in both tables.

An example of dumping data from two tables using the same where clause could look like this, where we are selecting category_id from tables "mytable" and "anothertable":

mysqldump -t -u [username] -p test mytable anothertable --where="category_id = 1"

If category_id exists in both tables then the dump will run without error. If the column doesn't exist, you'll see an error like this:

mysqldump: mysqldump: Couldn't execute 'SELECT /*!40001 SQL_NO_CACHE */ * FROM `anothertable` WHERE category_id=1': Unknown column 'category_id' in 'where clause' (1054)
mysqldump: Got error: 1054: Unknown column 'category_id' in 'where clause' when retrieving data from server

mysqldump is an excellent tool for exporting data from MySQL databases.
Using the --where or -w flags allows you to selectively export data
from one or more tables which saves you having to export all data from
a table if you only need a specific subset.