Home ยป How can I backup multi-gigabyte databases in MySQL? Bonus points for speedier restores

How can I backup multi-gigabyte databases in MySQL? Bonus points for speedier restores

Solutons:


We have around 90 GB in a MySQL, so here are some suggestions:

  • You can try the following call: mysqldump -u USER -pPASS --single-transaction -Q --databases foo bar (Use the docs to check if those switches work for you.)
  • If you use MYISAM-tables only, then copying the database to another drive and tar’ing it, might also be an alternative. This will not work with INNODB though, since it keeps an additional file. Also, you have to test this carefully.
  • Enable binary logs, and back them up. (This is probably my favorite!)
  • Setup replication and to do the backup on one of your slaves, and then let it catch up with the rest. It’s pretty nice when it runs, but the problem is getting it running. Replicating is no fun with MySQL. ๐Ÿ™
  • Does your filesystem support snapshots? If it does, then that is a great opportunity to use them.
  • Use a commercial solution, such as Amanda.

The best way would be without interrupting normal operation. For high-reliability, fault-tolerant systems, you have 2 DBs that are kept in sync, and you backup the secondary. (eg this howtoforge article)

Otherwise, read the manual: use myseldump as you currently do, or use the mysqlhotcopy script (use just like mysqldump), or stop the DB and simply copy the frm, MID, MYI files (using rsync).

I think the stop+copy files mechanism is the fastest you’re going to get.

If some of your databases are read-only or modified rarely single it out as a less frequent cron.

Related Solutions

Don’t understand how my mum’s Gmail account was hacked

IMPORTANT: this is based on data I got from your link, but the server might implement some protection. For example, once it has sent its "silver bullet" against a victim, it might answer with a faked "silver bullet" to the same request, so that anyone...

What is /storage/emulated/0/?

/storage/emulated/0/Download is the actual path to the files. /sdcard/Download is a symlink to the actual path of /storage/emulated/0/Download However, the actual files are located in the filesystem in /data/media, which is then mounted to /storage/emulated/0...

How can I pass a command line argument into a shell script?

The shell command and any arguments to that command appear as numbered shell variables: $0 has the string value of the command itself, something like script, ./script, /home/user/bin/script or whatever. Any arguments appear as "$1", "$2", "$3" and so on. The...

What is pointer to string in C?

argv is an array of pointers pointing to zero terminated c-strings. I painted the following pretty picture to help you visualize something about the pointers. And here is a code example that shows you how an operating system would pass arguments to your...

How do mobile carriers know video resolution over HTTPS connections?

This is an active area of research. I happen to have done some work in this area, so I'll share what I can about the basic idea (this work was with industry partners and I can't share the secret details ๐Ÿ™‚ ). The tl;dr is that it's often possible to identify an...

How do I change the name of my Android device?

To change the hostname (device name) you have to use the terminal (as root): For Eclair (2.1): echo MYNAME > /proc/sys/kernel/hostname For Froyo (2.2): (works also on most 2.3) setprop net.hostname MYNAME Then restart your wi-fi. To see the change, type...

How does reverse SSH tunneling work?

I love explaining this kind of thing through visualization. ๐Ÿ™‚ Think of your SSH connections as tubes. Big tubes. Normally, you'll reach through these tubes to run a shell on a remote computer. The shell runs in a virtual terminal (tty). But you know this part...

Difference between database vs user vs schema

In Oracle, users and schemas are essentially the same thing. You can consider that a user is the account you use to connect to a database, and a schema is the set of objects (tables, views, etc.) that belong to that account. See this post on Stack Overflow:...

What’s the output of this code written in java?

//if you're using Eclipse, press ctrl-shift-f to "beautify" your code and make it easier to read int arr[] = new int[3]; //create a new array containing 3 elements for (int i = 0; i < 3; i++) { arr[i] = i;//assign each successive value of i to an entry in...

How safe are password managers like LastPass?

We should distinguish between offline password managers (like Password Safe) and online password managers (like LastPass). Offline password managers carry relatively little risk. It is true that the saved passwords are a single point of failure. But then, your...

Can anyone tell me why this program go to infinite times?

while (i <= 2) { while (i > 0) { a = a + b; i--; <- out the inner while loop when i = 0 } printf("%d", a); i++; <- at here, the i==0 each time, so infinity loop } Because your nested loop always restores the value of i to 0, And 0 <= 2 is always...

How to conditionally do something if a command succeeded or failed

How to conditionally do something if a command succeeded or failed That's exactly what bash's if statement does: if command ; then echo "Command succeeded" else echo "Command failed" fi Adding information from comments: you don't need to use the [ ... ] syntax...

How to turn JSON array into Postgres array?

Postgres 9.4 or newer Obviously inspired by this post, Postgres 9.4 added the missing function(s): Thanks to Laurence Rowe for the patch and Andrew Dunstan for committing! json_array_elements_text(json) jsonb_array_elements_text(jsonb) To unnest the JSON array....

Implementing a 2D destructible landscape (like Worms)

I don't know how the landscape in worms was implemented exactly, but I'm pretty sure they used a bitmap for the landscape (at least in the older games of the series). A very basic approach would be a bitmap image (B/W) where black pixels represent air and white...

Huge procedurally generated ‘wilderness’ worlds

I think I better understand what you are asking now. Noise is not random - it's random-looking but is completely based on a mathematical formula and is repeatable. All the information is encoded in the formula. This means that you can have a formula that...

How does one prevent homing missiles from orbiting their targets?

First of all, you should make all calculations about what acceleration to apply in the missile's frame of reference (that's where the missile is stationary and everything else moves around it, also often called "object coordinates" or "local coordinates" in...