Home » Amazon EC2 Data Persistence

Amazon EC2 Data Persistence

Solutons:


Like others have said, EBS–Elastic Block Storage. I am using it myself now that it is released to the general public. It is better than S3 on multiple points:

  • EBS are fast. Faster than even the local mounts, according to Amazon.
  • EBS mounts as proper devices. Unlike S3, which you’ll need custom S3 oject access logic in your code, or middleware (JungleDisk, ElasticDisk, et al) which present their own problems and costs
  • EBS are easy to back up. Amazon give one the ability to take snap shots, which are saved on S3
  • EBS are portable between instances–volumes can be unmounted from one instance, and attached to another instance
  • EBS devices can even be RAID’ed together for improved reliability

My experience with EBS so far has been the most positive thing about AWS I’ve dealt with to date.


Update: While my experience with EBS has been positive, others have had issues. Very specifically EBS do not implement fsync() correctly. Ted Dziuba has some interesting words about this in his blog post Amazon — The Purpose of Pain: Myth 2: Architecture Will Save You from Cloud Failures

This gets even more entertaining with Amazon Elastic Block Store,
which, as the Reddit administrators have found, will happily accept
calls to fsync(), and lie to your face, saying that the data has been
written to disk, when it may not have been.

EBS would certainly work for a database, and is one of the examples in Amazon’s EBS Description. “Amazon EBS is particularly suited for applications that require a database…”

EBS will work just like a block device (think hard disk), giving you broad freedom and comfortability using it. S3 is conceptually more like really fast FTP, with a special API. You could conceivably use it as part of a database, but standard databases don’t run on it (yet?).

You will want to review Amazon’s descriptions of performance (EBS > S3), durability (S3 > EBS) and price (depends).

As mentioned in other answers, EBS is the standard solution for persistent and convenient disk storage. It should be your default option. Indeed, the newer EC2 instance types use it by default, over the original, non-persisted instance storage.

However, when considering persistence, you’ll also want to carefully consider availability (avoiding times when the data is not available) and durability (avoiding loss of data) for your data.

EBS covers the basic use cases, but keep in mind:

  • S3 is designed for higher durability than EBS. Simply put, they keep more copies of your data, and claim extremely high 99.999999999% durability for S3 (see the S3 FAQ). The actual numeric probability here (which is so high it makes martian invasions look more probable) is not as important as the fact that AWS has staked their reputation on S3 durability and has a very good record here. The same is not true for EBS.
  • While Amazon will not give statistics on this directly, many people believe instance storage has historically offered higher availability than EBS.

Recent AWS outages, such a severe multi-day failure in 2011 and another in 2012, give some illustrations of the complexities of EBS and the small but non-negligible risks of outages and data loss.

Bottom line: To be sure you won’t lose your data, keep data backups in S3. EBS snapshots are an easy way to do this for EBS. If high availability is critical, consider also using instance storage in multiple availability zones (in addition to, or instead of, EBS).

Related Solutions

Pin-board effect with CSS [closed]

You can use JavaScript to accomplish this but it can't be done with CSS floats alone. A library like jQuery masonry will do it well. The reason? The specs on floats. In particular #5 which says, "The outer top of a floating box may not be higher than the outer...

Calculate the sum with minimum usage of numbers

Here's a hint: 23 : 11 + 11+ 1 ( 3 magic numbers) 120: 110+ 10 (2 magic numbers) The highest digit in the target number is the answer, since you need exactly k magic numbers (all having 1 in the relevant position) in order for the sum to contain the digit k. So...

Why not drop the “auto” keyword? [duplicate]

Your proposal would be rejected on the basis of backward compatibility alone. But let's say for the sake of argument that the standards committee like your idea. You don't take into account the numerous ways you can initialize a variable widget w; // (a) widget...

Recursive to iterative using a systematic method [closed]

So, to restate the question. We have a function f, in our case fac. def fac(n): if n==0: return 1 else: return n*fac(n-1) It is implemented recursively. We want to implement a function facOpt that does the same thing but iteratively. fac is written almost in...

How can I match values in one file to ranges from another?

if the data file sizes are not huge, there is a simpler way $ join input1 input2 | awk '$5<$4 && $3<$5 {print $2, $5-$3+1}' B100002 32 B100043 15 B123465 3 This Perl code seems to solve your problem It is a common idiom: to load the entire...

Javascript difference between “=” and “===” [duplicate]

You need to use == or === for equality checking. = is the assignment operator. You can read about assignment operators here on MDN. As a quick reference as you are learning JS: = assignment operator == equal to === equal value and equal type != not equal !==...

Compiler complains about misplaced else [closed]

Your compiler complains about an misplaced else because, well, there is an else without a preceding if: // ... for (j=1; j<n-i; j++) { if(a[j]<=a[j+1]) { // ... } // END OF IF } // END OF FOR else { continue; } // ... The else in your code does not follow...

Bootstrap – custom alerts with progress bar

/* !important are just used to overide the bootstrap css in the snippet */ .alertContainer { border-radius: 0 !important; border-width: 0 !important; padding: 0 !important; height: auto !important; position: absolute !important; bottom: 15px !important; left:...

How to Garbage Collect an external Javascript load?

Yes, s.onload = null is useful and will garbage collect! As of 2019, it is not possible to explicitly or programmatically trigger garbage collection in JavaScript. That means it collects when it wants. Although there is cases where setting to null may do a GC...

Math programming with python

At first, what you are looking for is the modulo operator and the function math.floor() Modulo from wikipedia: In computing, the modulo operation finds the remainder after division of one number by another (sometimes called modulus). for example: 12%12=0...

Android slide over letters to create a word [closed]

Here some advice you can use: First for each cell you can create an object that represents the state of that cell: class Cell { char mChar; int row,column; boolean isSelected; } then you can create a 2D array of your cells Cell[][] mTable = ... For views you...

Sum two integers in Java

You reused the x and y variable names (hence the variable x is already defined in method main error), and forgot to assign the ints read from the Scanner to the x and y variables. Besides, there's no need to create two Scanner objects. public static void...

Extend three classes that implements an interface in Java

Using this simplified implementation of the library, using method() instead of M(): interface IFC { void method(); } class A implements IFC { public void method() { System.out.println("method in A"); }; } As akuzminykh mentions in their comment You'd write a...