Home » How to send 100,000 emails weekly? [closed]

How to send 100,000 emails weekly? [closed]

Solutons:


Short answer: While it’s technically possible to send 100k e-mails each week yourself, the simplest, easiest and cheapest solution is to outsource this to one of the companies that specialize in it (I did say “cheapest”: there’s no limit to the amount of development time (and therefore money) that you can sink into this when trying to DIY).

Long answer: If you decide that you absolutely want to do this yourself, prepare for a world of hurt (after all, this is e-mail/e-fail we’re talking about). You’ll need:

  • e-mail content that is not spam (otherwise you’ll run into additional major roadblocks on every step, even legal repercussions)
  • in addition, your content should be easy to distinguish from spam – that may be a bit hard to do in some cases (I heard that a certain pharmaceutical company had to all but abandon e-mail, as their brand names are quite common in spams)
  • a configurable SMTP server of your own, one which won’t buckle when you dump 100k e-mails onto it (your ISP’s upstream server won’t be sufficient here and you’ll make the ISP violently unhappy; we used two dedicated boxes)
  • some mail wrapper (e.g. PhpMailer if PHP’s your poison of choice; using PHP’s mail() is horrible enough by itself)
  • your own sender function to run in a loop, create the mails and pass them to the wrapper (note that you may run into PHP’s memory limits if your app has a memory leak; you may need to recycle the sending process periodically, or even better, decouple the “creating e-mails” and “sending e-mails” altogether)

Surprisingly, that was the easy part. The hard part is actually sending it:

  • some servers will ban you when you send too many mails close together, so you need to shuffle and watch your queue (e.g. send one mail to joe@example.com, then three to other domains, only then another to otheraddress@example.com)
  • you need to have correct PTR, SPF, DKIM records
  • handling remote server timeouts, misconfigured DNS records and other network pleasantries
  • handling invalid e-mails (and no, regex is the wrong tool for that)
  • handling unsubscriptions (many legitimate newsletters have been reclassified as spam due to many frustrated users who couldn’t unsubscribe in one step and instead chose to “mark as spam” – the spam filters do learn, esp. with large e-mail providers)
  • handling bounces and rejects (“no such mailbox ojhn@example.com”,”mailbox john@example.com full”)
  • handling blacklisting and removal from blacklists (Sure, you’re not sending spam. Some recipients won’t be so sure – with such large list, it will happen sometimes, no matter what precautions you take. Some people (e.g. your not-so-scrupulous competitors) might even go as far to falsely report your mailings as spam – it does happen. On average, it takes weeks to get yourself removed from a blacklist.)

And to top it off, you’ll have to manage the legal part of it (various federal, state, and local laws; and even different tangles of laws once you send outside the U.S. (note: you have no way of finding if joe@hotmail.com lives in Southwest Elbonia, the country with world’s most draconian antispam laws)).

I’m pretty sure I missed a few heads of this hydra – are you still sure you want to do this yourself? If so, there’ll be another wave, this time merely the annoying problems inherent in sending an e-mail. (You see, SMTP is a store-and-forward protocol, which means that your e-mail will be shuffled across many SMTP servers around the Internet, in the hope that the next one is a bit closer to the final recipient. Basically, the e-mail is sent to an SMTP server, which puts it into its forward queue; when time comes, it will forward it further to a different SMTP server, until it reaches the SMTP server for the given domain. This forward could happen immediately, or in a few minutes, or hours, or days, or never.) Thus, you’ll see the following issues – most of which could happen en route as well as at the destination:

  • the remote SMTP servers don’t want to talk to your SMTP server
  • your mails are getting marked as spam (<blink> is not your friend here, nor is <font color=...>)
  • your mails are delivered days, even weeks late (contrary to popular opinion, SMTP is designed to make a best effort to deliver the message sometime in the future – not to deliver it now)
  • your mails are not delivered at all (already sent from e-mail server on hop #4, not sent yet from server on hop #5, the server that currently holds the message crashes, data is lost)
  • your mails are mangled by some braindead server en route (this one is somewhat solvable with base64 encoding, but then the size goes up and the e-mail looks more suspicious)
  • your mails are delivered and the recipients seem not to want them (“I’m sure I didn’t sign up for this, I remember exactly what I did a year ago” (of course you do, sir))
  • users with various versions of Microsoft Outlook and its special handling of Internet mail
  • wizard’s apprentice mode (a self-reinforcing positive feedback loop – in other words, automated e-mails as replies to automated e-mails as replies to…; you really don’t want to be the one to set this off, as you’d anger half the internet at yourself)

and it’ll be your job to troubleshoot and solve this (hint: you can’t, mostly). The people who run a legit mass-mailing businesses know that in the end you can’t solve it, and that they can’t solve it either – and they have the reasons well researched, documented and outlined (maybe even as a Powerpoint presentation – complete with sounds and cool transitions – that your bosses can understand), as they’ve had to explain this a million times before. Plus, for the problems that are actually solvable, they know very well how to solve them.

If, after all this, you are not discouraged and still want to do this, go right ahead: it’s even possible that you’ll find a better way to do this. Just know that the road ahead won’t be easy – sending e-mail is trivial, getting it delivered is hard.

People have recommended MailChimp which is a good vendor for bulk email. If you’re looking for a good vendor for transactional email, I might be able to help.

Over the past 6 months, we used four different SMTP vendors with the goal of figuring out which was the best one.

Here’s a summary of what we found…

AuthSMTP

  • Cheapest around
  • No analysis/reporting
  • No tracking for opens/clicks
  • Had slight hesitation on some sends

Postmark

  • Very cheap, but not as cheap as AuthSMTP
  • Beautiful cpanel but no tracking on opens/clicks
  • Send-level activity tracking so you can open a single email that was sent and look at how it looked and the delivery data.
  • Have to use API. Sending by SMTP was recently introduced but it’s buggy. For instance, we noticed that quotes (“) in the subject line are stripped.
  • Cannot send any attachment you want. Must be on approved list of file types and under a certain size. (10 MB I think)
  • Requires a set list of from names/addresses.

JangoSMTP

  • Expensive in relation to the others – more than 10 times in some cases
  • Ugly cpanel but great tracking on opens/clicks with email-level detail
  • Had hesitation, at times, when sending. On two occasions, sends took an hour to be delivered
  • Requires a set list of from name/addresses.

SendGrid

  • Not quite a cheap as AuthSMTP but still very cheap. Many customers can exist on 200 free sends per day.
  • Decent cpanel but no in-depth detail on open/click tracking
  • Lots of API options. Options (open/click tracking, etc) can be custom defined on an email-by-email basis. Inbound (reply) email can be posted to our HTTP end point.
  • Absolutely zero hesitation on sends. Every email sent landed in the inbox almost immediately.
  • Can send from any from name/address.

Conclusion

SendGrid was the best with Postmark coming in second place. We never saw any hesitation in send times with either of those two – in some cases we sent several hundred emails at once – and they both have the best ROI, given a solid featureset.

Here is what I did recently in PHP on one of my bigger systems:

  1. User inputs newsletter text and selects the recipients (which generates a query to retrieve the email addresses for later).

  2. Add the newsletter text and recipients query to a row in mysql table called *email_queue*

    • (The table email_queue has the columns “to” “subject” “body” “priority”)
  3. I created another script, which runs every minute as a cron job. It uses the SwiftMailer class. This script simply:

    • during business hours, sends all email with priority == 0

    • after hours, send other emails by priority

Depending on the hosts settings, I can now have it throttle using standard swiftmailers plugins like antiflood and throttle…

$mailer->registerPlugin(new Swift_Plugins_AntiFloodPlugin(50, 30));

and

$mailer->registerPlugin(new Swift_Plugins_ThrottlerPlugin( 100, Swift_Plugins_ThrottlerPlugin::MESSAGES_PER_MINUTE ));

etc, etc..

I have expanded it way beyond this pseudocode, with attachments, and many other configurable settings, but it works very well as long as your server is setup correctly to send email. (Probably wont work on shared hosting, but in theory it should…) Swiftmailer even has a setting

$message->setReturnPath

Which I now use to track bounces…

Happy Trails! (Happy Emails?)

Related Solutions

Performance issue with this code [closed]

In short: You should create,open,use,close,dispose Connections where you're using them. The best way is to use the using-statement. By not closing the connection as soon as possible, the Connection-Pool needs to create new physical connections to the dbms which...

Compare a pointer to an integer in C [closed]

Here's what I think you meant to post, it still doesn't compile though, since you can't compare a pointer to a char /* *Description: Construction of a social network */ #include <stdio.h> #include <strings.h> #include <stdlib.h> #define SIZE...

Autocomplete json in textbox

If you are using jQuery UI, the jQuery documentation on autocomplete is straightforward. Put your array as the source: and it should work automatically. IMHO, You seriously need to spend some time for googling and looking into the documentations. jQuery UI...

having all my scores and names in one big array

You need to initialize your array outside of your loop: name_arr = [] while int(students)>int(student): name = input ("what is your name ") score = input ("what is your score ") student = student + 1 name_arr.append(name) name_arr.append(score)...

pacman “exists on filesystem” error

After pacman finally deprecated the --force option and made the surrogate --overwrite option work as expected, the following usage pattern should be noted. A command to reproduce the --force option that blindly overwrites anything that conflicts is this: sudo...

How to determine the maximum number to pass to make -j option?

nproc gives the number of CPU cores/threads available, e.g. 8 on a quad-core CPU supporting two-way SMT. The number of jobs you can run in parallel with make using the -j option depends on a number of factors: the amount of available memory the amount of memory...

Number of Nearest ‘True’ in a matrix or list of list

Definitely not the best way to do it, but it's one that works: import numpy as np mas1 = np.array([[True, False, True], [ False, True, True], [ False, True, False]]) mas_answer = np.ndarray(shape=mas1.shape) for i in range(mas1.shape[0]): for j in...

Trying to display Json data from a web url into a table

You can take this json and put it in the loop through length of the json and show data into the table. This is how i solved it <?php try{ $url="the json url goes here"; // path to your JSON file $data = file_get_contents($url); // put the contents of the...

View v is unreachable statement

Anything else is written after the return keyword it's unreachable. Remove return super.getView(position, convertView, parent); from the first line of your function. This is a warning, telling you that static analysis of the code shows that some of your code...

index out of range but is in fact in range [closed]

Well try to debug your code by yourself first. Anyhow for your question Why is this happening? : It gives you error in postCode = split_address[4] because your list has 4 elements 0,1,2,3 and you are accessing the 4th element which is not present.. you don't...

Ubuntu update error: “waiting for unattended-upgr to exit”

I would first try a softer way. Stop the automatic updater. sudo dpkg-reconfigure -plow unattended-upgrades At the first prompt, choose not to download and install updates. Make a reboot. Make sure any packages in an unclean state are installed correctly. sudo...

how to Styling classes with the same name in a file css [closed]

You need to use :nth-of-type(n) selector. // For First Right Class Div #container .right:nth-of-type(1) { } // For First Left Class Div #container .left:nth-of-type(1) { } Hence for every div you need to change n value. Your question is extremely unclear but I...

Java – different parameters resulting to different outputs

What I think you're trying to achieve is that when you call your method "horn" with some parameter it has to either use "Beep!" or "Boop!". First of: void horn(a,b) Is not a valid function signature in Java, in a java function you always have to specify what...

Cannot use method returned value into another method

Using @super's suggestion and a little warning fixing. The two important changes are in the line as suggested by @super: printf("r=%.3f; phi=%.3fn",distanta(),phi()); The variables 'r' and 'unghi' are both variables local to member functions and cannot be...

Class has no member speak? [closed]

void::speak(); //THE GLOBAL SCOPE HAS NO SPEAK It's interpreting this as void ::speak() where leading an identifier (a name) with :: indicates to C++, "Look in the global scope of all names". :: is the "scope resolution operator" In the header file, you should...

Convert code with multiple lines into one line

Read the docs! A simple statement is comprised within a single logical line. Several simple statements may occur on a single line separated by semicolons. Search Stack Overflow! How to put multiple statements in one line? Or google, to find converters for more...

How to POSITION my Marker to Always Follow the Slider-Handle?

Youc can set a position to image using Jquery See fiddle //set a begining position to img var slider = $(".slider")[0]; var sliderPos = slider.value / slider.max; var pixelPostion = slider.clientWidth * sliderPos; $(".img").css("left",pixelPostion-7 + "px");...

css nth-child() check board pattern [closed]

This is pretty simple, as the pattern is repeated over 2 rows of 4, you just need to apply styles to 8n + i for the chequered pattern: .flex { display: flex; width: 400px; /* width of four squares */ flex-direction: row; flex-wrap: wrap; } .square { width:...