Monday, June 1, 2009
New iPost blog launches
The blogs of several iPost bloggers have been consolidated into a single new blog. Check it out at http://www.notjustanothermarketingblog.com/ for the latest from iPost.
Saturday, November 22, 2008
Email Postage: Payment and Responsibility
If an ISP were to charge a fee for delivery of email to its users, what expectations would you have about the success of that delivery?
Brian Krebs of the Washington Post reported recently on a study by University researchers in California that dissects some of the economics of spam email. Although it presents a sobering picture of the gullibility of the average email recipient, it also suggests that the return on a spammer's email blast is much lower than previous estimates. It takes about a billion messages to get a return of less than $10,000.
This has contributed to a renewed discussion of "email postage," that is, the idea of charging a fee for every email message, similar to the USPS and other postal services charging for every physical letter. If it cost as little as 1/100 of a cent to deliver each of those billion messages, proponents say, all spammer profits would be wiped out and it would no longer be worth their while to flood the net with junk. At such a tiny rate, postage bills for ordinary users would total less than a few dollars, and usually only a few cents, per year, and could be absorbed into the fees already paid for internet access, so only senders of millions of messages would even notice.
Another advantage to a "sender pays" model of email is that it might suddenly become very expensive to have your personal PC taken over by malicious software and turned into a "spam bot." A few days after Krebs reviewed the UC research, he also reported on the takedown of McColo Corporation, an ISP that appears to have been hosting the command controls for a large number of bots. Almost immediately following McColo's loss of connectivity, sites that monitor the volume spam reported that the amount of junk email reaching their sensors had fallen by 30% to 75%. Clearly botnets are a huge contributor to the spam problem, and anything that helps to identify compromised PCs and encourages cleaning them up, is a good thing.
Unfortunately, spam reduction isn't a sufficient incentive to put an email postage system in place. Ignoring the technical hurdles involved in organizing reliable payment for "stamps" and preventing forgery or re-use of stamp tokens, consider the shift in attitude that would inevitably accompany an imposition of fees by an ISP when accepting email.
In today's email infrastructure, the majority of the cost is borne by the recipient's mailbox provider. ISPs and other providers owe nothing to the senders of email; they charge the recipient, either directly or by sprinkling their user interface with advertising. Consequently the provider has no obligation, and no incentive aside from maintaining the goodwill of their users, to provide reliable delivery of email that originates outside their own network.
An email postage system would turn that on its head. When you put a stamp on a letter and turn it over to the postal service, you have both a belief and and expectation that the post office is going to do its job and carry that letter to the addressee. Would it be any different if you bought that stamp for an email message?
Accepting payment from the sender of an email obligates the ISP to make a best effort to get the message into the recipient's mailbox. The ISP becomes liable to both the sender and the recipient if a system failure results in the loss of paid email. No more silent acceptance of email which is then discarded undelivered, and maybe no more junk folders unless each mailbox owner creates his own personal set of junk rules. A strong expectation that anyone willing to pay the full postage rate should be exempt from delaying tactics like greylisting and rate-limiting.
I don't think ISPs are prepared to take on that responsibility. I think they'd rather stay right where they are, in complete control of what they allow on their networks and free from real obligations to provide reliable email delivery. Right now they're the good guys, valiantly defending their users from the relentless siege of an army of baddies. Postage would be tantamount to taking bribes to let the baddies in; consider the (largely misguided) fracas that resulted when AOL announced a partnership with Goodmail. Does a tiny fraction of a cent per email outweigh the cost of implementing a postage system and marketing it to the users?
I'd like to be wrong about this. What are your thoughts? How would you react if you had to pay postage for your email?
Brian Krebs of the Washington Post reported recently on a study by University researchers in California that dissects some of the economics of spam email. Although it presents a sobering picture of the gullibility of the average email recipient, it also suggests that the return on a spammer's email blast is much lower than previous estimates. It takes about a billion messages to get a return of less than $10,000.
This has contributed to a renewed discussion of "email postage," that is, the idea of charging a fee for every email message, similar to the USPS and other postal services charging for every physical letter. If it cost as little as 1/100 of a cent to deliver each of those billion messages, proponents say, all spammer profits would be wiped out and it would no longer be worth their while to flood the net with junk. At such a tiny rate, postage bills for ordinary users would total less than a few dollars, and usually only a few cents, per year, and could be absorbed into the fees already paid for internet access, so only senders of millions of messages would even notice.
Another advantage to a "sender pays" model of email is that it might suddenly become very expensive to have your personal PC taken over by malicious software and turned into a "spam bot." A few days after Krebs reviewed the UC research, he also reported on the takedown of McColo Corporation, an ISP that appears to have been hosting the command controls for a large number of bots. Almost immediately following McColo's loss of connectivity, sites that monitor the volume spam reported that the amount of junk email reaching their sensors had fallen by 30% to 75%. Clearly botnets are a huge contributor to the spam problem, and anything that helps to identify compromised PCs and encourages cleaning them up, is a good thing.
Unfortunately, spam reduction isn't a sufficient incentive to put an email postage system in place. Ignoring the technical hurdles involved in organizing reliable payment for "stamps" and preventing forgery or re-use of stamp tokens, consider the shift in attitude that would inevitably accompany an imposition of fees by an ISP when accepting email.
In today's email infrastructure, the majority of the cost is borne by the recipient's mailbox provider. ISPs and other providers owe nothing to the senders of email; they charge the recipient, either directly or by sprinkling their user interface with advertising. Consequently the provider has no obligation, and no incentive aside from maintaining the goodwill of their users, to provide reliable delivery of email that originates outside their own network.
An email postage system would turn that on its head. When you put a stamp on a letter and turn it over to the postal service, you have both a belief and and expectation that the post office is going to do its job and carry that letter to the addressee. Would it be any different if you bought that stamp for an email message?
Accepting payment from the sender of an email obligates the ISP to make a best effort to get the message into the recipient's mailbox. The ISP becomes liable to both the sender and the recipient if a system failure results in the loss of paid email. No more silent acceptance of email which is then discarded undelivered, and maybe no more junk folders unless each mailbox owner creates his own personal set of junk rules. A strong expectation that anyone willing to pay the full postage rate should be exempt from delaying tactics like greylisting and rate-limiting.
I don't think ISPs are prepared to take on that responsibility. I think they'd rather stay right where they are, in complete control of what they allow on their networks and free from real obligations to provide reliable email delivery. Right now they're the good guys, valiantly defending their users from the relentless siege of an army of baddies. Postage would be tantamount to taking bribes to let the baddies in; consider the (largely misguided) fracas that resulted when AOL announced a partnership with Goodmail. Does a tiny fraction of a cent per email outweigh the cost of implementing a postage system and marketing it to the users?
I'd like to be wrong about this. What are your thoughts? How would you react if you had to pay postage for your email?
Wednesday, October 1, 2008
Standards Reborne - SMTP and the Internet Message Format
With remarkably little flag-waving1 the Internet Engineering Task Force (IETF) today published new updates to the venerable Simple Mail Transfer Protocol (SMTP, RFC5321) and Internet Message Format (RFC5322) standards. These documents are the most recent reflection of more than twenty-six years of effort that began when Jon Postel and Dave Crocker penned2 the early drafts of Request For Comments 821 and 822. Those standards created the email universe as we know it today, building on the cooperative structure I mentioned in an earlier post.
The most recent previous update to this pair of standards was in 2001, and an enormous amount of discussion and refinement has transpired since then. John Klensin and Pete Resnick are to be commended for their patience and perseverance in marshaling the ideas of the loosely allied (and occasionally antagonistic) participants in the "working groups" of academic and industry experts and engineers who finally reached this consensus. These new publications are draft standards, final specifications that are unlikely to change further.
What, then, does this mean to you as a user of the Internet for email? Surprisingly, almost nothing. These two standards do not introduce any new security features or delivery hurdles, although other forthcoming proposals may (so I'm not likely to run out of things to write about).
The IETF strives to produce standards that reflect widely accepted current practice. Publication must be supported by evidence of multiple, interoperable (e.g., successfully exchanging data) implementations of communication techniques and data formats. In this case, that means that RFCs 5321 and 5322 provide a detailed description of exactly how your email is already meant to be working. You shouldn't need to change a thing.
Most importantly it means that everyone who wishes to engage in the free exchange of email on the Internet has a new and clearer definition of what to expect at the most basic level of communication between senders and recipients and among the servers that move messages from place to place. iPost works hard to make sure your messages, and our handling of them on your behalf, conform to these standards so that you can reach your email audience.
(Update 2008/10/02: MailChannels has a detailed list of the changes since the 2001 proposed standard.)
1Standards-bearing ... get it?
2Typed, really, but "penned" sounds more elegant.
The most recent previous update to this pair of standards was in 2001, and an enormous amount of discussion and refinement has transpired since then. John Klensin and Pete Resnick are to be commended for their patience and perseverance in marshaling the ideas of the loosely allied (and occasionally antagonistic) participants in the "working groups" of academic and industry experts and engineers who finally reached this consensus. These new publications are draft standards, final specifications that are unlikely to change further.
What, then, does this mean to you as a user of the Internet for email? Surprisingly, almost nothing. These two standards do not introduce any new security features or delivery hurdles, although other forthcoming proposals may (so I'm not likely to run out of things to write about).
The IETF strives to produce standards that reflect widely accepted current practice. Publication must be supported by evidence of multiple, interoperable (e.g., successfully exchanging data) implementations of communication techniques and data formats. In this case, that means that RFCs 5321 and 5322 provide a detailed description of exactly how your email is already meant to be working. You shouldn't need to change a thing.
Most importantly it means that everyone who wishes to engage in the free exchange of email on the Internet has a new and clearer definition of what to expect at the most basic level of communication between senders and recipients and among the servers that move messages from place to place. iPost works hard to make sure your messages, and our handling of them on your behalf, conform to these standards so that you can reach your email audience.
(Update 2008/10/02: MailChannels has a detailed list of the changes since the 2001 proposed standard.)
1Standards-bearing ... get it?
2Typed, really, but "penned" sounds more elegant.
Wednesday, August 13, 2008
Receivers take advantage of "Store and Forward", and so should senders
Recently I shed some light on what happens behind the scenes when an email message is sent on its way. The "store and forward" model is not a perfect guarantee of delivery, but is very robust in the face of temporary breakdowns. You may have noticed, however, that if a message has to be passed along through several "hops", only the final "hop" can determine exactly when or even whether the message has reached its intended recipient. To deal with this, the Simple Mail Transfer Protocol (SMTP), the standard way to transmit email over the Internet, defines all the elements necessary to complete a successful transfer of email and to report either a temporary service interruption or a complete failure.
This is where notification messages come in. Often appearing to come from some cryptic sender such as a "Mailer Daemon" (which is just another name for mail transport software), notifications tell us that a recipient does not exist or that a mailbox is full, or about other errors. Each of the "hops" through which a message passes, including the last, may send back one of these "bounces" to let you know that something unexpected is happening to your email.
It's also possible for a sender to request that a notification be sent at final delivery when everything has gone right. Keith Moore's lament, as the NYT reported, is that end-user email software rarely makes it simple enough to answer such a request for it to be worth the sender's effort to ask.
Part of the reason for using the store and forward email model and this system of notifications is its ability to allow receivers of email to make assertions about the availability of their resources. Returning to the analogy of the phone call, when a particular mail exchanger is becoming overloaded, it can "answer the phone" but then immediately request that the sender "call back later," confident that the sender has stored a copy of the message for later forwarding. When you hear the term "soft bounce," it refers to such a delay.
Increasingly, however, receivers are using the "call back later" mechanism to control on the rate at which they accept email from any particular source as a matter only of policy, not of necessity. Recently, the operator of one of the most venerable newsletters on the Internet reported that his deliveries were being blocked by a major ISP. In fact, that ISP was engaging in a delaying tactic, repeatedly asking that the newsletter sender wait and try again.
Intentional delays are, of course, not the only kind. It is rare, for example, that the computer that acts as a mail exchanger is the same one that eventually places the email in a recipient's mailbox, and even more rare for that to be the same computer that eventually presents the email to the recipient. The store and forward model is as prevalent in the systems employed inside domains as it is outside, so it is always possible that a delay is introduced at some point.
So is there anything that you, as an email sender, can do about this? Yes. Remember that the nature of store and forward is to give email the best chance of eventually arriving. It may not get there in an instant, but the chances are that it will get there. This is critical to keep in mind when planning email marketing campaigns. Do you have a last-minute offer? Consider sending it to a smaller, more actively responsive list of recipients, or include a consolation offer so the email remains relevant to those who miss the deadline through no fault of their own. Announcing a one-time event? Plan ahead so there is time for the mail to arrive. You may want to consider a second reminder email, sent closer to the date but only to those who have already reacted to the initial announcement.
Email has persistence by design, as well as (usually) immediacy. It's wise to exploit both.
This is where notification messages come in. Often appearing to come from some cryptic sender such as a "Mailer Daemon" (which is just another name for mail transport software), notifications tell us that a recipient does not exist or that a mailbox is full, or about other errors. Each of the "hops" through which a message passes, including the last, may send back one of these "bounces" to let you know that something unexpected is happening to your email.
It's also possible for a sender to request that a notification be sent at final delivery when everything has gone right. Keith Moore's lament, as the NYT reported, is that end-user email software rarely makes it simple enough to answer such a request for it to be worth the sender's effort to ask.
Part of the reason for using the store and forward email model and this system of notifications is its ability to allow receivers of email to make assertions about the availability of their resources. Returning to the analogy of the phone call, when a particular mail exchanger is becoming overloaded, it can "answer the phone" but then immediately request that the sender "call back later," confident that the sender has stored a copy of the message for later forwarding. When you hear the term "soft bounce," it refers to such a delay.
Increasingly, however, receivers are using the "call back later" mechanism to control on the rate at which they accept email from any particular source as a matter only of policy, not of necessity. Recently, the operator of one of the most venerable newsletters on the Internet reported that his deliveries were being blocked by a major ISP. In fact, that ISP was engaging in a delaying tactic, repeatedly asking that the newsletter sender wait and try again.
Intentional delays are, of course, not the only kind. It is rare, for example, that the computer that acts as a mail exchanger is the same one that eventually places the email in a recipient's mailbox, and even more rare for that to be the same computer that eventually presents the email to the recipient. The store and forward model is as prevalent in the systems employed inside domains as it is outside, so it is always possible that a delay is introduced at some point.
So is there anything that you, as an email sender, can do about this? Yes. Remember that the nature of store and forward is to give email the best chance of eventually arriving. It may not get there in an instant, but the chances are that it will get there. This is critical to keep in mind when planning email marketing campaigns. Do you have a last-minute offer? Consider sending it to a smaller, more actively responsive list of recipients, or include a consolation offer so the email remains relevant to those who miss the deadline through no fault of their own. Announcing a one-time event? Plan ahead so there is time for the mail to arrive. You may want to consider a second reminder email, sent closer to the date but only to those who have already reacted to the initial announcement.
Email has persistence by design, as well as (usually) immediacy. It's wise to exploit both.
Wednesday, August 6, 2008
Is your email being "miscarried"?
Networking consultant Keith Moore doesn't ask for return receipts to confirm that his email has been delivered. This is somewhat ironic, as he was co-author of the 1996 specification that generates a confirmation message when an e-mail safely reaches its destination server. Moore, an advocate of measures to confirm email delivery, is "frustrated because end users' e-mail software still lacks the design capability to use the server-to-server messages about completed email delivery" without user intervention, reports the New York Times.
We're all familiar with email. Messages to and from colleagues arrive in seconds. Such efficiency often leads to the expectation that every email, no matter where it originates or is destined to go, must behave in the very same way. Yet, "in the wild" of the Internet the actual dynamics of email transmission may be different each and every time an email is sent -- even when the sender and the recipient are the same.
Why? It goes back to the creation of the Internet and its architecture. Initially the network was a cooperating group of educational institutions and a few technology corporations. Because data transmission lines were expensive, few of these institutions were directly connected to one another. As a result, senders had to completely describe each of the "hops" a message would need to make from one computer to the next to which it was connected until it reached its destination. Sometimes those transmission lines or the computers connected to them would temporarily stop working, so each computer stored the message locally before passing it along, then deleted the copy after it was passed.
This ensured a robust system because at each step of the way a copy of the file existed on both sides of the transfer. If a line or computer failed during transmission it was possible to wait for the problem to be corrected and then pick up the transmission at a prior step. Called "store and forward" transmission, this technique is still used today. The difference is that transmission details are now mostly hidden from the sender and the recipient. Also the number of hops required to send an email is now far less because the modern Internet passes data through a connected series of transmission lines at a much smaller granularity than that of entire files, providing the illusion that any two computers can connect directly to one another.
The advent of such "direct" connections has led to its own set of problems. Two computers communicating over the network is similar to a phone call, and although it is possible for a computer to answer a lot of calls, it eventually reaches a limit. To keep email flowing, a mechanism is used to identify more than one "phone number" that can be used to answer the "calls" going to the same destination. Called "mail exchangers," these numbers are announced by an Internet domain by publishing "MX records" that also include instructions that senders are expected to obey for how to choose an exchanger for each new email transmission. Generally, the more end recipients an Internet domain represents, the more mail exchangers it will advertise.
AOL, Yahoo, MSN and other large ISPs publish many MX records, as do large corporations. Smaller businesses publish fewer records or share MXs with other small businesses through their ISP. Each record represents a finite resource under the control of its provider. Line and computer failures are far less common than they were back when "store and forward" transmission was invented, but they still happen, and in most cases the system works and gets your email through. Keep in mind, however, that for email the Internet is still largely a cooperative that depends more on the resources of the receivers than of the senders.
Successful email campaigns rely on emails reaching their destination. They rely on the ecosystem working. In a future post, I'll discuss some of the ways that system can work against you, but also some ways you can take better advantage of it.
We're all familiar with email. Messages to and from colleagues arrive in seconds. Such efficiency often leads to the expectation that every email, no matter where it originates or is destined to go, must behave in the very same way. Yet, "in the wild" of the Internet the actual dynamics of email transmission may be different each and every time an email is sent -- even when the sender and the recipient are the same.
Why? It goes back to the creation of the Internet and its architecture. Initially the network was a cooperating group of educational institutions and a few technology corporations. Because data transmission lines were expensive, few of these institutions were directly connected to one another. As a result, senders had to completely describe each of the "hops" a message would need to make from one computer to the next to which it was connected until it reached its destination. Sometimes those transmission lines or the computers connected to them would temporarily stop working, so each computer stored the message locally before passing it along, then deleted the copy after it was passed.
This ensured a robust system because at each step of the way a copy of the file existed on both sides of the transfer. If a line or computer failed during transmission it was possible to wait for the problem to be corrected and then pick up the transmission at a prior step. Called "store and forward" transmission, this technique is still used today. The difference is that transmission details are now mostly hidden from the sender and the recipient. Also the number of hops required to send an email is now far less because the modern Internet passes data through a connected series of transmission lines at a much smaller granularity than that of entire files, providing the illusion that any two computers can connect directly to one another.
The advent of such "direct" connections has led to its own set of problems. Two computers communicating over the network is similar to a phone call, and although it is possible for a computer to answer a lot of calls, it eventually reaches a limit. To keep email flowing, a mechanism is used to identify more than one "phone number" that can be used to answer the "calls" going to the same destination. Called "mail exchangers," these numbers are announced by an Internet domain by publishing "MX records" that also include instructions that senders are expected to obey for how to choose an exchanger for each new email transmission. Generally, the more end recipients an Internet domain represents, the more mail exchangers it will advertise.
AOL, Yahoo, MSN and other large ISPs publish many MX records, as do large corporations. Smaller businesses publish fewer records or share MXs with other small businesses through their ISP. Each record represents a finite resource under the control of its provider. Line and computer failures are far less common than they were back when "store and forward" transmission was invented, but they still happen, and in most cases the system works and gets your email through. Keep in mind, however, that for email the Internet is still largely a cooperative that depends more on the resources of the receivers than of the senders.
Successful email campaigns rely on emails reaching their destination. They rely on the ecosystem working. In a future post, I'll discuss some of the ways that system can work against you, but also some ways you can take better advantage of it.
Subscribe to:
Posts (Atom)
