Forwarding Pitfalls

Or, Forwarding Considered Harmful

Overview

Many Lab members regularly forward their mail, and read it at some other address than the one they usually give out. This has been a standard and routine practice since the early days of email, but it’s becoming increasingly unreliable, and we wanted to call the community’s attention to some pitfalls with forwarding (especially, but not exclusively, with forwarding to Gmail).

On the modern internet, email delivery is a constantly escalating arms race among spammers and scammers (and sometimes spies), legitimate email senders, and email providers. Email reliability overall is going down, and that’s especially the case for forwarded email, and even more so for mail forwarded to Gmail.

Moreover, when mail is forwarded to Gmail, if Google doesn’t accept it, or accepts it and doesn’t deliver it, the fact that Google’s decisions about delivering mail are largely dynamic and automated means that often nobody (not even Google engineers) can determine why Google did not deliver a message.

See also

• The Email Forwarding section on our Email Management page.
• IS&T’s forwarding documentation in the IS&T Knowledge Base (only accessible from MIT networks or the MIT VPN).

TIG’s recommended practice

By far, the best practice for email is to give out the email address where you actually read your mail, so that if that’s Gmail, people are sending mail directly to your Gmail address; if it’s IS&T’s Exchange infrastructure, people are sending mail directly to your @mit.edu address; if it’s your CSAIL IMAP account, people are sending mail directly to your @csail.mit.edu address, and so on.

Forwarding can be a useful tool when changing email addresses, or so that old published addresses don’t just stop working completely, but in our opinion it’s not reliable enough to substitute for telling your correspondents what the actual email address is where you read your mail.

Forwarding within CSAIL can also be very useful for various reasons (and is generally much more reliable than forwarding from a CSAIL address to a non-CSAIL address, at least assuming the destination CSAIL address doesn’t itself forward outside of CSAIL). For instance, forwarding mysoftware-bugs@csail.mit.edu to me@csail.mit.edu, assuming you read your mail on CSAIL’s IMAP server and don’t forward it elsewhere, should be almost as reliable as if the mysoftware-bugs alias weren’t involved.

Specific pitfalls when forwarding to Gmail

• A Google employee has told us in so many words that Google does not expect forwarding to Gmail to be reliable (as opposed to an individual sending mail directly to a Gmail address for a recipient who reads it there).
• Since a lot of Gmail’s mail handling is automated and driven by machine-learning, its decisions about whether to accept a message and, once accepted, whether to actually deliver it or discard it are fairly inscrutable.
• When it’s not confident of its ability to classify a message as spam or not, Google will often defer delivery, which causes a message to be delayed significantly.
• One factor Gmail seems to take into account is the number of similar messages Gmail users have gotten in a short period of time. This causes problems for mailing-list messages or other messages sent to multiple recipients, where often the first recipient or two will get a message right away, some recipients will eventually get the message but it will be delayed, and some recipients won’t get the message at all (whether Gmail accepts it or not).
• Gmail will only accept mail below 25MB in size. Some of our users often get messages larger than that. (And messages larger than that are disproportionately likely to be important, like PDFs awaiting a signature.)
  • Note: For some enterprise Gmail customers, this limit is actually 50MB. That affects people forwarding to an institutional domain whose mail is handled by Gmail; we don’t believe it affects people forwarding to an individual Gmail account who pay Google for preferential treatment (like Google One) or additional storage.
• This can be a problem with mail forwarded from Gmail as well as with mail forwarded to Gmail, but when Google thinks that both ends of an email message are inside Gmail, it will not pass along a message to outside (non-Gmail) email servers even if it normally should. For instance, if g.hopper@gmail.com sends mail to c.babbage@csail.mit.edu, and c.babbage@csail.mit.edu forwards to c.babbage@gmail.com, and Google knows that c.babbage@gmail.com is configured to be able to send mail with a From: line saying c.babbage@csail.mit.edu, then the mail will never pass through CSAIL’s servers, since Gmail “knows” that it’s the final destination for mail addressed to c.babbage@csail.mit.edu. That’s OK if only forwarding is involved, but not ideal if the point of giving out a CSAIL address is to allow CSAIL’s servers to do mail filtering or archiving, or to have a CSAIL mailbox as a backup copy. (Or if Gmail is supposed to be the backup copy, and Prof. Babbage normally reads his mail at CSAIL.)

(It’s worth noting that Google and Microsoft between them handle a massive fraction of the email traffic of North America and Western Europe, and Google handles so much personal, non-business-related mail, that for any two people in the US sending email to each other there’s a good likelihood the email never needs to leave Google.)

General pitfalls with email forwarding

In addition to some specific pitfalls of forwarding to Gmail, there are a bunch of general problems with forwarding, of varying degrees of severity:

• Each hop introduces more chances for failure and delay
  • If your mail has to go through three mail servers to reach its destination instead of one, that’s three times the opportunities for spam- or malware-filtering to incorrectly refuse or discard it.
  • If your mail has to go through three mail servers instead of one that’s three times as much chance that a transient hardware or networking problem delays your message significantly.
  • Each hop can introduce new reasons why a message might be refused at later stages. (For instance, any server in the chain could have its IP address on a blocklist, or any server in the chain could reformat the message or add a disclaimer to it in a way that would break cryptographic signatures.)
  • Mail servers impose an upper limit on the number of other mail servers that a message can have traversed. That limit is generally very high, and forwarding through CSAIL typically only adds one to four servers to it, but some large commercial providers like Microsoft and Google can add a dozen or more internal servers, so forwarding into Exchange and then out again, or out of Gmail and then back in, can quickly hit that limit.
• Forwarding can sometimes mess with where bounces go. For instance, when mail is forwarded through an @mit.edu address, IS&T changes the envelope sender to point at the original @mit.edu recipient, not the original sender. So if a message is sent from k.goedel@csail.mit.edu to a.lovelace@mit.edu and forwarded from there to a.lovelace@csail.mit.edu, and CSAIL’s mail servers reject it (correctly or incorrectly) as malware, the bounce generated will go to a.lovelace@mit.edu instead of the original sender, k.goedel@csail.mit.edu (where it will be forwarded again). So the original sender will not see the bounce. Moreover, depending why the original message was refused, the bounce itself might be refused for the same reason, which would mean the message was lost and nobody saw a bounce.
• Forwarding makes it much, much harder to figure out when, whether, and where delivery problems are occurring, since no one group of sysadmins will have access to all the relevant logs.
• Similarly, when responsibility for delivery of a message is spread across multiple institutions, it’s hard for an end user (message sender or intended recipient) to figure out whom to ask for help.
• Forwarding addresses have an unfortunate tendency to “bit-rot” over time, for various reasons. So a forwarding address that is reliable in 2025 may only intermittently work in 2030 and may consistently fail in 2035 — by which time you may get mail forwarded from that address infrequently enough that you don’t notice it’s no longer working, and you no longer remember which of your correspondents — like the domain registrar where you paid for several years registration of an important domain in advance — might have the old address.
  • This dynamic is especially notable for forwarding addresses set up before the advent of email spam. For instance, many of our legacy @ai.mit.edu or @lcs.mit.edu email addresses forward to addresses that either no longer exist or that are much more strict about what kinds of mail they will accept than they were when the forwarding was set up.

Some concrete examples

These are fictionalized examples, but based on real problems TIG sysadmins have seen with mail delivery. We may add to this list as we run across new examples.

1. You’re working on buying a house. You give out your CSAIL email address, which forwards to your Gmail account. Eventually it’s time for you to sign the contract, so the seller’s attorney sends you the contract as a large PDF, making the message almost (but not quite) 25MB large. You forward it to your lawyer for review and advice. Your lawyer replies, quoting the contract, but the lawyer’s reply plus the long legalese disclaimer at the bottom of your lawyer’s message causes the reply to be over 25MB, so Gmail doesn’ accept it at 3pm on a Friday. Since you expected a nearly immediate response, you ask your lawyer about it at 4:30. Your lawyer replies to the effect of “I already sent you all that” and quotes her original 3pm reply — again, of course, making the message larger than 25MB so Gmail won’t accept it.
2. You give your @mit.edu address to your bank. Your @mit.edu address forwards to your @csail.mit.edu address, which forwards to Gmail. Your bank publishes information in DNS saying “don’t accept mail claiming to be from us unless it comes from one of our email servers or is cryptographically signed by us”, which is normally not a problem, but in some mysterious circumstances Microsoft Exchange can break those cryptographic signatures. Since Google knows this and (we’re guessing) has backchannels with Microsoft since they both handle so much mail, the broken signature might not be a problem if the message were forwarded directly from Microsoft to Gmail, but when it’s forwarded through CSAIL Google isn’t expecting broken signatures and refuses it. Due to some slightly unusual address rewriting MIT’s Exchange-based email infrastructure does (intended to work around other forwarding problems), the bounce message does not go to the bank, so they have no way of knowing you didn’t get the message. And since you get lots of bounce messages (mostly of forwarded spam) and usually delete them, you’re likely to miss the bounce message when it goes to you instead — and that’s even if Gmail doesn’t decide on its own that the bounce is likely to be spam and hide it.
3. You run your own mail server on its own custom domain in our OpenStack cluster and give out that address because it’s short and memorable. You forward from that server to your CSAIL address. In addition to handling incoming mail, you run a bunch of cron jobs on that server. A routine security upgrade requires manual intervention to replace a deprecated feature in your email-server configuration, so your mail system breaks. All the cron jobs sending output to the broken mail system (including the cron jobs you set up to tell you if anything is wrong on the server) cause the server’s virtual root drive to fill up. You don’t notice this for several days, when a piece of mail you’re expecting doesn’t show up. By that point, it’s been so long that some of the servers trying to send you mail have given up.
4. You forward your CSAIL email address to your Gmail address, and you ask a colleague to send you a small dataset to your CSAIL email address. Your colleague knows the tarfile is pretty large for email, but also knows they’ve been able to send it to CSAIL addresses in the past, since it’s only 35MB, so they send it to you and don’t expect problems. But since you forward to Gmail and Gmail refuses messages larger than 25MB, the file does not get delivered. For mysterious reasons known only to Gmail, the message was deferred for about half an hour before Gmail finally refused it, and the sender gets a lot of bounce messages, so they don’t click through when they get the bounce message and don’t notice the dataset didn’t get through. If you’d have asked the sender to send directly to your Gmail address, they’d have been alert to the likelihood of problems and the bounce message might have arrived more quickly and been obviously a bounce to their email to you.