Icon The Kermit Project   |   Now hosted by Panix.com
New York City USA   •   kermit@kermitproject.org
…since 1981

Using the MM email client in the Modern World

Frank da Cruz
fdc@columbia.edu
Most recent update: Wed Nov 23 19:47:38 2022 (replace ftp links by http links)

References:   Introduction to MM (Original MM Authors, 1990)  ||  MM Manual Columbia U (1992-96)

This page is for experienced MM users and for MM developers. Readers who want an overview of what MM is and how it works should consult the two references just above, and for technical terms the glossary at the end. Postscript July 2022: The work I did on MM, described in the history section below, ceased in 2015 when Columbia made it impossible for me to use MM by removing both sendmail and our POP3 server and forcing everyone onto Gmail.

Synopsis

Columbia MM version 0.96.0(14), March 3, 2014

MM (Mail Manager) is an interactive text-mode mail client for UNIX written in the C language that, although it does not include instrinsic support for MIME or attachments, has a lot of other features that appeal to people who deal mainly with text and who like to keep their hands on the keyboard and work fast. It is noted for its powerful message selection and management features and its user-friendly (truly) DEC-20 style command language. MM was enormously popular at Columbia University and elsewhere from the mid-1970s to the late 1990s, when GUIs took over. See the below for some history. Also see the Wikipedia page.

The term Columbia MM is used to distinguish the C-Language version of MM written at Columbia University from the original DECSYSTEM-20 version written in PDP-10 assembly language on which it was based (see history), but since Columbia MM is no longer used or developed or maintained at Columbia, perhaps a more appropriate name would be CMM (MM written in C).

The previous major Columbia MM release was in the late 1990s for Year-2000 updates. The 2014 release described in this page includes:

Thus using the techniques described below with the new MM you can receive email even on Unix computers that no longer support mailspool delivery by using secure POP3 (Post Office Protocol) to fetch your new mail; you can send and receive mail in ISO 8859, UTF8, or other character set; you can have incoming messages automatically decoded (from Base 64 or Quoted Printable); you can read HTML or PDF or Microsoft Word as plain text. And with MM you also control the storage and archiving of your own email. It is not on the corporate mail server, it is not in the "cloud", it is wherever you want it put it in case you don't want every corporation and government agency digging through it.

Building and installing MM

  1. Cd to the desired directory. When you unpack the source-code archive it will create a subdirectory containing the MM files.
  2. Download the following source-code archives:

    https://kermitproject.org/ftp/kermit/mm/mm-0.96.0.14.tar.gz

    The previous version is here: https://kermitproject.org/ftp/kermit/mm/mm-0.95.0.tar.gz

  3. gunzip the tar.gz file you downloaded
  4. tar xvf the .tar result This creates a directory that contains all the MM and CCMD files.
    It has a name like mm-0.95.0 or mm-0.96.0.14.
    The number after the last decimal point changes automatically every time the program is compiled but does not necessarily indicate any changes.
  5. cd to this directory, which has several subdirectories including mm and ccmd.
  6. cd ccmd
  7. make clobber   (removes any object files and binaries)
  8. make xxx
  9. cd ../mm
  10. make clobber   (removes any object files and binaries)
  11. make xxx
where xxx is the appropriate target name for your computer:
aixrs6000 aixrt all bsd43 dynix211 freebsd linux hpux
hpux-gcc hpux11 hpux11-gcc irix4 irix5 irix6 isi40 macosx mtxinu43
netbsd next next-gcc openbsd osf1 osf1-4 osf1-5 osf1-c++ osf1-posix
solaris solaris-c++ solaris-gcc solaris10 solaris2.1 solaris2.2
solaris2.2-gcc solaris2.3 solaris2.3-c++ solaris2.3-gcc solaris2.4
solaris2.x sunos sunos3.4 sunos3.5 sunos4.x sunos5.3 sysv3b2 sysv52
ultrix ultrix20 ultrix40 ultrix41 ultrix42 ultrix4x umaxv

This should produce an executable called 'mm' in the mm/mm/ directory, which, if mm is to be used by multiple users, should be put in some directory in the the system-wide PATH, otherwise put it in some directory that is in your own PATH. There's also the matter of installing help files in some common location that MM knows about. I can figure this out if anybody asks me. There's an old installation document here but it was written before MM's help text was externalized.

After the first time you build MM, you shouldn't need to remake ccmd; that is, you should be able to just give one 'make' command in mm directory.

Currently I'm working in NetBSD 6.1.2, and not paying much attention to other platforms because I'm not touching anything platform-specific (but I do build it occasionally on Linux and on Solaris just to make sure it can still be done). However, since all varieties of Unix are known to be in constant flux, you are likely to get some (maybe massive amounts of) political-correctness warnings from the compilers (such as pointer-type mismatches) or linkers (such as warnings about the use of "deprecated" APIs). If an mm binary is produced that starts and does its work, I wouldn't worry about the warnings. If you feel like fixing them, be aware that any fix you make for your own platform is likely to break MM on other platforms.

If MM fails to build on your computer, first be sure you followed the instructions above, and if you did then you'll need need to figure out why it failed and let me know.

The MM distribution has four subdirectories (some of which have their own subdirectories):

ccmd
Source for Columbia's "COMD JSYS in C" package, which provides TOPS-20 style command parsing for Unix applications. This is an almost complete and perfect replica of the actual TOPS-20 COMND JSYS (see references).
mm
Source code for MM itself, which is combined at link time with the CCMD modules.
notes
Assorted edit histories and commentary by many people, of interest to anyone who might want to build, port, or further develop MM.
scripts
Kermit scripts for use with MM, including the POP3 scripts and XLATE scripts mentioned in this page.
Don't try to reorganize or rename the files or directories; the Makefiles are keyed on the current organization.

History

The original MM was written in PDP-10 assembler by Stuart McLure Cracraft, Mike McMahon, and Mark Crispin (among others) for the DESYSTEM-20 timesharing system in the late 1970s to mid 1980s and came to Columbia in about 1978 where it was adopted enthusiastically by thousands of computer users who depended on it until 1988 when the last DEC-20 was decommissioned. HERE, by the way, is my MM.INIT file from TOPS-20 MM, which shows not only that the original MM had somewhat different syntax for its initialization file but also that it could read "bboards" [bulletin boards], a precursor to newsgroups).

Columbia MM was written at Columbia University in K&R C in 1984-88 (before ANSI C existed) for the Unix timesharing system that was to replace our DEC-20s (see history) and was released to the Columbia user community in 1988 as our last DEC-20 was turned off, so users could migrate their mailboxes and still have the same high level of email service rather than have to deal with the cryptic and barely functional Unix mail clients available at the time.

Columbia MM was designed for portability and was built not only on different Unix platforms but also on non-Unix platforms including MS-DOS and possibly VMS. The original work was done as part of the Hermit Project under a grant from Digital Equipment Corporation, 1984-87. The programmers were Chris Maio, Melissa Metz, Fuat Baran, and Howie Kaye; others may have included Andy Lowry, Delores Ng, Bill Catchings, Bill Schilit, maybe even me, who remembers. The principal investigator for the DEC grant was Frank da Cruz (me).

The new MM, which was pretty much a clone of DEC-20 MM, was Columbia's primary email client starting in 1988. Version 0.90 (1990) added various "MM for dummies" features ("novice" mode, ?-help categorized by topic, etc) specified by a committee of Columbia bigwigs, and saw heavy service for some years but was gradually overtaken by Pine*, and then later by PC- and Web-based mail. Y2K and Y2K1 patches were added to MM in 1999-2001. I took over maintenence and development of it in 2002, shortly before Columbia was about to discontinue mailspool service, which would have left MM users high and dry. I did a fair amount of original work and also incorporated important changes and improvements from elsewhere, notably from Nelson Beebe at the University of Utah (who converted it to ANSI C [see notes] ) and Ken Harrenstien at Google (who improved the portability features [notes]).

In any case MM is now used mainly by touch-typing GUI-and-menu-hating die-hards like me, for reasons such as those mentioned here. It's worth noting, however, that (strange as it may seem today) prior to about 1995, everybody at Columbia used MM for email: students, faculty, researchers, administrators, librarians, office workers, cleaners, even the president. Tens of thousands of people. In those days a text-mode command-and-prompt user interface was not scary or quaint, it was how computers were used. Programs from that era tend to be more efficient, powerful, and stable than their modern counterparts, and they induce less repetitive strain injury. MM is still my primary mail client and with the changes described in this page, especially the Metamail interface, it is suprisingly useful even in 2014.
________________________________
The main problem with MM at Columbia in later years was that Columbia's Unix timesharing system (the "Cunix cluster") ran on a collection of hosts. When you logged in, you were assigned to a specific host by a load balancer and when you ran MM, it placed an NFS-wide kernel lock on your mailbox. If your session became disconnected and you logged in again, you probably would arrive at a different host and there was no way to clear the lock, e.g. by killing your previous session. So users had to contact the sysadmins to do it. With some 60,000 email users, this was clearly an "unsustainable" situation. It might have been possible to convert MM to file-based locks (which have their own set of problems), but by that time the writing was on the wall: users wanted to be able to access their email from laptops and cell phones as well as from the central timesharing systems so an IMAP client was required. At the time Pine was the quintessential IMAP client for Unix shell users.

New MM features since version 0.90.0

  1. BUG FIXES AND PORTABILITY
  2. READING MODERN MESSAGE TYPES IN MM
  3. DOWNLOADING MESSAGES TO YOUR DESKTOP
  4. RETRIEVING INCOMING MAIL FROM A POP3 SERVER

1. Bug Fixes and Portability

Columbia MM 0.91 (September 2002) incorporated all patches to 0.90 (notably the Year-2000 ones), fixed some other problems, and added a couple new features (downloading messages, handling of 8-bit message text). Like all MM versions, it is distributed together with Columbia CCMD (a TOPS-20 COMND JSYS simulator, also a Hermit Project product), upon which MM depends. Both are in source-code form, and must be built from source.

After the patches came numerous bug fixes and file reorganization from Ken Harrenstien, including new simplified makefiles and conversion of many files to the "new" (1990) OS-dependent (or feature-dependent) structure that started in version 0.90. Ken's changes are documented in HERE. I carried this forward a bit in getting the new version to build on Linux (with help from Chiaki Ishikawa and Adam Sampson), so now the Makefile automatically links config.h to the right OS-dependent config.h-blah file. Later, the mm and ccmd makefiles were replaced by new ones from Nelson Beebe.

The main "functional" bug fix to 0.91 is a lot of code to check for, protect against, and allow recovery from truncated outgoing messages upon return from the editor. Users will notice that .mm-xxx temporary files are no longer deleted immediately upon return from EMACS or whatever your editor is, so if "display" should reveal that some or all of the outgoing message text is missing, you can rescue it from the .mm-outgoing-pid file, whose name is announced to you each time you return from the editor. This feature is enabled only in Solaris and SunOS, because those are the only platforms where the problem was observed, but can be compiled in to any other platform too (-DFDC_EDITFIX). In any platform that has this feature, the new command:

SET USER-TEMP-FILES-KEEP { ON, OFF }

can be used to turn it on and off.

Columbia MM change summary.

[ Ken Harrenstien's update notes ] [ My update notes ] [ Nelson Beebe's notes ]   [ MM source code ]

2. Reading modern message types in MM

When MM was first written, email was just text, period. Each email message had a few header lines (From, To, Subject, Date, etc) and then a body that consisted of "bare" ASCII text with no encoding of any kind. Today's email is full of every imaginable kind of thing -- pictures, music, spreadsheets, movies, security keys, you name it -- that MM knows nothing about. Furthermore, ASCII is adequate only for English and a handful of other languages (Welsh, Dutch, Spanish without accents...), but now you might receive email written in any language at all, encoded in any one of hundreds or thousands of different character sets that MM doesn't understand, and then the message itself might be further rendered into some hideous "transport encoding" to protect it against hazards that don't exist any more (seriously, when was the last time email was routed through a card punch?).

When reading mail with MM in recent years I am more than likely see messages that look like this:

<div>set speed 9600</div><div>set stop-bits 1</div><div>set parity none</di=
v><div>set terminal bytesize 8</div><div>set carrier-watch off</div><div>se=
t flow-control rts/cts</div><div><br></div><div>local ON_CTRLC</div><div>
define ON_CTRLC {</div><div>=A0 =A0 =A0 =A0 echo Control-C</div><div>}</div=
><div><br></div><div>define \%x 0</div><div>while true {</div><div>=A0 =A0 =
=A0 =A0 input 10 \5</div><div>=A0 =A0 =A0 =A0 if ( SUCCESS ) {</div><div>=
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 increment \%x</div>
<div>=A0 =A0 =A0 =A0 }</div><div>=A0 =A0 =A0 =A0 echo { \%x - \v(instatus) =
}</div><div><br></div><div>}</div></div><div><br></div><div>The other scrip=
t is sender.ksc</div><div><br></div><div><div>cls</div><div>set modem type =
or even like this:
opaADtSUUUALSUUtACgAnnpT3aJFwE3Nj7xP9KjB5pG+aTA+lIfQe3ES5781HnsBTpOWwOg4poy3
0HU0DJY4tw3FgvpmoyeSM9O9OZjjGce1NVc9SAPU0DfZHovwzb/Q9SH/AE0jP6GtXxtaefoq3Cj5
7aQPn/ZPBrI+G6FINRYHchaPn3+auyvIFvLOe2b7sqFD+IpvYyekjzqykyBWzA4wK5u0LQyGGQkP
GxRh7g4rbgl4FczVmd8XdF6QblrlNeiAkVwOM4rqgdyVh63Dut2I6jmnB2kmKqrxZzKHDYPSpFQ7
yAeoIx+FRdCat25BGW25Ug5OOldp55ErbXweVb1qZWCN/wDWpskLKE46AqfqCR/SgB9gz2qriHtK
c53nIpGnT7wBz9KaUygJFSRwEqwPUUXFoL5rOOFNOWORlOWPA7VJFCRwcZHqakCRqwBJbHYU7iZW
wIxwTu7Yp+H+UsxVOeM1OUxkbQoHIA5qPcACAfoTTUmIh4Q7R9MVEytn1zUvlvIcLmraxKg+Y5am
wK6R+VH/ALR/Sqs4DnBOAO9WpXxkeoqIp+7HvyaTBEEcLg8EGgo+Dwaej7eMjjrT1fpxUorUj2OB
0PSm+W2fu96tBwx68ccU3cOT33etAiJYWKZqSOI557GpA2I+BSK+B+NAAUC4qF+c1ISc49qjYYya
ARQbOT6UlPmheGV4pAQwPT8MimVzs6EJmlpD60CkAtHFFFAxeKSig0AGM96Mc9aKMYoEGKXAFJmk

and often containing URLs that look like this:

http://www.btnp.org/index.php?option=3Dcom_content&amp;view=3Darticle&amp;i=
d=3D14=8&Itemid=3D525
and therefore can't be clicked on. All this kind of defeats the original purpose of email, especially when that which is being encoded is nothing more than Plain Old Text.

But no more! The following sections show you how to make MM handle every kind of text encoding so you can read every message that arrives, at least the textual parts.

2.1 Reading non-ASCII text in MM

The lowest common denominator encoding for computer text is US-ASCII (ASCII for short, "ass-key"), which is basically the Roman alpabet (letters A-Z, upper and lowercase), the digits 0-9, and a few common punctuation marks. Every email client can read and send ASCII text. But as noted, ASCII can't represent most languages, so it is increasingly common to receive email encoded in some other character set like Latin Alphabet 1 (for Western European languages), Latin Alphabet 2 (for Eastern Europen languages), Latin/Cyrillic (for Russian), or UNICODE UTF-8, which is a relatively new "Universal character set" that can represent all written languages. To mention only a few of hundreds or even thousands (see some other ones) (and some tables of their repertoires).

When an incoming email message is encoded in a character set different from the one you are using in your terminal emulator or window, you'll see nonsense for any non-ASCII characters, for example:

¿Ã?slenska / Icelandic: Ã?g get etið gler án þess að meiða mig.
instead of:
Íslenska / Icelandic: Ég get etið gler án þess að meiða mig.
There are two ways to deal with this:
  1. Change your terminal emulator or window to use the character set that is used in the message. Inconvenient because you might have to reconfigure your terminal for each message.
  2. Have MM automatically convert each message's character set to the one used by your terminal. The method for doing this is described (for the first time) below in Section 2.5.
Previously I had a long section here discussing Option 1, but there is no longer any reason to have to manually adjust your terminal emulator for every message. Keep reading!
MM itself does not do any kind of character-set conversion, which is good because this lets you have control, and therefore use any character sets you wish without interference. Although I had been using MM for decades, I always believed it could not compose or send 8-bit text messages, and therefore used some other mail client to send or reply to mail in Spanish, German, or Russian (etc) that was encoded as Latin-1, KOI8-R, UTF-8, ... It turns out MM can be used for this purpose after all if you compose your messages in an editor and not in MM itself, but some new headers had to be added for outgoing messages conform to MIME rules.

When used "bare", MM displays incoming messages literally and saves them literally into your mail file. It doesn't strip the 8th bit and it doesn't intepret the message content in any way. If an incoming message contains 8-bit data, MM normally sends it directly to your screen when you read it. For this data to be intelligible, you need a terminal emulator (or xterm window, console screen, etc) that either uses the same character set or knows how to convert from the message character set to the terminal character set.

I use MM on a Unix host through a Telnet or SSH session (or other kind of connection) from Kermit 95 on Windows, which understands nearly 100 different character sets: the ISO 8859 Latin Alphabets, the ISO 646 7-bit national sets, various Cyrillic KOI encodings, numerous PC code pages and other proprietary sets such as HP-Roman8 and DG International, as well as Unicode UTF-8, the latter including thousands of characters in dozens of scripts (Latin, Greek, Cyrillic, Hebrew, Japanese, you name it – even Runes).

Since most of the mail I read is in English, Spanish, German, or other West European languages, I would normally use ISO 8859-1 Latin Alphabet 1 as Kermit 95's terminal character-set. In practice this doesn't work out because of pollution of the Internet by Microsoft Code Pages such as CP1252 (the MS equivalent of Latin-1), which populate the control-character range with graphic characters like “smart quotes”, which, when received by a standards-compliant terminal emulator set to use the international standard ISO 8859-1 character set, results in disaster, the quote characters triggering control functions in the emulator (or real terminal). In short, in Kermit 95 I use the following setting:

set terminal character-set cp1252

which allows me to read both Latin-1 and Microsoft text, but disables certain control functions.

If I receive a message in some other character set, I have to change K95's set to match. To do this, I have to know the name of the character set to choose. A properly formatted e-mail message identifies its text character set in a header like this:

Content-Type: text/plain; charset=xxx

where xxx is a standard MIME character-set name, such as ISO-8859-1 or UTF-8. If a message arrives and looks like garbage, you can change Kermit's character set to match (you can do this on the K95 2.0 toolbar) and redisplay the message.

When using Kermit to access MM, you should be aware that Kermit's character-set names are not always the same as the names shown in the Content-Type header (because Kermit predates MIME). For example:

MIME Name Kermit Name Remarks
ISO-8859-1 Latin1 West European Latin
ISO-8859-2 Latin2 East European Latin
ISO-8859-3 Latin3 Mainly for Turkish
ISO-8859-4 Latin4 Mainly for Sami
ISO-8859-5 Cyrillic-ISO Cyrillic + ASCII
ISO-8859-6 Arabic-ISO Arabic + ASCII
ISO-8859-7 Greek-ISO Greek + ASCII
ISO-8859-8 Hebrew-ISO Hebrew + ASCII
ISO-8859-9 Latin5 ISO 8859-9 is Latin Alphabet 5
ISO-8859-15 Latin9 ISO 8859-15 is Latin Alphabet 9 (has Euro)
UTF-8 UTF8 Unicode UTF-8
NOTE: C-Kermit 9.0 accepts the MIME names, and the next release of Kermit 95 will do so as well. Click here to see the full equivalence table of Kermit and MIME names.

If I receive a message in Russian, I can still read it but I have to tell K95 to:

set terminal character-set iso-8859-5 (or KOI8-R, or whatever)

and then I have to put back the regular character set afterwards. By the same token, if I receive a message encoded in the new Universal Character, UNICODE, in its UTF-8 format, I can tell K95 to:

set terminal character-set utf-8

to be able to read the message. And then put it back to cp1252. As you can see, this becomes tedious. Hence the other method described below.

You can also change K95's character set in the K95 Toolbar, with your mouse.

What I have just suggested is setting your terminal emulator to use the same character set as the email, but since each message might have a different character set, a more flexible method is necessary, described below in Section 2.5, where each message is automatically converted to the character set used by your terminal.

2.2. Sending and replying to non-ASCII email

When you give MM a SEND or REPLY command, it normally enters "direct text input mode", collecting message-body characters straight from your keyboard. When it does this, it strips the 8th bit of each character. This is unfortunate but unavoidable; it occurs at such a low level in the command parser that it could not be changed without breaking everything else (believe me, I tried).

Luckily if you tell MM to:

set use-editor-always true

the command parser is bypassed during message composition and the 8th bit is not stripped. If you are replying to a message and including the original message, its 8-bit characters are preserved. You can enter 8-bit characters yourself using any method allowed by your editor and/or terminal emulator (e.g. Kermit's Compose key).

MM and EMACS
The method desribed here starts a fresh copy of EMACS each time you create or reply to a message. Nelson Beebe suggests another approach, which I didn't know about: to use 'emacslient', which is EMACS in a configuration where repeated invocations use the same instance of EMACS. This not only saves some wear and tear on the computer, but keeps your message files available even after you (seem to) exit EMACS, and also gives you access to any other files you might have been editing outside of MM when you compose an MM message. I haven't tried it myself yet, but you can find out about emacsclient with 'man emacsclient', or by looking it up in Google.

Then before sending the message, you should tell MM to include the headers that identify the character set so the recipient's mail client will know how to display the message. This requires a command that was added in MM 0.91:

content-type xxx

where xxx is a MIME content type (the default, if you give a CONTENT-TYPE command without an argument, is "text/plain; charset=ISO-8859-1"). When you give this command, MM writes out a full Content-Type header containing the text you supplied, e.g.:

Content-Type: text/plain; charset=UTF-8

When you give a CONTENT-TYPE command, MM also inserts two other headers that are required in MIME messages:

MIME-Version: 1.0
Content-Transfer-Encoding: 8bit

Should it be necessary for you to specify other values for these headings, you can do so with the following commands (also added in MM 0.91):

mime-version text
transfer-encoding text

The MIME-Version value would normally be "1.0" and the Transfer-Encoding value should be something like "8bit" or "7bit", but no verification is done. Note that MM does not do any kind of quoted-printable or Base64 transfer encoding, which are other possible values for this field.

If you send a lot of mail in a non-ASCII character set such as ISO 8859-1 or UTF-8, you can have MM include the needed headers in every outgoing message automatically by creating a file containing the desired headers and then telling MM to SET HEADER-OPTIONS-FILE filespec. For example, create the file .mm-header-options in your login directory, put the lines like the following in it:

MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit

and then put the following command in your .mminit file:

set header-options-file ~/.mm-header-options

Then, if you need to send a message in some other character-set you can override these options by giving a "content-type" command at the MM Send> prompt for that message.

MM and EMACS and Microsoft “Smart Quotes”
When you receive an email encoded in Microsoft Code Page 1252, which is Microsoft's Windows version of ISO 8859-1 Latin Alphabet 1 “extended” to use cutesy quotation marks by putting them in positions are reserved by ISO Standards 8859 and 2022 for control characters, thus wreaking havoc all over the Internet... When you receive one of these messages, even if your terminal emulator is using CP1252 as its encoding, and then you go to reply to the message (e.g. "reply sender including"), EMACS does not know what to do with these abominations and you will see a screen full of octal escapes like:
I have a question for you\205I\221m looking over Qatar addresses and am wondering what the \223PH\224 could stand for\205
Maybe you can tell EMACS to switch its encoding to CP1252, but the approach I prefer is to convert them to standard encoding. Put this in your .emacs file:

(defun myreplace (s1 s2)
  "Replace s1 with s2 in current buffer"
  (interactive)
  (beginning-of-buffer)
  (while (search-forward s1 nil t)
    (replace-match s2 nil t)))

(defun unsmart-quotes ()
  "Replace Microsoft punctuation with ASCII"
  (interactive)
  (myreplace "\205" "...")
  (myreplace "\223" "\042")
  (myreplace "\224" "\042")
  (myreplace "\227" "--")
  (myreplace "\221" "\047")
  (myreplace "\205" "..."))

Then whenever you see octal escapes in your MM-Outgoing buffer, just tell EMACS to ESC-x unsmart-quotes.

2.3. Using MM to Send Email in HTML Format

It's a little clunky but you can do it. Just make sure the message body is legal HTML, e.g.:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html><head><title>Blah blah</title>
<META http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
</head><body>
(Text goes here with any desired markup)
</body>
</html>

(In this example I'm assuming you're including 8-bit characters from ISO 8859-1; if you are using some other character set that is not ASCII, you should substitute its MIME name.) Then tell the content type -- html encoded in such-and-such a character set:

content-type text/html; charset=ISO-8859-1

The content-type command automatically adds the following headers:

MIME-Version: 1.0
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit

Or (what I usually do)... Just keep an HTML message template called, say, msg.eml that you can edit any time you want to send an HTML message. Here's a sample template. Note the blank line separating the headers from the message body; the blank line is essential:

To: fill this in
Subject: fill this in
bcc: .
MIME-Version: 1.0
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html><head><title>
fill this in - same as Subject
</title>
<META http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
</head><body>
Text goes here with any desired markup
</body>
</html>

Edit the file into the desired message and then tell MM to:

restore-draft msg.eml

For reading HTML mail, see the next sections.

2.4. Attachments

MM doesn't do attachments. You can't send them, and it doesn't decode them when receiving email (unless you have set up Metamail [next section] as MM's "crt-filter"). This way you never get any nasty surprises (who was it who thought it would be a good idea that I could send you an email containing code that would execute on your computer???)

To send email with attachments, use Pine (or any other modern mail client).

To decode incoming mail with attachments, you can:

  1. Use Metamail (next section) as MM's CRT or MIME filter; or:
  2. Use MM's COPY command to save the message to a file, then run the file through Metamail; or:
  3. Download the message using MM's DOWNLOAD command (Section 3) and look at it in your desktop mail client.
For downloading, Windows users should give the downloaded message a filename that ends with .eml, which is associated with the Windows email reader. The default filename for the DOWNLOAD command is mm.eml. If your terminal emulator is Kermit 95, escape back to it (Alt-x) and tell it to "run start name.eml", and a graphical version of the message pops up on your screen. A handy macro to have defined in K95 would be:
define z run start \v(filename)
(Kermit's \v(filename) variable contains the name of the file most recently transferred).

Windows 7 and 8 (and probably later) do not come with an .eml file viewer, which was included in all earlier Windows versions. To be able to view E-Mail messages in Windows 7 and later, you have download Windows Live Mail from Microsoft (it's free) and install it.

2.5. Encoded HTML Mail

By 2014, 30 years after Columbia MM was begun and probably forty years after TOPS-20 MM first appeared, much – if not most – email is arriving in HTML format. Not only that, the HTML is encoded (entirely unnecessarily) as "quoted printable" or even Base 64. Section 3 tells how you can have MM export a message to your desktop mail client for viewing, but that's a bit inconvenient if you have to do for 75% of your messages. Here's a shortcut that works pretty adequately for HMTL messages:
  1. Eyeball the message to see if there is some chance it is not junk or spam. Most HTML messages are unwanted promotions, phishing, etc. Just delete them.
  2. To see an intelligible version of the text of the message, just give a PRINT command at the MM's Read> prompt.
This requires that in your .mminit file you have set your print-filter like so:

set print-filter metamail | lynx -stdin -dump | more
That is, run the message through metamail and pipe the result the lynx (the text-mode Web client), and to the 'more' pager. This technique is a conservative subset of the more general technique described in the next section.

2.5. Using Metamail/Mailcap in MM to deal with message parts and content types

This section is new for 2014. It presents a way to use a feature of MM (the "set crt-filter" command) that was always there together with an external program, Metamail, to solve all the problems with incoming MIME and multipart mail, or better still to use a brand-new feature "set mime-filter" to achieve the same result without undesirable side effects.

Metamail is a program written at Bell Communications Research 1991 and apparently not touched since then that offers an extensible framework for handling each part of a multipart MIME message according to its MIME type – plain text, HTML, image, etc – by following rules that you put in your ~/.mailcap file. These rules should work uniformly with all text-based mail clients that use Metamail to display messages.

Metamail takes care of each "content-type" that appears in an incoming MIME message according to the commands that you put in your Mailcap file to associate each content type with a command or program that knows how to display it on your screen; "man metamail" and "man mailcap" for further information. HINT: you might want to put some Metamail configurations in your Unix profile (see man page), e.g.:

export METAMAIL_TMPDIR=.
export MM_NOASK=1
export MM_QUIET=1
export KEYHEADS="Date:From:Subject:To:CC:bcc:Content-Type"
The KEYHEADS one is needed for telling Metamail which mail headers you want to see.

The best thing about Metamail is that it undoes all the gross "quoted printable" and "Base 64" encodings that make most incoming text ugly or unreadable for no good reason whatsoever, before handing the message (or its parts) off to the helpers you have specified in your .mailcap file. And also that it manages the Multipart/alternative scheme in which the same text is presented in several ways, giving you only one copy of it, instead of multiple redundant copies.

MM 0.96.0 with explicit MIME support

MM version 0.96.0 (February-March 2014) lets you specify a filter to use only for email messages, as distinct from all other types of text that MM displays:
set mime-filter programname
This is just like MM's crt-filter but it applies only to email messages, when you give the READ, TYPE, or DISPLAY command. Except when you precede READ, TYPE, or DISPLAY by the word LITERAL, which case the actual complete, full, encoded email message is shown literally. So for MM 0.96, you would put something like this in your .mminit file (bash syntax, other shells can vary):
set crt-filter more
set mime-filter metamail 2> /dev/null | more
In the example, 'more' and 'metamail' are programs that should be runnable by you just by typing their names, i.e. they are in your PATH; if they are not, you can put their full path:
set crt-filter /usr/bin/more
set mime-filter /usr/local/bin/metamail 2> /dev/null | more
The CRT filter (CRT means Cathode Ray Tube, the main component of a 1970s-80s video terminal) is used for all text that is not an email message. The MIME filter is used only for email messages. The "2> /dev/null" part redirects stderr to the null device ("bit bucket"); that is, it hides the error messages that happen when you quit out of 'more' before the end, such as this one (while reading an HTML-format message):
Broken Pipe
Command failed:  lynx -dump -assume_charset='UTF-8'
-display_charset='iso-8859-1' -force_html './mm..4aWAm'
If the "2>" notation is not understood by your Unix shell you can use whatever its redirection syntax is, or just live with the error messages:
set crt-filter more
set mime-filter metamail | more

Using Metamail in older MM versions

In MM 0.95 and earlier, there is no MIME filter, but you can define MM's CRT filter to be Metamail:

set crt-filter metamail | more
set use-crt-filter-always true  ← required for Metamail

The drawback here is that the CRT filter operates on all text that MM displays, not just email messages. But Metamail only works on real email messages. If you feed it any other kind of text, it says it can't find the mail headers; all you see is an error message. The result is that any text produced by non-mail reading commands such as HEADERS, HELP, or SHOW can not be seen:

MM>show
metamail: Could not find end of mail headers.
MM>

Configuring Metamail with the .mailcap file

Once you have told MM to use Metamail, you have to tell Metamail how you want it to handle your email. But first check if your computer already has an adequate setup. You can test for this by telling MM to
set mime-filter metamail | more
(MM 0.96 or later; use crt-filter for 0.95 or earlier) and then reading some mail. If you don't like the results, you can make your own .mailcap file (it goes in your login directory, ”man mailcap” for details). Here's the one I use:
text/html; lynx -dump -assume_charset=%{charset} \
 -display_charset='iso-8859-1' -force_html '%s' ; nametemplate=%s.html
text/*;  xlate %s %{charset} iso-8859-1
application/pdf; ( echo "[FROM PDF]" \; pdftotext %s - ) ; nametemplate=%s.pdf
application/msword; ( echo "[FROM MSWORD]" \; catdoc %s ) ; nametemplate=%s.doc
audio/*; echo [SKIPPING %t]
image/*; echo [SKIPPING %t]
application/*;  echo [SKIPPING %t]
This says I want to see text files and HTML files (without markup), and also if somebody sends me a Microsoft Word or a PDF document, I want to see the text from it, but I don't want to see anything else because I'm using what amounts to a text terminal: a Unix shell session; not a GUI, not an X window. No temporary files are left behind (as Metamail is normally wont to do, filling up your disk over time behind your back). If you normally use MM in an X Window, of course you can make Metamail rules that display images in a separate window on your screen, instead of discarding them. Let's look at each rule:

text/html; lynx -dump -assume_charset=%{charset} \
-display_charset='iso-8859-1' -force_html '%s' ; nametemplate=%s.html
This rule (continued onto a second line because it's too long to fit across a normal-width web page) is for messages in HTML (Web page) format. It sends them to Lynx (a plain-text Web browser, another choice would be w3m) for interpretation and also converts the Web-page character set to the character set my terminal uses, which happens to be ISO-8859-1**, even if the HTML document fails to announce its character set, in which case the character-set announced in the message's Content-type is used. man lynx for details about Lynx.

text/*; xlate %s %{charset} iso-8859-1
This rule converts plain text files ("text/plain" or "text/anything-else-except-html"), whose character-sets are identified, into my terminal character set. So, for example, if Spanish or German text arrives in UTF-8 (or IBM Code Page 437 or anything else) I will see it rendered correctly on my screen. The conversion program is a short Kermit "kerbang" script***, most of which is just argument-checking:
#!/usr/local/bin/kermit +
.default = iso-8859-1
if not def \%1 exit 1 "Usage: xlate filename cset1 cset2"
if not exist \%1 exit 1 "xlate: File \%1 not found"
if not def \%2 .\%2 = \m(default)
if not def \%3 .\%3 = \m(default)
if ( equ \%2 \m(default) || equ \%2 us-ascii ) { type /nopage \%1, exit 0 }
xlate \%1 \%2 \%3
if fail type /nopage \%1
exit 0
This script should be stored somewhere in your PATH, given eXecute permission (chmod +x), and the top line changed (if necessary) to reflect the full path of C-Kermit on your computer.

You could also use ' iconv' or 'recode' or other character-set conversion program but they tend to be less forgiving in odd situations, e.g. when the message does not announce its character set.

application/msword; ( echo "[FROM MSWORD]" \; catdoc %s ) ; nametemplate=%s.doc
This rule uses a program called "catdoc" to extract the text from a Microsoft Word file.

application/pdf; ( echo "[FROM PDF]" \; pdftotext %s - ) ; nametemplate=%s.pdf
This rule uses "pdftotext" to display the text (if any) from a PDF file.

audio/*; echo [SKIPPING %t]
image/*; echo [SKIPPING %t]
application/*; echo [SKIPPING %t]
These rules tell Metamail to just print a message (like “[SKIPPING image/jpg]”) for any content type that can't be shown as plain text, effectively discarding all other application-specific and multimedia attachments without storing them in temporary files. If I care about seeing and/or hearing these attachments I can always download the message to my desktop and see them there (next section).

You can add other rules like this for other content types that you have converters for.

The legible message display accomplished through Metamail and its helpers occurs only on your screen, while READing or DISPLAYing a message in MM. If you REPLY { SENDER, ALL } INCLUDING, the editor will be loaded with the original, undecoded message. Ditto for DOWNLOAD, COPY, MOVE, WRITE, etc. Note however that you can COPY any message to an external file and then run it through metamail again, outside of MM, to make a legible copy on disk:

MM Read> copy message1.txt
MM Read>
MM>exit
$ metamail message1.txt > message1-decoded.txt

Conversely, suppose you want to see what the message really looks like, without running it through Metamail. One way is to tell MM to EDIT the message. A more natural way, available only in MM 0.96 and up, is to LITERAL TYPE the message (this works in the new MM because TYPE and READ use the mime-filter, but LITERAL TYPE uses the crt-filter). Alternatively if you prefer to eyeball each message before seeing a legible version of it, you can set up MM like this:

set crt-filter more
set print-filter metamail | more
This way you only "print" the messages you actually want to read**** (and, presumably, delete the other ones). You can do this in any version of MM, but of course only if you don't already use MM's PRINT command for actually sending messages to a printer.

Problems

  1. REPLY SENDER (or ALL) INCLUDING includes the entire undecoded uninterpreted message in the reply, not what Metamail shows you. When the original message is buried in some transfer encoding or includes huge attachments, it might be better to just say REPLY (without INCLUDING) and use your mouse to copy and paste the desired text.

  2. When displaying a list of message headers, notably Subject: and From:, MM does not interpret any MIME encodings in them, for example:
    Subject: =?utf-8?Q?=DB=B7=E1=BC=B6=E1=B8=80=C9=A0=E1=B9=99=E1=B8=80?=
    The text is shown correctly, however, when you READ or TYPE the message.

  3. When composing messages to send, if you put non-ASCII characters in the Subject or any other header, MM doesn't encode them in any way. (The only way to put non-ASCII characters in the headers is by editing them in a text editor; if you "use-editor-always" [section 2.2] you always have the opportunity to do this.)

  4. Incoming messages that that have non-ASCII characters (or more accurately said, bytes with their high-order bit set to one) in their headers have all those bytes converted to 'X'. This has been a feature of MM since Day 1.

  5. Sometimes Metamail fails to fully decode quoted-printable or base64 message bodies or parts. This might happen because of badly formed Content-type headers, incorrect punctuation or quoting by the originator of the message. But it sometimes also happens with messages that are perfectly correct, and it happens consistently with certain messages. MM is correctly passing these messages to Metamail, but Metamail is not doing its full job. For example, in a certain message it converts the character set but does not undo the Quoted-Printable transfer encoding, even though the headers say:
    Content-Type: text/plain; charset="utf-8"
    MIME-Version: 1.0
    Content-Transfer-Encoding: quoted-printable
    Content-Description: Mail message body
    
    Yet, if the same message is run through Metamail outside of MM (using the same .mailcap), it is fully and correctly interpreted.

Some of these problems might be worth another release. Another possibility for a new release might be an ATTACH command, to add one or more attachments to a message body.
________________________________
You can invoke metamail with the -p option to have it do its own paging, or you can pipe commands through 'more' (or 'less') in your mailcap rules, but either way, more than one screenful can be displayed without pausing sometimes. This happens because metamail does one piece at a time, and the paging is applied separately to each piece; if a piece is short, it doesn't pause at the end, and then next piece starts the line count over again, not knowing about the previous piece; for example, between the message headers and the first content part. Piping Metamail itself through 'more' seems to do the trick.
**  I could just as easily translate to UTF-8 and that way I could also see Russian, Arabic, Hebrew, Mongolian, etc, on my screen, but I don't use UTF-8 in my shell sessions because so far many of the tools I use can't deal with UTF-8. By the way, Kermit doesn't know every character-set on earth, so if you see a message like:
?No keywords match - GBK
?Invalid: xlate \%1 \%2 \%3
File: ~fdc/bin/xlate, Line: 8
and you care about the indicated character set, you'll need to use iconv or recode instead of the xlate script.
***  If necessary, change the first line to show the actual path of the C-Kermit executable on your computer. The script should work with any C-Kermit version 8.0 or later.
****  This would be the paranoid setup. However, having every message decoded automatically turns out to be pretty safe. For example, in HTML messages decoded by Lynx, you see the actual URLs, not the spoofed ones that you would see if you used a graphical Web browser to interpret the message.

2.6. Modern MM configuation summary

For reference here are all the configurations items mentioned in Section 2 of this document. These are the values that I use, you might want or need to use different ones; for example, "utf-8" instead of "iso-8859-1", or 'iconv' or 'recode instead of 'xlate'.
Your Unix shell profile (e.g. ~/.bash_profile)
export METAMAIL_TMPDIR=.
export MM_NOASK=1
export MM_QUIET=1
export KEYHEADS="Date:From:Subject:To:CC:bcc:Content-Type"
~/.mminit (for MM 0.91 or 0.95)
set use-editor-always true
set header-options-file ~/.mm-header-options
set crt-filter metamail 2> /dev/null | more
set use-crt-filter-always true
set print-filter more (for viewing messages literally)
~/.mminit (for MM 0.96)
set use-editor-always true
set header-options-file ~/.mm-header-options
set mime-filter metamail 2> /dev/null | more
set crt-filter more
~/.mm-header-options
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
~/.mailcap
text/html; lynx -dump -assume_charset=%{charset} \
 -display_charset='iso-8859-1' -force_html '%s' ; nametemplate=%s.html
text/*;  xlate %s %{charset} iso-8859-1
application/pdf; ( echo "[FROM PDF]" \; pdftotext %s - ) ; nametemplate=%s.pdf
application/msword; ( echo "[FROM MSWORD]" \; catdoc %s ) ; nametemplate=%s.doc
audio/*; echo [SKIPPING %t]
image/*; echo [SKIPPING %t]
application/*;  echo [SKIPPING %t]

3. Downloading Messages to Your Desktop

Sometimes you receive a message full of attachments that MM can't show you, not even through Metamail, such as JPG images, or video files. You have verified by looking at it in MM that the message is genuine, and you want to see it as intended. For this purpose, version 0.91 of MM added the following new commands for downloading messages to your desktop computer so you can view them in a GUI mail client (or use them for any other purpose that occurs to you):

SET DOWNLOAD-FILTER [ text ]
Sets the download filter (program) to the one given by "text". This program is run when you give a DOWNLOAD command. The default download-filter is

kermit -Ts - -a %s

that is, send standard input (which is fed to Kermit by MM) in text mode under the name given by the replacement string; the default replacement string is "mm.eml".

The download program must be able to send from standard input (as Kermit does with its "-s -" command-line option). XYZMODEM programs (e.g. Omen rzsz or GNU lrzsz) don't seem to allow this, but you can work around them as follows:

set download-filter cat > %s; sz -a %s; rm %s

You can have up to four %s's in the download-filter; each one is replaced by the same string.

DOWNLOAD [ /FILENAME:name ]
In Read or Send mode, downloads the current message under the given name; if you don't give a name "mm.eml" is used. Headers are treated according MM's DONT-PRINT-HEADERS and ONLY-PRINT-HEADERS settings. If you specify a filename, and if MM's DOWNLOAD-FILTER contains "%s", the "%s" is replaced by the given name. So for example, with the default download filter ("kermit -Ts - -a %s"):

DOWNLOAD (by itself):
Sends the message as "mm.eml" ("kermit -Ts - -a mm.eml").

DOWNLOAD /FILENAME:foo.bar
Sends the message as "foo.bar" ("kermit -Ts - -a foo.bar").

DOWNLOAD /FILENAME:~/Mail/msg.txt
The filename can include a path, which will be used if the file-transfer software knows what to do with it (Kermit does).

You should be able to include almost any printable character except Space in the filename; however, backslashes must be doubled:

DOWNLOAD /FILE:c:\\some_directory\\some_subdirectory\\some_file.txt

But note that when downloading to Kermit 95, you can use forward slash as the directory separator to avoid the ugliness:

DOWNLOAD /FILE:c:/some_directory/some_subdirectory/some_file.txt

DOWNLOAD [ /SEPARATE-PAGES ] [ /FILENAME:name ] [ message-sequence ]
At top level, downloads the given message sequence (default = current message). /FILENAME: works as in Read Mode. If the /SEPARATE-PAGES switch is included, multiple messages are separated by formfeeds. If you specify a message sequence indicating more than one message, the entire sequence is downloaded as one file.
You can also prefix the DOWNLOAD command with LITERAL (as with TYPE, PRINT):

LITERAL DOWNLOAD [ /SEPARATE-PAGES ] [ /FILENAME:name ] [ message-sequence ]
Includes all the message headers, regardless of DONT-PRINT-HEADERS and ONLY-PRINT-HEADERS settings.

When accessing MM from any reasonably modern version of Kermit (such as Kermit 95 on Windows or C-Kermit 7.0 or later on Unix), Kermit's "autodownload" feature kicks in (yes) automatically – as soon as the MM's download-filter program starts sending with either Kermit or Zmodem protocol, the terminal program switches to file-reception mode, and then back to terminal mode when the transfer is complete. Thus the DOWNLOAD command does the entire job – no "escaping back" or re-connecting is needed.

If you are accessing MM from Kermit 95 on Windows, you can download MIME messages that contain enclosures directly into your Windows mail agent. This can be dangerous unless you know what you're doing and have inspected the message carefully in MM to ensure its payload is not lethal (further explanation is beyond the scope of this document). Tell MM to:

DOWNLOAD /FILE:xxx.eml

Then escape back to the K-95> prompt and tell K95 to:

RUN START xxx.eml

This tells Windows to start whatever program is associated with the .EML (e-mail) extension (Outlook, whatever) and feed the xxx.eml file to it so you can experience it in all its multimedia glory. The fact that you saw it in MM first allowed you to prescreen it for safety, an opportunity you would not have if you used a PC-based mail client to begin with. HINT: Define a macro in Kermit 95:

define z run start \v(filename)

and then just type "z" at the K-95> prompt to read the message. In fact this works with any kind of file that you download, for which Windows has an associated application (.xls, .doc, .pdf, etc). \v(filename) is a built-in Kermit variable that contains the pathname of the file most recently downloaded.

There is even a way to have Kermit 95, upon receiving a file, automatically start the associated application with the just-received file loaded into it (email, HTML, a JPG image, an MS Word file, a PDF file for Acrobat Viewer, an Excel spreadsheet, etc). Obviously this would be risky so I won't explain it unless somebody asks me.

4. Retrieving Incoming Mail from a POP3 Server

Beta.03
2006/01/09
File locking reinstated, using the same kind of kernel locks as before. This is not final but it's better than no locking at all; i.e. it's better to protect the mail file from corruption than it is to be locked out of it by an orphaned MM process. If it turns out that no other mail client uses the same locking convention, maybe we can convert to file-based locks, which users can clean up without divine intervention. Also: the REMAIL command was fixed to not rewrite the From: address to be the remailer's address, and to write the Resent-Date correctly. Also: some files were renamed to allow MM to built on case-insensitive file systems like Mac OS X.
Beta.02
2005/12/07
MM's version number took a big leap from 0.91 to 0.95 today after I integrated Nelson Beebe's University of Utah version 0.94 with my version 0.91; 0.91 + 0.94 = 0.95. The benefit is that this version builds not only on Solaris but also Linux (I just tried it) and presumably also all the other platforms (such as Mac OS X, FreeBSD, HP-UX, etc) where Nelson claims it builds and runs. I tested the Solaris and Linux versions with Methods 0 through 2 and all seem to work fine. IF ANYBODY AT COLUMBIA IS INTERESTED IN TESTING IT ON LINUX, MAC OS X, OR OTHER PLATFORMS, PLEASE LET ME KNOW and I'll set up access to the binaries and scripts. Note: as an artifact of today's work, MM no longer locks the mail file; this will be addressed in some fashion in a subsequent edit.
Beta.01
2005/12/04
POP interface added. The actual POP interface is external, like movemail, but MM needed a lot of adaptation to the idea that mail is coming from a (possibly slow and overloaded) network server, rather than from a locally visible spool file.

This section was written at Columbia University in 2005-06 before I was laid off in 2011, mainly for Columbia readers, and therefore contains many words like us and our that no longer apply. The POP3 feature, however, can be used anywhere that offers POP3 mail delivery, either in clear text or secured by SSL/TLS.

Columbia MM is a text-based email client that runs on Columbia's central Cunix machines (Unix-based multiuser timesharing systems). In some ways it is similar to Pine, in others not so much. Columbia MM is an adaptation of the original ARPANET MM program that was used for email on our much-loved DEC-20s (1977-88). Columbia MM was written to give our users the same familiar email client when they migrated from the DEC-20s to Unix; it was the near-universal email client at Columbia until the mid-1990s (IBM mainfame users had another home-grown MM clone, VMM, written as a Wylbur Exec).

At Columbia, many of us have been using MM for decades on our central Unix systems without bothering much about the changing landscape. MM uses the traditional Unix-based mail delivery system, in which each user's incoming mail is deposited by the mail server in a "spool file", such as /var/spool/mail/f/d/fdc, which MM checks from time to time. When it finds new messages, it moves them to your own mail file, where you can read them, reply to them, delete them, etc.

The Problem

All that is about to change.* As users increasingly move away from Cunix based mail to a more distributed environment in which mail can be read using clients on PCs, Macintoshses, workstations in labs, Blackberrys, iPods, cell phones, etc, the model for delivering mail has changed. Since any given user might want to access his or her mail from different computers at different times, the mail is no longer delivered to the user's computer, but rather, kept on a central server, which is accessed by a protocol such as IMAP. Pine also uses IMAP; that's why you can access your mail with Pine on Cunix, and then access the same mail from your laptop using some other mail client, bypassing Cunix altogether.

MM doesn't follow this model and neither do its users. But the number of diehard MM users is dwindling to the point where it is no longer practical to support the traditional mail delivery method supported by MM alongside the new one demanded by everybody else. The issue came to a head when demand for our mailservers outstripped their capacity, forcing a radical change in mail delivery: a switch to a higher-performance multi-server design from Carnegie Mellon University called Cyrus. The new servers and software have been running at Columbia for some months, alongside the traditional mail spool. Each user is on one system or another. Users are moved in groups from the old system to the new one. The cutover is proceeding rapidly and will be complete in early 2006. Once the cutover is complete, the traditional mail spool will be decomissioned.

_________________________
This document was written before the cutover. In fact all this has changed as of late 2006, when the cutover was completed, and I have been using MM exclusively as my email client, served by POP as described below, ever since.

The Solution

Although it might be possible to change MM into an IMAP client, this would be a lot of work for a very small clientele. Anyway, MM is extremely stable. That's one of the reasons we like it so much, so let's not break it. To keep MM viable, we need a way to connect it to Cyrus. That way is an older server-based mail delivery protocol called POP, or Post Office Protocol, which, like IMAP, is supported by Cyrus. But unlike IMAP, POP moves your incoming mail to the computer you are using, rather than holding on to it and letting you manage it on your server-based "mailbox". But MM wants your email to be delivered to the computer you are using, so POP is just the ticket.

An interface between MM and the POP server has been written using Columbia's other well-known tool, Kermit, and is presently ready for testing (a new version of Kermit is needed for this). But unfortunately, migration is not completely transparent because (a) POP (like IMAP) requires a login, and (b) POP (like IMAP) can be slower – sometimes a lot slower – than copying a file out of the mail spool.

If the new POP script were simply installed as a replacement for MM's movemail utility, you would have to type your password every single time MM checked for new mail. Furthermore, MM itself could "hang" for minutes at a time as the POP server is being accessed. (Supposedly this will not happen with Cyrus, but all resources are finite and can be overloaded.)

One of the reasons Kermit was chosen as the scripting language is that Kermit can make the kind of secure connection – in this case TLS (Transport Layer Security) – required by the server. The connection is encrypted and should be well protected from any kind of attack. Thus, when you are prompted for a password to be sent to the POP server, your user ID and password are sent on the encrypted channel, not in the clear.

Instructions for testing the new version follow. NOTE: During the development stage, messages are issued from MM and the POP script so we can watch what's happening. Later, these can be removed.

If you have problems or suggestions, send me email at fdc@columbia.edu.

[ C-Kermit Daily Build ]   [ Kermit POP Script ]   [ New MM Source Code ]

Method 1: Using the POP Script from within MM

This method is similar to how MM has always worked, with the POP script replacing the movemail program. MM checks for new mail itself at various times, and loads it if it finds any. Advantages: familiarity, ease of use. Disadvantage: If the POP server is overloaded, you might experience long pauses while MM tries to contact the POP server or download new messages. This will happen even when you don't expect it; MM checks for new mail at all sorts of times. When this involves simply looking at a local spool file, it's instantaneous, but not when it involves logging into the POP server.

Setting Up Method 1

  1. Make sure that MM is using "mbox" mail-file format ("set default-mail-type mbox").

  2. To use MM with the pop server, you have to put the following command in your ~/.mminit file:

    set movemail-path /p/kd/fdc/mm-pop/pop
    

  3. Create an .mm-use-pop file in your login directory, e.g.:

    touch ~/.mm-use-pop
    

    it doesn't matter what, if anything, is in the .mm-use-pop file. If it is present at all, it tells MM that it will be using a POP server, rather than local spool files.

Testing Method 1

  1. Add /p/kd/fdc/mm-pop/ to your PATH, ahead of everything else:

    export PATH=/p/kd/fdc/mm-pop/:$PATH
    

    This directory contains my working copy of C-Kermit, MM, and the two scripts. You can type this command at the shell prompt for temporary effect, or put it in your .bash_profile for permanent effect.

  2. Start MM in the normal way (type "mm" at the shell prompt).

  3. The first time MM needs to contact the POP server it prompts you for your password; enter your regular login password. It is kept in MM's memory for use each time it invokes the POP script, so you don't have to type it over and over again. NOTE: If you supply an incorrect password, the POP script won't be able to log in to the POP server. You'll have to exit MM and start it again in order to be prompted again for your password.

Now, whenever MM would have checked your mail spool, it logs in to the POP server instead.

NOTE: If you change your password after you start MM, you will need to exit from MM, start it again, and give it the new password.

Later, when you are migrated to Cyrus, a ~/.cyrustt file will appear in your home directory as part of the migration process, and MM and the POP script will automatically switch to the Cyrus server. Don't touch this file; if you remove it or rename it, you won't be able to read new mail at all using MM or Pine.

You can adjust the interval at which MM contacts the POP server with MM's SET CHECK-INTERVAL n command (where n is the number of seconds between checks). The default is 300 (5 minutes). As a good network citizen, you shouldn't make it much less than that, but in any case MM will not contact the server more than once in 30 seconds.

Using MM in this mode is not bad at all when the POP server is responsive; otherwise it can be distinctly unpleasant. That's why there's also a Method 2.

By the way, you can also run the pop script standalone. In that case it will prompt you for your password and then it will download new messages a file in your home directory called .pop_yyyymmdd_secondssincemidnight_pid.txt.

[ View the pop script ]   [ Back Out ]

Method 2: Running the POP Script in a Separate Window

This is an entirely new way of using MM that might seem a bit awkward, but you'll like it better at times when the POP server is slow. Disadvantages: You need two Cunix login windows. Advantages: MM will not pause for long periods waiting for the POP server to respond; you will have a window in which you see new mail notifications, without having to ask MM to check. After a week of using MM both ways, I think this one is far more pleasant and less disruptive.

Setting Up Method 2

  1. Make sure that MM is using "mbox" mail-file format ("set default-mail-type mbox").

  2. Put the following command in your ~/.mminit file so that MM will not try to move mail itself:

    set movemail-path
    

    That is, give an empty movemail path.

  3. Remove any .mm-use-pop file from your login directory:

    rm -f ~/.mm-use-pop
    

Testing Method 2

  1. As with Method 1, add the p/kd/fdc/mm-pop/ directory to your PATH, ahead of everything else:

    export PATH=/p/kd/fdc/mm-pop/:$PATH
    

  2. Log in to Cunix and start the mailcheck script:

    mailcheck
    

    which prompts for your password when you start it, and then checks the POP server periodically for new mail without your having to touch it again. Leave this window alone until you are ready to log out.

  3. In another window, log in to a separate Cunix session. This is where you run MM and do any other work you want to do there. Whenever new mail arrives, the POP script moves it to your ~/.mm-newmail file. Whenever ~/.mm-newmail appears, MM loads it automatically.

  4. When you are ready to log out, go back to the mailcheck window, interrupt the script with Ctrl-C, then log out. Then log out from your other Cunix windows.

NOTE: If you change your password after you start the mailcheck script, you will need Ctrl-C out of the script, start it again, and give it the new password.

[ View the mailcheck script ]   [ Back Out ]

Method 0: Traditional Direct Mailspool Delivery

During the transition phase to Cyrus, mail is still delivered in the traditional way to those who have not yet been moved to Cyrus. New mail is deposited in each user's mail spool, such as /var/spool/mail/a/b/abc123. If you have configured MM to use Methods 1 or 2 above and then wish to return to regular delivery, follow these steps:

  1. Remove all SET MOVEMAIL-PATH commands from your ~/.mminit file (or comment them out with '#').

  2. Remove any .mm-use-pop file from your login directory:   rm -f ~/.mm-use-pop

You don't have to change your PATH; the new MM should work fine with traditional mailspool delivery if you follow these steps. However, in case the test version has a bug and you want to run the regular version of MM instead of the test version, remove the /p/kd/fdc/mm-pop/ directory from your PATH in your ~./bash_profile (if you put it there), or at the shell prompt type (or paste):

export PATH=`echo $PATH | sed -e "s|/p/kd/fdc/mm-pop/:||"`

NOTE: When you log in to the POP server, if you have mail waiting in the spool, the POP server transfers it to its own space. From that moment, the only way to get this new mail is with a POP (or, presumably, IMAP) client.

MS: The University of Washington MM-Like IMAP Client

There is now an alternative to MM for those who prefer IMAP to POP; that is for those who prefer to have their email kept on a central server, accessible from many different clients and locations.

Mark Crispin, one of the original ARPANET MM authors, has a version of MM called MS. It uses Columbia CCMD (the DEC20-like command parser), but MS itself was written from scratch. It looks like ARPANET MM. Mark used it, along with a companion program called MailManager for the NeXT, for a few years prior to switching to Pine.

MS is simpler than MM; some people might like it for that reason alone (for example, there is no SET command and no options). Others will like it because, as an IMAP client, it won't have any problems with locked mailboxes as we have had with MM. MS uses the same c-client library that Pine uses, so MS and Pine are completely compatible and MS can access all the same types of protocols and mailbox formats that Pine can.

With Mark's help, I have configured and built MS to work at Columbia with our imap.columbia.edu server. I'm not sure yet how the Cyrus cutover will be managed; I'll probably have to change the code to look for ~./cyrustt, as has been done here with Pine and MM. Anyway, as of 5 Dec 2005, you can use MS to send mail and to read and manage your mail on the IMAP server. I have put a copy of MS in the same directory as the test version of MM, so if you followed the instructions above about setting your PATH, you can run it by typing "ms" at the shell prompt. If not, just do this:

export PATH=/p/kd/fdc/mm-pop/:$PATH

or this:

alias "ms=/p/kd/fdc/mm-pop/ms"

There is a short Unix "man page" for MS HERE. Anyway MM users should find it familiar enough. The same prompts, most of the same commands and options, the same ?-help and Tab-completion features, etc. But using IMAP itself is a bit different. To access your IMAP mailbox (the parts you type are shown in red):

MS>get {imap.columbia.edu/norsh}INBOX
[Trying IP address [128.59.48.1]]
{imap-vif.cc.columbia.edu/imap} username: fdc
Password:           
[[UNSEEN 1] first unseen message in mbox]
 Mailbox: {imap-vif.cc.columbia.edu:143/imap/tls/user="fdc"}INBOX,
 2 messages, 0 recent
MS>
MS>hEADERS (MESSAGES) unsEEN
U      1)  5-Dec somebody@xyzcorp.com Fantastic Bargains! (1650 chars)
U      2)  5-Dec nobody@abccorp.com   Hot Stock Tips! (5995 chars)
MS>read (MESSAGES) NEW      (This doesn't read them)
MS>read (MESSAGES) unsEEN   (This does)
The GET command connects you to your IMAP inbox. Only give this command once per session. After authenticating, use the CHECK command to check for new mail. If you don't access the IMAP server every XXX minutes or less, it logs you out. You can give another GET command to re-login (in this case you can just type "get" without the arguments).

SORRY, THE FOLLOWING LINKS ARE DEFUNCT (Mark Crispin died in 2012):
[ MS Man Page ]   [ IMAP Information Center ]   [ C-Client Source Code ]   [ MS Source Code ]

Glossary

ANSI Organization American National Standards Institute
ANSI C Programming language Standardized version of the C language
ASCII Character set Basic character set of the Internet (ABCs, digits, punctuation)
Base64 Transfer encoding A binary-to-text encoding sometimes applied even to text
C Programming language The language in which MM is written
COMND DEC-20 instruction System service (JSYS) for command parsing
CRT Technology Cathode Ray Tube, a 1980s synonym for Terminal
DEC Company Digital Equipment Corporation, maker of the DEC-20
DEC-20 Computer DECSYSTEM-20, DEC computer where MM was born
Ethernet Network type Since 1980, the predominant type of Local Area Network (LAN)
GUI User interface style Graphical User Interface, as opposed to text and terminal
HTML Markup language HyperText Markup Language, in which Web pages are written
IMAP Internet protocol Internet Message Access Protocol - server-based mailbox
ISO Organization International Organization for Standardization
ISO 8859 Character set family ISO family of standard single-byte character sets [see charts]
JSYS DEC-20 instruction Jump to System (call a subroutine in the operating system)
Kermit Program Terminal emulation and file transfer
.mailcap File Metamail configuration file
LAN Network type Local Area Network, such as Ethernet
Mailspool Email delivery method Traditional disk-based Unix email delivery method
Make Program Unix program for building software
Makefile File Control file for building a particular program with Make
Metamail Program Extensible decoder/intepreter for MIME email
MIME Internet protocol Multipurpose Internet Mail Extensions (attachments)
MM Program Mail Manager, 1970s-80s mail client, subject of this page
NFS LAN protocol Network File System, allowing computers to share disks
POP3 Internet protocol Post Office Protocol version 3 - server-based email delivery
Quoted Printable Transfer encoding A subset of ASCII for encoding non-ASCII (and ASCII) text
Shell Program Command interpreter for text-based Unix access
SSL/TLS Security method Transport Layer Security / Secure Sockets Layer
Terminal Device Keyboard and display for text-based computer access
TOPS-20 Operating system OS of the DEC-20
UNICODE Character set Universal character set (multibyte)
UNIX Operating system family    Original basis for Linux, Mac OS X, Android, many more
UTF-8 Character set Standard representation of Unicode for the Internet and in Unix
Zmodem    Program File transfer

MM User References

MM and CCMD Developer References


The Kermit Project   /   kermit@kermitproject.org [Top]