MailSneaker - Part 2: POPing and pickling !

Last Update: 23.01.2007. By kerim in python | tutorial

Ok .. last time we made a small imap fetcher.
I got some remarks that it would only look in the INBOX and that might not be what we want.
Just include a new parameter in the fetch method with the name of the inbox you want to scan and pass that on to the imaplib.
Today we will add a pop3 fetcher and a generic class for mailaccounts so that the user later can run the program against several accounts.

Day 2: pop3 fetcher
There is not much difference between pop3 and imap when it comes to ease of programming…
We start by including the apropriate module again…

import poplib

Then of course we need a server url, an account name and a pass just as we did with the imap fetcher

servername='pop.web.de'  
username= 'kmansour'  
password='lalala'

Again we need to connect:

server = poplib.POP3(servername)  
server.user(username)

You might get something like this:
‘+OK enter password’
as response (can be seen when using idle)
Enter:

server.pass_(password)

and you should be ready after recieving the message (depends on your provider):
‘+OK mailbox locked and ready’

Lets get the info we need.

stat = server.stat()

stat[0] contains the number of messages.

We do face a problem now however …
pop doesn’t tell you which mails are really NEW.
Normally the mail clients using pop would retrieve the mails and thus each new query would include only new mails since the ones from last time would be gone.
But what should we do in case you are one of those “webbrowser”-users that don’t use normal clients or have a client that doesnt delete the mails on the server after reading them ?
We need to track all old mails.
Since 1996 there is a nice feature calles UIDL which should have made it to all common providers by now. If yours doesn’t have it … change your provider.
What we need to do now is to ask for all UIDLs and compare that to a list we already should have for the “old” mails.

uidls=server.uidl()[1]

The best thing now is to have a dictionary or list of old mail uidls and see which of the just retrieved ones is really new.
I am not goind into detail about that now. You can use pickling to store the old mails in a file.
A question that arises is when and how we store the list of old uidls and when we should consider a new mail “read”.
With imap we have no problem because the server keeps track of that.
But with pop3 we shouldn’t for example include a new mail in the list of old ones as long as we are not sure that the user should have had a fair chance to read it.
If we check periodically for example we can’t simply assume that he read it between the two intervals.
If however he opened up a browser and went to the site we can savely assume that he either read it or doesn’t intend to do it anyway.
Alternatively we can also give him the option to say “mark all read”.
A bit sad that imap and pop3 differ in that aspect so much. And a reason why you should prefer imap over pop3.

Anyway … lets see the class without extra old mail / new mail logic (only rudimentary storage methods)

import poplib, pickle

class POP3Fetcher:  
    def __init__(self,sname, uname, passwd):  
        self.servername=sname  
        self.username=uname  
        self.password=passwd  
        self.newMails={}  
        self.oldMails={}  
        self.loadMailList()

    def fetch(self):  
        try:  
            server = poplib.POP3(servername)  
            server.user(username)  
            server.pass_(password)  
            uidls=server.uidl()[1]  
            for uidl in uidls:  
                if self.oldMails.get(uidl) == None:  
                    self.newMails[uidl]=uidl #store the mail in the list of already retrieved new mails   
            return self.newMails.keys()  
        except poplib.error_proto, detail:  
            # error handling (perhaps to many requests per time ?)  
            print "POP3 Protocol Error:", detail

    def loadMailList(self):  
        try:  
            file = open('readPopMails.data', 'r')  
            d=pickle.load(file)  
            if d!=None:  
                self.oldMails=d  
            file.close()  
        except :  
            print "File not existent or unpickle error"

    def saveMailList(self):  
        file = open('readPopMails.data', 'w')  
        pickle.dump(self.oldMails, file)  
        file.close()

    def markMailsRead(self):  
        for key in self.newMails:  
            self.oldMails[key]=key  
        self.saveMailList()


#--------------------testing---------------------  
servername='yoursite'  
username= 'yourID'  
password='yourPass'  
pop=POP3Fetcher(servername,username,password)  
print pop.fetch()  
pop.markMailsRead()

Bingo …

PS:In case you fear for your personal data in the stored file.... have a look at its contents :-)