Re: ...getting onto the soapbox: SMTP with DNS

Giganews Newsgroups
Subject: Re: ...getting onto the soapbox: SMTP with DNS
Posted by:  Martin James (mjames_falc…
Date: Tue, 4 May 2004

"Pierre Roux" <pier…> wrote in message
> This message doesn't contain any real problems, for discussion and
> only.
>  - DNS Lookup and SMTP Sending Code at bottom
> Part of what I'm playing with at the moment is DNS delivery of messages
> using TIdDNSResolver and TIdSMTP Objects in a list of threads (inside a
> service).

I have an app that does something like this.  It distributes mails retrieved
from a POP server account using a similar mechanism.  I dodged the need for
a 'delivery list' by creating a seperate 'outBox' file for each recipient.
I can do this because the delivery list is short & fixed at the time when
the message is retrieved from the POP account - my app looks it up in a
database where the recipients are stored against sender address.  For my
app, management of a seperate 'delivery list' file was just too messy.  I
could have stored the mail list progress, and indeed, the mails, in the
database, but this would have been more difficult to debug.

> A Timer triggers each SMTP delivery sequence.
> When the timer is triggered:
>  - Step through list of threads:
>  If Thread Busy, leave it alone
>  If Thread Not Busy, hand it a message, a delivery list and a list of DNS
> Servers
>          Let the thread resume (I start and kill a predetermined number
> threads at start-up and closing only)

I, too have SMTP threads to send on the messages, but they all wait on a
single semaphore queue. When the timer fires for a delivery sequence, the
outBox folder is searched for each outstanding mail item. Each is loaded
into request/response objects & posted to the queue.  The relevant mail file
is moved from  the outBox folder to a 'pending' folder.

> - The Thread Loop:
>  - Do a DNS Lookup until an MX record is found for the domain, or until
> run out of DNS servers (Code at bottom).
>  - Messages gets delivered to the mail server as shown by the MX record
> the DNS lookup, and becomes that servers problem (Code at bottom).

One of the SMTP threads gets a request/response object from the queue &
tries to mail it, using a DNS lookup as in your app.  The thread sets a
'success/fail' boolean in the R/R object, loads in any exception.message as
appropriate, & postMessages it back to the main thread.

>  - On success, remove the address from the delivery list. (If the address
> is there twice, deliver twice)
>  - If a message fails, increment the Failure TTL that lives inside the
> delivery list.  Three (or X) failures, drop a message into SMTP queue
> directory and tell the sender what's up. (Log SMTP Failures)

When a R/R object gets back to the main thread, it's checked for success & a
listView updated so theuser can see what's going on.  If the mail has gone
off OK, the mail file is moved from 'pending' to 'sent mail'.  If the mail
failed, the mail file is moved from 'pending' back to to 'outbox', os it can
be retried later.

>  - On completion, suspend the thread

My SMTP threads loop around to get the next R/R object from the semaphore
queue, (or wait on it if there are no entries).

> Unsolved funnies:
>  - On some mail servers the message would consistently keep on downloading
> until actually deleted from the server.  Seems to be related to the UID of
> the message, but not tested enough.

Not seen this one.

>  - MX Lookup returns the MX hostname as well as the MX IP Address.  So for
> two (MX) servers, I'd have four answers from the DNS Resolver, but step
> through them all at the moment.

I jut pick the one with the highest priority.

>  - SMTP Sending Threads won't die correctly yet, I'm probably just
> forgetting something, non-issue at the moment

I set the threads to 'terminated' & send enough 'suicide' notes to the queue
to force every waiting thread to wake up & realise it has to die.  There is
an issue with the threads that are stuck on SMTP send operations.  Since I
am not keeping any reference to the threads, I cannot directly call
'diconnect' on the connections of the threads to force those blocked to
except.  Not a serious problem at the moment - the app seems to shut down OK
& so I'm not going to fix something that 'aint broke.

> Stuff:
>  - Is this how 'REAL' SMTP Servers also do it?

Not sure...

Seems very awkward.

Well, it's a complex operation.

>  - I have NEVER had so many exception handling blocks in ANY piece of code
> before...

Well, I have only one exception handler in the threads.  If the DNS lookup
or SMTP send fails, (or there is an AV!),  it all comes back to this one
handler.  The handler stuffs the exception .message into the current R/R
object, sets 'fail' status & posts it off.



In response to

...getting onto the soapbox: SMTP with DNS posted by Pierre Roux on Tue, 4 May 2004