Re: Another Queue Question

seaPea <printerster@xxxxxxxxx> writes:

Andrzej Adam Filip wrote:
aldouk75-google@xxxxxxxxxxx writes:

Hi there, looking for some pointers,

Have a client that has had some serious downtime over the last few
days with their Linux Sendmail server.
The hardware has been fixed but we now have a situation that there are
280,000 files in /var/spool/mqueue.

The server is used for inbound email only and its directed to either
the clients Notes or MS Exchange infrastructure, all new email is
delivered quickly, just seems to be the old(er) messages that are
delayed, the number of files is decreasing but at a very slow rate.

The server is a Compaq DL380 with 6gb of memory, running linux with a
kernal of Linux 2.4.19-64GB-SMP.
The version of Sendmail is 3.0.6, this was a corporate image so I'm
not too sure on what open source build its based upon.

The current config used is very basic (so I am told):

divert(-1)dnl
# This file was created with Sendmail Switch.
divert(0)dnl
VERSIONID(`@(#)xxxx.m4 1.0 (sendmail) 1/30/2008')
MODIFY_MAILER_FLAGS(`LOCAL', `-S')dnl
define(`QUEUE_DIR', `/var/spool/mqueue/')dnl
define(`confBIND_OPTS', `+WorkAroundBrokenAAAA')dnl
define(`confLOG_LEVEL', `10')dnl
define(`confQUEUE_LA', `50')dnl
define(`confQUEUE_SORT_ORDER', `Modification')dnl
define(`confREFUSE_LA', `65')dnl
define(`confRUN_AS_USER', `smadmin')dnl
FEATURE(`access_db', `DATABASE_MAP_TYPE -T<TMPF>
MAIL_SETTINGS_DIR`'access')dnl
FEATURE(`blacklist_recipients')dnl
FEATURE(`generics_entire_domain')dnl
FEATURE(`genericstable', `DATABASE_MAP_TYPE
MAIL_SETTINGS_DIR`'genericstable')dnl
FEATURE(`mailertable', `DATABASE_MAP_TYPE
MAIL_SETTINGS_DIR`'mailertable')dnl
FEATURE(`no_default_msa')dnl
FEATURE(`nouucp', `reject')dnl
FEATURE(`relay_entire_domain')dnl
FEATURE(`use_cw_file')dnl
FEATURE(`virtusertable', `DATABASE_MAP_TYPE
MAIL_SETTINGS_DIR`'virtusertable')dnl
MAILER(`local')dnl
MAILER(`smtp')dnl

The Linux server in itself doesnt seem to be under too much load
either:

18:11:12 CPU %user %nice %system %idle intr/s
18:11:12 all 5.31 0.01 5.07 89.61 1233.16

Any pointers on what we can do to increase the throughput of this box
would be greatly appreciated..

[ for "internet to internal" gateway]

First: Flush the queue faster using command below:
/usr/sbin/sendmail -q -OQueueSortOrder=random
More elaborate options required if you use more than four internal
destinations for SMTP connections.
Start *first* command with additional "-v" optional.
If queue is processed without porblems than start a few more (3-5)
(without -v) as background jobs.

Second:
1) setup separate queue group per every destination of
"to internal" SMTP connections with multiple queue directories in
every group (e.g. 4 directories)
2) configure sendmail to accept and process ETRN from intenal servers
It should allow the servers to request flushing queue upon then own
request without waiting for next queue run

Hi Andrew,
I came across a similiar problem back when sendmail8.11 was
involved. The workaround that we were advised to do was
* shutdown sendmail
* move out the files from mqueue
* restart sendmail but turn off acceptance of new email from the
outside
* move back the files to mqueue 4000 at a time
* turn back on acceptance of new email from the outside when last
batch back in
This was done on a 700mhz pc running RH9 with 768M RAM. Took less than
an hour.

Perl motto: There is more than one way to do things right :-)
My motto: Usually *WHY* is more important than *WHAT* :-)

With such number of messages in queue there are (AFAIR) two main source
of possible problems:
a) memory overload by data required to process the queue in "sorted" way
"QueueSortOrder=random" I suggested avoids this trap
b) file system performance problems with *MANY* files in a (queue) directory
[ file systems: e.g. ext2, ufs, ...]
*many* file system types do linear search for files in directory

I have addressed "a", recipe you have mentioned may address "b".

--
[pl>en: Andrew] Andrzej Adam Filip : anfi@xxxxxxxxxxxx : anfi@xxxxxxxx
Open-Sendmail: http://open-sendmail.sourceforge.net/
"Would I turn on the gas if my pal Mugsy were in there?"
"You might, rabbit, you might!"
-- Looney Tunes, Bugs and Thugs (1954, Friz Freleng)
----
http://groups.google.com/groups?selm=87myqnqb3i@xxxxxxxxxxxxxxxxxxxxxxxx
.

Relevant Pages

  • Re: Proliant DL360 with sendmail on RedHat 8.0
    ... > We are useing an email generator which can generate many SMTP connection to ... > sendmail server. ... You may consider creation of qf, df and xf subdirectories in queue ... it is enabled by default for Linux. ...
    (comp.os.linux.hardware)
  • Re: Sendmail issue
    ... mailq returns the queue length. ... You should have about 2x the length of the mail queue. ... Since sendmail isn't aware ... mail gets held on the AIX server and then gets released all at once. ...
    (AIX-L)
  • Infinite loop of deferred/failure messages
    ... We have a linux server running sendmail, ... control messagefest if the exchange server is down. ...
    (comp.mail.sendmail)
  • Re: Infinite loop of deferred/failure messages
    ... We have a linux server running sendmail, ... control messagefest if the exchange server is down. ...
    (comp.mail.sendmail)
  • Re: hostst that use 127.0.0.1 as their MX - how to prevent my mailbox from filling up
    ... The error on the bounce was: 451 4.4.1 timeout writing message to mail.advancedbioenergy.com.: Broken pipe... ... Deferred Message could not be delivered for 5 days Message will be deleted from queue ... It looks like your server had timeouts while trying to communicate with the receiving server. ... However Sendmail should try to fit as much data in an ethernet frame / TCP window as it can. ...
    (comp.mail.sendmail)