Configuring spfmilter or bms with pymilter

[virendrasuryavanshi]

I have setup the pymilter with sendmail and it is able to read the subject on the running socket now I want to filter the mails and modify the subject to mention whether it's a spam or not along with mentioning the score using the spfmilter or bms but I am not able to figure out how can I configure the bms or spfmilter with the already running pymilter. Can you please suggest how should I do it?
Thanks.

[virendrasuryavanshi]

@sdgathman

[sdgathman]

pymilter is a library. By "running", I'm guessing you mean running the sample? Is the sample actually useful? If not, don't run it. :-) The milter API does support multiple milters. They each need their own socket or internet port. Sendmail calls them in the order they are declared.

You have a design choice of creating a single big milter (like bms.py), or several smaller ones like spfmilter.py and dkimmilter.py. If you do create a single big milter, I suggest a plug-in approach. That was the mistake I made with bms.py - everything is all intermingled.

[virendrasuryavanshi]

Yeah, I was running the sample.py just to test the pymilter, which was properly working. But the problem where I really encountered was when I cloned the milter repo and tried to run bms milter or spfmilter or dkim-milter for the local mails received by sendmail

I am getting the following errors which I am not able to figure out. Can you please have a look at it, it would be a great help. Thanks.

[sdgathman]

Well, the spfmilter problem is easy: you have to configure the socket in sendmail to match each milter you run. That is true no matter what language the milter is written in.

The bms.py milter is missing permission to write to /var/log/milter directory. On the redhat and fedora rpm, I run bms.py as the mail user, and /var/log/milter is writable by mail. I only have rpm packaging to set all that stuff automatically. What system are you packaging/installing for? Look at milter.spec to get an idea of the setup required.

The dkim-milter.py requires python-pydkim (or dkimpy). It looks you need that. Packaging has dependency info. I have only used RPM for installable packages. Are you building from source? Are you creating an installable package for your system? What system?

[sdgathman]

Here are the installed files for the milter RPM:

$ rpm -qlv milter
-rwxr-xr-x    1 root    root                      123 Nov  8  2017 /etc/cron.daily/milter
-rw-r--r--    1 root    root                      192 Nov  8  2017 /etc/logrotate.d/milter
-rw-r--r--    1 mail    mail                    12174 Nov  8  2017 /etc/mail/pymilter.cfg
-rw-rw-r--    1 root    root                      283 Nov  7  2017 /usr/lib/systemd/system/bmsmilter.service
-rwxr-xr-x    1 mail    mail                     1206 Nov  8  2017 /usr/libexec/milter/ban2zone.py
-rwxr-xr-x    1 mail    mail                   101375 Nov  8  2017 /usr/libexec/milter/bms.py
-rw-r--r--    1 mail    mail                     1510 Nov  8  2017 /usr/share/sendmail-cf/hack/rhsbl.m4
drwxr-xr-x    2 mail    mail                        0 Nov  8  2017 /var/lib/milter
-rw-r--r--    1 mail    mail                     1116 Nov  8  2017 /var/lib/milter/fail.txt
-rw-r--r--    1 mail    mail                      867 Nov  8  2017 /var/lib/milter/heloerror.txt
-rw-r--r--    1 mail    mail                     1351 Nov  8  2017 /var/lib/milter/neutral.txt
-rw-r--r--    1 mail    mail                      908 Nov  8  2017 /var/lib/milter/permerror.txt
-rw-r--r--    1 mail    mail                      834 Nov  8  2017 /var/lib/milter/quarantine.txt
-rw-r--r--    1 mail    mail                      771 Nov  8  2017 /var/lib/milter/softfail.txt
-rw-r--r--    1 mail    mail                     2546 Nov  8  2017 /var/lib/milter/strike3.txt
-rw-r--r--    1 mail    mail                      833 Nov  8  2017 /var/lib/milter/temperror.txt
drwxr-xr-x    2 mail    mail                        0 Nov  8  2017 /var/log/milter/save

[virendrasuryavanshi]

Yes, I am building it from source. Did you built it in RHEL? I am building it in Ubuntu 16.04.

[sdgathman]

Do you know how to make debian/ubuntu packages? I don't. I would be glad to include your debian packaging. I don't even know what the debian equivalent of a SPEC file is, however. All I know is the binary packages are called *.dpkg and are installed with apt-get.

[sdgathman]

In the Redhat/Fedora world, we use 'mock' to build packages in a container, which makes sure all the BuildRequires are correct. Many of the run-time requirements are detected automatically.

[sdgathman]

There are many optional dependencies for the bms milter. For instance, pydspam is required for the bayesian content filter feature. RPM now has metadata for suggested dependencies - something debian has had for a long time.

[sdgathman]

Yes, I am building it from source. Did you built it in RHEL? I am building it in Ubuntu 16.04.

Yes, I build for RedHat and Fedora. I run mail servers on CentOS, currently CentOS-7.

[virendrasuryavanshi]

Hey Stuart @sdgathman , After your comments I started building my own milter with the help of the following documentation (https://pythonhosted.org/pymilter/annotated.html) but there is one issue with which I am stucked that is milter.addheader, milter.chgheader functions are throwing the following errors which I am not able to figure out. Can you please have a look at it.

[sdgathman]

add/chg header and replacebody must be called from eom callback. Annoying, I know. The pythonhosted.org site is readonly now, and won't accept updates. So I moved docs to pymilter.org. I haven't advertised it yet, as it needs to be checked out. Please take a look.

For instance, https://www.pymilter.org/pymilter/classMilter_1_1Base.html#a23df18e239f441049e17d4e87b61be1c is addheader - which mentions that it must be called from eom. Since there are so many things that can only be called from eom, there should probably be a mention of that in the overview.

libmilter give no indication of the cause of the milter.error. However, if there was a simple and reliable way for the addheader method to determine whether it was being called from eom() callback, we could throw a more informative exception. Maybe set a flag in the Milter.Base object when calling eom(), and ensuring it gets cleared again with try: except:

[virendrasuryavanshi]

add/chg header and replacebody must be called from eom callback. Annoying, I know. The pythonhosted.org site is readonly now, and won't accept updates. So I moved docs to pymilter.org. I haven't advertised it yet, as it needs to be checked out. Please take a look.

For instance, https://www.pymilter.org/pymilter/classMilter_1_1Base.html#a23df18e239f441049e17d4e87b61be1c is addheader - which mentions that it must be called from eom. Since there are so many things that can only be called from eom, there should probably be a mention of that in the overview.

libmilter give no indication of the cause of the milter.error. However, if there was a simple and reliable way for the addheader method to determine whether it was being called from eom() callback, we could throw a more informative exception. Maybe set a flag in the Milter.Base object when calling eom(), and ensuring it gets cleared again with try: except:

Thanks, I couldn't have figured out this easily.

Do you have tested or know that can we integrate SpamAssassin and ClamAV with pymilter?