advertisement

Kamaelia Grey

50 %
50 %
advertisement
Information about Kamaelia Grey

Published on November 6, 2007

Author: kamaelian

Source: slideshare.net

Description

This presentation was given at Python North West. It explains a complete Kamaelia application for greylisting which was written specifically to eliminate my personal spam problem. It walks through the code as well (though that's best looked at with the code side by side!)
advertisement

Relevant disclosure? - Kamaelia came out of my day work @ BBC Research - Kamaelia grey came out of my personal frustration, and has nothing to do with the BBC :-) Michael Sparks , [email_address] Kamaelia Grey

Drastic Spam reduction through Greylisting Greylisting made easy through natural concurrency Natural concurrency made easy through Kamaelia ... since that's what Kamaelia is for Kamaelia Grey

Problem? Spam was getting ridiculous One weekend the number of emails to home address topped 1000 96% was spam 4% wasn't, but was mainly mailing lists Around 1% interesting Next to impossible to find NB: I run my own mail server :-)

Spam was getting ridiculous

One weekend the number of emails to home address topped 1000

96% was spam

4% wasn't, but was mainly mailing lists

Around 1% interesting

Next to impossible to find

NB: I run my own mail server :-)

Kamaelia Day project at the BBC since 2003 Toolkit for making concurrency easy to work with in the context of Networking (servers/clients) Multimedia TV Pretty much anything Designed to make writing servers fast, scalable & reliable

Day project at the BBC since 2003

Toolkit for making concurrency easy to work with in the context of

Networking (servers/clients)

Multimedia

TV

Pretty much anything

Designed to make writing servers fast, scalable & reliable

Greylisting 1 Client connects, wants to send us mail Have we let their email through before? YES! I CAN HAZ ACCEPT ALL UR EMAIL

Client connects, wants to send us mail

Have we let their email through before?

YES!

Greylisting Client connects, wants to send us mail Have we seen them before? NO! KTHXBYE

Client connects, wants to send us mail

Have we seen them before?

NO!

Greylisting Client comes back after KTHXBYE Is it a resend of the same message after a reasonable delay?

Client comes back after KTHXBYE

Is it a resend of the same message after a reasonable delay?

Greylisting Client comes back after KTHXBYE Is it a resend of the same message after a reasonable delay? YES! I CAN HAZ ACCEPT ALL UR EMAIL

Client comes back after KTHXBYE

Is it a resend of the same message after a reasonable delay?

YES!

Greylisting Client comes back after KTHXBYE Is it a resend of the same message after a reasonable delay? NO! KTHXBYE

Client comes back after KTHXBYE

Is it a resend of the same message after a reasonable delay?

NO!

Where?

Here!

Requirements Python 2.3 onwards Must run your own mail server already Requires Kamaelia, but the tar ball includes that. I developed it under Linux I've got it deployed on Mac OS X

Python 2.3 onwards

Must run your own mail server already

Requires Kamaelia, but the tar ball includes that.

I developed it under Linux

I've got it deployed on Mac OS X

Mail serving UNTRUSTED INTERNET

Mail serving UNTRUSTED INTERNET MAIL SERVER 25

Mail serving UNTRUSTED INTERNET MAIL SERVER 25

Mail serving UNTRUSTED INTERNET MAIL SERVER 25

Greylisting UNTRUSTED INTERNET MAIL SERVER 8025

Greylisting UNTRUSTED INTERNET MAIL SERVER 8025 Kam Grey 25

Greylisting UNTRUSTED INTERNET MAIL SERVER 8025 Kam Grey 25

Installing Grab the code & install python setup.py install Configure edit /etc/greylist.conf then run /etc/init.d/kamaeliagrey start I actually just use App/runGreylister.sh

Grab the code & install

python setup.py install

Configure

edit /etc/greylist.conf

then run

/etc/init.d/kamaeliagrey start

I actually just use

App/runGreylister.sh

Basic config port = 25 servername = mail.cerenity.org serverid = Kamaelia-SMTP 1.0 smtp_ip = 192.168.2.9 smtp_port = 8025 inactivity_timeout = 60 greylist_log = /var/log/greylist.log greylist_debuglog = /var/log/greylist-debug.log attempters_db = /var/spool/attempters.dbm greylisted_db = /var/spool/greylisted.dbm

Local config allowed_senders: 127.0.0.1 allowed_sender_nets: 192.168.2 allowed_domains: private.thwackety.com thwackety.com thwackety.net yeoldeclue.com michaelsparks.info lansdowneresidents.org polinasparks.com pixienest.com kamaelia.org owiki.org cerenity.org

Exceptions # Standard triples which always allows delivery to domains we consider # accepting for. # # Format: # IP address, mail-from id, recipient id # whitelisted_triples: 213.38.186.202 <post@mx1.redcats.co.uk> <polina@thwackety.com>

Exceptions # Some non-standard triples for which we always allow delivery # to domains we consider accepting for. # Format: # claimed sender name, IP prefix, recipient # whitelisted_nonstandard_triples: listmail.artsfb.org.uk 62.73.155.19 <polina@thwackety.com> mx-out.facebook.com 204.15.20 <ms@cerenity.org> mx-out.facebook.com 204.15.20 <polina@cerenity.org> fallbackmx-out.facebook.com 204.15.20 <ms@cerenity.org> fallbackmx-out.facebook.com 204.15.20 <polina@cerenity.org>

Logging? Standard log: /var/log/greylist.log 20071106172215.940 | dsl88.241-20782.ttnet.net.tr | 88.241.81.46 | <linhotepmet@hotep.de> | <ms@cerenity.org> | DEFERRED | 20071106172231.269 | dsl88.241-20782.ttnet.net.tr | 88.241.81.46 | <linkinosoftmet@kinosoft.de> | <ms@cerenity.org> | DEFERRED | 20071106172238.610 | 34-148.privatnet.cz | 88.146.148.34 | <ggybaseh@walla.com> | <messages@cerenity.org> | DEFERRED | 20071106172244.333 | dsl88.241-20782.ttnet.net.tr | 88.241.81.46 | <linlinasolutionsmet@linasolutions.de> | <ms@cerenity.org> | DEFERRED | 20071106172247.885 | mta410.k.cheetahmail.com | 208.49.63.136 | <bo-b00hg7jaukqbyva8zuwb9b9y5ph44j@b.emails.dixons.co.uk> | <zathras-pcworld@thwackety.com> | ACCEPTED |

Logging? Debug log: /var/log/greylist.log 250 OK 250 ACCEPTED 451 4.7.1 Please try again later 220 mail.cerenity.org ESMTP Kamaelia-SMTP 1.0 Tue Nov 6 17:26:01 2007 500 Command Not Recognised 250 mail.cerenity.org Hello lse 89.252.24.7 250 OK Note – this is what's being sent over SMTP

But also... Debug log: /var/log/greylist-debug.log *debug* THREADS['Kamaelia.Chassis.Pipeline.Pipeline_7', 'Kamaelia.Internet.Selector.Selector_11', '__main__.GreylistServer_8', '__main__.PeriodicWakeup_5', '__main__.TCPS_10', '__main__.WakeableIntrospector_6'] Note – this says the current internal components running

Internals Debug log: /var/log/greylist-debug.log ['Kamaelia.Chassis.Pipeline.Pipeline_7', 'Kamaelia.Internet.Selector.Selector_11', '__main__.GreylistServer_8', '__main__.PeriodicWakeup_5', '__main__.TCPS_10', '__main__.WakeableIntrospector_6'] Note – this says the current internal components running

Internals Debug log: /var/log/greylist-debug.log Kamaelia.Internet.Selector.Selector_11 – wakes system on network events __main__.TCPS_10 – sits inside and handles listening for '__main__.GreylistServer_8 – A configured network server Kamaelia.Chassis.Pipeline.Pipeline_7 – For debugging '__main__.PeriodicWakeup_5 – For debugging '__main__.WakeableIntrospector_6 – For debugging Note – this says the current internal components running

No connection! TCPServer Greylist Server Selector

New connection! TCPServer Greylist Server Selector Connected Socket Adapter GreyListing Policy

More connections! TCPServer Greylist Server Selector Connected Socket Adapter GreyListing Policy Connected Socket Adapter GreyListing Policy Connected Socket Adapter GreyListing Policy Connected Socket Adapter GreyListing Policy

Accepted! GreyListing Policy TCPClient Connected Socket Adapter MAIL SERVER 8025

Rejected! Connected Socket Adapter GreyListing Policy

Greylisting GreyListing Policy Concrete Mail Handler Mail Handler

Internals The following pages are intended as a walk through of the key highlights of the code-base. It skips the body of the code since that's best looked at by scrolling through code and chatting rather than dumping into slides. In a presentation its easy to say this, and then after going through slides giving road pointers go through the code. Presentation files are a little more limited. The code referred to is here: http://tinyurl.com/2sbjxl

Internals class MailHandler(Axon.Component.component): logfile = &quot;greylist.log&quot; debuglogfile = &quot;greylist-debug.log&quot;

Internals class MailHandler(Axon.Component.component): def __init__(self,**argd): def logging_recv_connection(self): def getline(self): def handleCommand(self,command): def noteToLog(self, line): def noteToDebugLog(self, line): def netPrint(self, *args): def lastline(self): def main(self):

Internals class ConcreteMailHandler(MailHandler): Inboxes = { &quot;inbox&quot; : &quot;Data from the client connecting to the server comes in here&quot;, &quot;control&quot; : &quot;Shutdown & control messages regarding client side socket handling&quot;, &quot;tcp_inbox&quot; : &quot;This is where we get respones from the real SMTP server&quot;, &quot;tcp_control&quot; : &quot;This is where we get shutdown information from the real SMTP server&quot;, } Outboxes = { &quot;outbox&quot; : &quot;Data sent here goes back the the client connecting to the server&quot;, &quot;signal&quot; : &quot;Shutdown & control messages regarding client side socket handling&quot;, &quot;tcp_outbox&quot; : &quot;Data sent here is sent to the real SMTP server&quot;, &quot;tcp_signal&quot; : &quot;We send messages here to shutdown the connection to the real SMTP connection&quot;, } peer = &quot;*** UNDEFINED ***&quot; peerport = &quot;*** UNDEFINED ***&quot; local = &quot;*** UNDEFINED ***&quot; localport = &quot;*** UNDEFINED ***&quot; servername = &quot;Testing.server.local&quot; serverid = &quot;MPS SMTP 1.0&quot; smtp_ip = &quot;192.168.2.9&quot; smtp_port = 25

Internals class ConcreteMailHandler(MailHandler): def connectToRealSMTPServer(self): def __init__(self, **argv): def error(self, message): def RelayError(self): def handleConnect(self): def handleEhlo(self,command): def handleHelo(self,command): def handleHelp(self,command): def handleVrfy(self,command): def handleRset(self,command): def handleNoop(self,command): def handleMail(self,command): def handleRcpt(self,command): def handleData(self, command): def handleQuit(self,command): def shouldWeAcceptMail(self): return False # Default policy - don't accept any email def deferMail(self): def acceptMail(self): def getline_fromsmtpserver(self): def handleDisconnect(self):

Internals class GreyListingPolicy(ConcreteMailHandler): allowed_senders = [] allowed_sender_nets = [] allowed_domains = [ ]

Internals class GreyListingPolicy(ConcreteMailHandler): (continued) def shouldWeAcceptMail(self): def sentFromAllowedIPAddress(self): def sentFromAllowedNetwork(self): def sentToADomainWeForwardFor(self): def isGreylisted(self, recipient): def whiteListed(self, recipient): def logResult(self):

Internals class GreylistServer(MoreComplexServer): logfile = config[&quot;greylist_log&quot;] debuglogfile = config[&quot;greylist_debuglog&quot;] socketOptions=(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) port = config[&quot;port&quot;] class TCPS(TCPServer): CSA = NoActivityTimeout(ConnectedSocketAdapter, timeout=config[&quot;inactivity_timeout&quot;], debug=False) ....

Internals class GreylistServer(MoreComplexServer): (continued) # ... class protocol(GreyListingPolicy): servername = config[&quot;servername&quot;] serverid = config[&quot;serverid&quot;] smtp_ip = config[&quot;smtp_ip&quot;] smtp_port = config[&quot;smtp_port&quot;] allowed_senders = config[&quot;allowed_senders&quot;] allowed_sender_nets = config[&quot;allowed_sender_nets&quot;] allowed_domains = config[&quot;allowed_domains&quot;] whitelisted_triples = config[&quot;whitelisted_triples&quot;] whitelisted_nonstandard_triples = config[&quot;whitelisted_nonstandard_triples&quot;]

And that... Saves me having to wade through now literally tens of thousands of spams over the past two months: # grep DEFERRED /var/log/greylist.log |wc -l 73798

Time to write? 2 days start to finish including protocol Bug fixes ~1-2 days tops

Questions? Thank you :-) I hope it's useful to you :)

Add a comment

Related pages

Kamaelia: Grey - KamaeliaGrey

What is Kamaelia: Grey ? It's a greylisting SMTP proxy. It doesn't replace your existing mail server, but does sit between your mail server and the outside ...
Read more

Kamaelia

Kamaelia Grey. Spam reduction through greylisting. Kamaelia Grey is a SMTP proxy for your inbound email, rejecting email likely to be spam. Mail likely to ...
Read more

greylisting.py - kamaelia - Kamaelia is a toolset that ...

Kamaelia is a toolset that makes working with concurrency fun, simple and reusable.
Read more

Mango CAMELIA - Übergangsjacke - medium grey - Zalando.de

Mango CAMELIA - Übergangsjacke - medium grey für 21,49 € (17.03.16) versandkostenfrei bei Zalando bestellen.
Read more

Camelia Gray | Facebook

Camelia Gray is on Facebook. Join Facebook to connect with Camelia Gray and others you may know. Facebook gives people the power to share and makes the...
Read more

Taratata Ring Camélia Grey

Zauberhafter Ring aus der Camélia Kollektion, mit einer silberfarbenen Blüte im Onlineshop.
Read more

Camelia Grey | LinkedIn

View Camelia Grey’s professional profile on LinkedIn. LinkedIn is the world's largest business network, helping professionals like Camelia Grey discover ...
Read more

/trunk/Code/Python/Apps/Kamaelia-Grey/ - kamaelia ...

Kamaelia is a toolset that makes working with concurrency fun, simple and reusable.
Read more

Camellia sinensis - Wikipedia, the free encyclopedia

Camellia sinensis; Camellia sinensis ... The seeds of Camellia sinensis and Camellia oleifera can be pressed to yield tea oil, ... Earl Grey ; Breakfast ...
Read more