http message store with replication

  • IMAP server will store email via http put to a webserver location. The active webserver locations would be stored in a configuration file. There could be any number of servers active at once. The storage would grab all of them and store in two locations. This would include a server host name and path. The path could be changed to store messages on a different mount location on the machine if drives are added.
  • Possibly encrypt stored messages
  • The email message url will be stored with the message data in the database.
  • A separate process will replicate stored email files to at least two server locations for failover.
  • When an email is viewed the IMAP server will lookup all the urls for the message file. If it fails to get the message from one location it will get it from the second location. It should randomly choose which location to use to help with load balancing.
  • Speed things up at some point with a local cache.
  • LDAP and MySQL would need to be replicated to at least one machine.

whats good about this idea

  • Infinite storage scalability via web servers. If you are low on storage just add a webserver or add a drive to a webserver and mount it. Then add the server to the IMAP storage servers configuration
  • Failover. Each message is stored in at least two locations. If the IMAP server can't get the message from one location it would get it from the other.
  • It would make a lot of sense to have frontend servers, the imap server, and web storage servers with two providers on separate hosting providers. This way if one goes down the other will likely be up.
  • dbmail system of db storage is great. However I my storage is restricted by the drives attached to my database. This is system could scale horizontally at frontend, like dbmail does not, but also horizontally at the storage level. This would allow for the use of inexpensive webserver machines as large storage farms for message files.
msg_file_http_failover.txt · Last modified: 2012/02/27 21:50 by bas
DBMail is developed by Paul J Stevens together with developers world-wide