[Design.Indexes.MainIndex.txt] ('dovecot.index') * [Design.Indexes.TransactionLog.txt] ('dovecot.index.log' and 'dovecot.index.log.2') * [Design.Indexes.Cache.txt] ('dovecot.index.cache') See for more generic information about what they contain and why. The index files can be accessed using [Design.Indexes.MailIndexApi.txt]. Locking ------- The index files are designed so that readers cannot block a writer, and write locks are always short enough not to cause other processes to wait too long. Dovecot v0.99's index files didn't do this, and it was common to get lock timeouts when using multiple connections to the same large mailbox. The main index file is the only file which has read locks. They can however block the writer only for two seconds (and even this could be changed to not block at all). The writes are locked only for the duration of the mailbox synchronization. Transaction logs don't require read locks. The writing is locked for the duration of the mailbox synchronization, and also for single transaction appends. Cache files doesn't require read locks. They're locked for writing only for the duration of allocating space inside the file. The actual writing inside the allocated space is done without any locks being held. In future these could be improved even further. For example there's no need to keep any index files locked while synchronizing, as long the mailbox backend takes care of the locking issues. Also writing to transaction log could work in a similar way to cache files: Lock, allocate space, unlock, write. Lockless integers ----------------- Dovecot uses several different techniques to allow reading files without locking them. One of them uses fields in a "lockless integer" format. Initially these fields have "unset" value. They can be set to a wanted value in range 0..2^28 (with 32bit fields) once, but they cannot be changed. It would be possible to set them back to "unset", but setting them the second time isn't safe anymore, so Dovecot never does this. The lockless integers work by allocating one bit from each byte of the value to "this value is set" flag. The reader then verifies that the flag is set for the value's all bytes. If all of them aren't set, the value is still "unset". Dovecot uses the highest bit for this flag. So for example: * 0x00000000: The value is unset * 0xFFFF7FFF: The value is unset, because one of the bytes didn't have the highest bit set * 0xFFFFFFFF: The value is 2^28-1 * 0x80808080: The value is 0 * 0x80808180: The value is 0x80 Dovecot contains 'mail_index_uint32_to_offset()' and 'mail_index_offset_to_uint32()' functions to translate values between integers and lockless integers. The "unset" value is returned as 0, so it's not possible to differentiate between "unset" and "set" 0 values. (This file was created from the wiki on 2011-01-13 04:52) wiki/Quota.Configuration.txt000066600000025043150501203730012153 0ustar00Quota Configuration =================== The configuration is split into multiple settings: First you have the quota root backend configuration. Quota root is a concept from IMAP Quota specifications [http://www.rfc-editor.org/rfc/rfc2087.txt]. Normally you'll have only one quota root, but in theory there could be e.g. "user quota" and "domain quota" roots. It's unspecified how the quota roots interact with each others (if at all). In some systems for example INBOX could have a completely different quota root from the rest of the mailboxes (e.g. INBOX in '/var/mail/' partition and others in '/home/' partition). Quota root configuration includes the backend name, quota root name and its parameters, if there are any: ---%<------------------------------------------------------------------------- quota = [:[:]] ---%<------------------------------------------------------------------------- The quota root name is just an arbitrary string that is sent to IMAP clients, which in turn may show it to the user. The name has no meaning. By default an empty string is used, but you may want to change that since some clients (Apple Mail) break and don't show quota at all then. You can define multiple quota roots by appending an increasing number, for example: ---%<------------------------------------------------------------------------- plugin { quota = maildir:User quota quota2 = fs:Disk quota #quota3 = ... } ---%<------------------------------------------------------------------------- Quota rules ----------- Quota rules configure the actual quota limits. The syntax is: ---%<------------------------------------------------------------------------- quota_rule = : #quota_rule2 = ... #quota_rule3 = ..etc.. ---%<------------------------------------------------------------------------- "*" as the mailbox name configures the default limit, which is applied on top of a mailbox-specific limit if found. So for example: ---%<------------------------------------------------------------------------- quota_rule = *:storage=1G quota_rule2 = Trash:storage=100M quota_rule3 = SPAM:ignore ---%<------------------------------------------------------------------------- This means that the user has 1GB quota, but when saving messages to Trash mailbox it's possible to use up to 1,1GB of quota. The quota isn't specifically assigned to Trash, so if you had 1GB of mails in Trash you could still save 100MB of mails to Trash, but nothing to other mailboxes. The idea of this is mostly to allow the clients' move-to-Trash feature work while user is deleting messages to get under quota. Additionally, any messages in the SPAM folder are ignored per the 'ignore' directive and would not count against the quota. "?" as the mailbox name works almost like "*". The difference is that "?" is used only if quota backend doesn't override the limit. For example with Maildir++ [http://www.inter7.com/courierimap/README.maildirquota.html] quota if 'maildirsize' file exists the limits are taken from it, but if it doesn't exist the "?" limits are used. "*" can't be used as a generic wildcard in mailbox names, so for example "box*" wouldn't match "boxes". As shown in the above example, the first quota rule is named 'quota_rule' while the following rules have an increasing digit in them. You can have as many quota rules as you want. Limit configuration ------------------- The following limit names are supported: * *storage*: Quota limit in kilobytes, 0 means unlimited. * *bytes*: Quota limit in bytes, 0 means unlimited. * *messages*: Quota limit in number of messages, 0 means unlimited. This probably isn't very useful. * *backend*: Quota backend-specific limit configuration. * *ignore*: Don't include the specified mailbox in quota at all. All of these support also b/k/M/G/T/% suffixes. So storage=100M and bytes=100M both mean the exact same thing. messages=1k also means 1024 messages (not 1000). Percents are relative to the default rule. For example: ---%<------------------------------------------------------------------------- plugin { quota = maildir:User quota quota_rule = *:storage=1GB # 10% of 1GB = 100MB quota_rule2 = Trash:storage=10%% # 20% of 1GB = 200MB quota_rule3 = Spam:storage=20%% } ---%<------------------------------------------------------------------------- Note that % is written twice to escape it, because <%variables> [Variables.txt] are expanded in plugin section. [UserDatabase.txt] configuration may or may not require this escaping. Backend-specific configuration currently is used only with Maildir++ quota backend. It means you can have the quota in Maildir++ format (e.g. "10000000S"). Per-user quota -------------- You can override quota rules in your [UserDatabase.txt]'s [UserDatabase.ExtraFields.txt]. Keep all the global settings in plugin section and override only those settings you need to in your userdb. For example: ---%<------------------------------------------------------------------------- plugin { quota = maildir:User quota quota_rule = *:storage=1G quota_rule2 = Trash:storage=100M } ---%<------------------------------------------------------------------------- Next override the default 1GB quota for users: ---%<------------------------------------------------------------------------- # LDAP: user_attrs = homeDirectory=home,quotaBytes=quota_rule=*:bytes=%$ # MySQL: user_query = select uid, gid, home, \ concat('*:bytes=', quota_bytes) as quota_rule \ from users where userid = '%u' # MySQL with userdb prefetch: Remember to prefix quota_rule with userdb_ # (just like all other userdb extra fields): password_query = select userid as user, password, \ uid as userdb_uid, gid as userdb_gid, \ concat('*:bytes=', quota_bytes) as userdb_quota_rule \ from users where userid = '%u' # Example passwd-file entries: user:{plain}pass:1000:1000::/home/user::userdb_quota_rule=*:bytes=100M user2:{plain}pass2:1001:1001::/home/user2::userdb_quota_rule=*:bytes=200M ---%<------------------------------------------------------------------------- Quota for public namespaces --------------------------- You can create a separate namespace-specific quota that's shared between all users. This is done simply by adding ':ns=' parameter to quota setting. For example you could have something like: ---%<------------------------------------------------------------------------- namespace { type = public prefix = Public/ #location = .. } plugin { quota = maildir:User quota quota2 = maildir:Shared quota:ns=Public/ #quota_rules and quota2_rules.. } ---%<------------------------------------------------------------------------- Custom Quota Exceeded Message ----------------------------- You can configure Dovecot to send a custom string instead of the default quota exceeded message. You could have something like: ---%<------------------------------------------------------------------------- plugin { quota_exceeded_message = Quota exceeded, please go to http://www.example.com/over_quota_help for instructions on how to fix this. } ---%<------------------------------------------------------------------------- Quota warnings -------------- You can configure Dovecot to run an external command when user's quota exceeds a specified limit. Note that the warning is ONLY executed at the exact time when the limit is being crossed, so when you're testing it you have to do it by crossing the limit by saving a new mail. If something else besides Dovecot updates quota so that the limit is crossed, the warning is never executed. The syntax is: ---%<------------------------------------------------------------------------- quota_warning =

#quota_warning2 = ... #quota_warning3 = ..etc.. ---%<------------------------------------------------------------------------- Limit configuration is the exact same as for rules. Usually you want to use percents instead of absolute limits. Only the command for the first exceeded limit is executed, so configure the highest limit first. The actual commands that are run need to be created as services. An example configuration: ---%<------------------------------------------------------------------------- plugin { quota_warning = storage=95%% quota-warning 95 %u quota_warning2 = storage=80%% quota-warning 80 %u } service quota-warning { executable = script /usr/local/bin/quota-warning.sh # use some unprivileged user for executing the quota warnings user = vmail unix_listener quota-warning { } } ---%<------------------------------------------------------------------------- With the above example when user's quota exceeds 80%, 'quota-warning.sh' is executed with parameter 80. The same goes for when quota exceeds 95%. If user suddenly receives a huge mail and the quota jumps from 70% to 99%, only the 95 script is executed. You have to create the 'quota-warning.sh' script yourself. Here is an example that sends a mail to the user: ---%<------------------------------------------------------------------------- #!/bin/sh PERCENT=$1 USER=$2 cat << EOF | /usr/local/libexec/dovecot/dovecot-lda -d $USER -o "plugin/quota=maildir:User quota:noenforcing" From: postmaster@domain.com Subject: quota warning Your mailbox is now $PERCENT% full. EOF ---%<------------------------------------------------------------------------- The quota enforcing is disabled to avoid looping. You'll of course need to change the 'plugin/quota' value to match the quota backend and other configuration you use. Basically preserve your original "quota" setting and just insert ":noenforcing" to proper location in it. For example with dict quota, you can use something like:'-o "plugin/quota=dict:User quota::noenforcing:proxy::quota"' Per-user quota limit configuration examples ------------------------------------------- SQL --- ---%<------------------------------------------------------------------------- # MySQL, quota in bytes: user_query = SELECT home, uid, gid, concat('*:bytes=', quota_bytes) AS quota_rule FROM users WHERE userid = '%u' # PostgreSQL, SQLite, quota in bytes: user_query = SELECT home, uid, gid, '*:bytes=' || quota_bytes AS quota_rule FROM users WHERE userid = '%u' ---%<------------------------------------------------------------------------- LDAP ---- ---%<------------------------------------------------------------------------- user_attrs = homeDirectory=home, uidNumber=uid, gidNumber=gid, quotaBytes=quota_rule=*:bytes=%$ ---%<------------------------------------------------------------------------- (This file was created from the wiki on 2011-01-13 04:52) wiki/AuthDatabase.LDAP.AuthBinds.txt000066600000006324150501203730013202 0ustar00Passdb LDAP with authentication binds ===================================== Advantages over [AuthDatabase.LDAP.PasswordLookups.txt]: * LDAP server verifies the password, so Dovecot doesn't need to know what format the password is stored in. * A bit more secure, as a security hole in Dovecot doesn't give attacker access to all the users' passwords. You can enable authentication binds by setting 'auth_bind=yes'. Next Dovecot needs to know what DN to use in the binding. There are two ways to configure this: lookup or template. DN lookup --------- DN is looked up by sending a 'pass_filter' LDAP request and getting the DN from the reply. This is very similar to doing a [AuthDatabase.LDAP.PasswordLookups.txt]. The only difference is that userPassword attribute isn't returned. Just as with password lookups, the 'pass_attrs' may contain special [PasswordDatabase.ExtraFields.txt]. Example: ---%<------------------------------------------------------------------------- auth_bind = yes pass_attrs = uid=user pass_filter = (&(objectClass=posixAccount)(uid=%u)) ---%<------------------------------------------------------------------------- DN template ----------- The main reason to use DN template is to avoid doing the DN lookup, so that the authentication consists only of one LDAP request. With IMAP and POP3 logins the same optimization can be done by using [UserDatabase.Prefetch.txt] and returning userdb info in the DN lookup (a total of two LDAP requests per login in both cases). If you're also using Dovecot for SMTP AUTH, it doesn't do a userdb lookup so the prefetch optimization doesn't help. If you're using DN template, 'pass_attrs' and 'pass_filter' settings are completely ignored. That means you can't make passdb return any [PasswordDatabase.ExtraFields.txt]. You should also set 'auth_username_format = %Lu' in 'dovecot.conf' to normalize the username by lowercasing it. Example: ---%<------------------------------------------------------------------------- auth_bind = yes auth_bind_userdn = cn=%u,ou=people,o=org ---%<------------------------------------------------------------------------- Connection optimization ----------------------- When using * auth binds and * userdb ldap lookups, the userdb lookups should use a separate connection to the LDAP server. That way it can send LDAP requests asynchronously to the server, which improves the performance. This can be done by specifying different filenames in the LDAP passdb and userdb args. The second file could be a symlink to the first one. For example: ---%<------------------------------------------------------------------------- passdb { driver = ldap args = /etc/dovecot/dovecot-ldap.conf.ext } userdb { driver = ldap args = /etc/dovecot/dovecot-ldap-userdb.conf.ext } ---%<------------------------------------------------------------------------- And create the symlink: ---%<------------------------------------------------------------------------- ln -s /etc/dovecot/dovecot-ldap.conf.ext /etc/dovecot/dovecot-ldap-userdb.conf.ext ---%<------------------------------------------------------------------------- (This file was created from the wiki on 2011-01-13 04:52) wiki/SSL.DovecotConfiguration.txt000066600000020736150501203730013053 0ustar00Dovecot SSL configuration ========================= The most important SSL settings are: ---%<------------------------------------------------------------------------- ssl = yes # Preferred permissions: root:root 0444 ssl_cert = [Authentication.Mechanisms.txt] only when SSL/TLS is used first. * 'ssl = required' requires SSL/TLS also for [Authentication.Mechanisms.txt]. * If you have only plaintext mechanisms enabled ('auth { mechanisms = plain login }'), you can use either (or both) of the above settings. They behave exactly the same way then. Multiple SSL certificates ------------------------- TLS SNI (Server Name Indication) support required ------------------------------------------------- /It is important to note that having multiple SSL certificates per IP will not be compatible with all clients, especially mobile ones. It is a TLS SNI limitation./ ---%<------------------------------------------------------------------------- local_name imap.example.org { ssl_cert = /var/lib/dovecot/ssl-parameters.ssl'. After the initial creation they're by default regenerated every week. With newer computers the generation shouldn't take more than a few seconds, but with older computers it can take as long as half an hour. The extra security gained by the regeneration is quite small, so with slower computers you might want to disable it: ---%<------------------------------------------------------------------------- ssl_parameters_regenerate = 0 ---%<------------------------------------------------------------------------- By default Dovecot's allowed ciphers list contains: ---%<------------------------------------------------------------------------- ssl_cipher_list = ALL:!LOW:!SSLv2:!EXP:!aNULL ---%<------------------------------------------------------------------------- Disallowing more won't really gain any security for those using better ciphers, but it does prevent people from accidentally using insecure ciphers. See http://www.openssl.org/docs/apps/ciphers.html for a list of the ciphers. SSL verbosity ------------- ---%<------------------------------------------------------------------------- verbose_ssl = yes ---%<------------------------------------------------------------------------- This will make Dovecot log all the problems it sees with SSL connections. Some errors might be caused by dropped connections, so it could be quite noisy. Client certificate verification/authentication ---------------------------------------------- If you want to require clients to present a valid SSL certificate, you'll need these settings: ---%<------------------------------------------------------------------------- ssl_ca = class3-revoke.pem ---%<------------------------------------------------------------------------- With the above settings if a client connects which doesn't present a certificate signed by one of the CAs in the 'ssl_ca' file, Dovecot won't let the user log in. You may also force the username to be taken from the certificate by setting 'ssl_username_from_cert = yes'. * The text is looked up from subject DN's specified field using OpenSSL's 'X509_NAME_get_text_by_NID()' function. * By default the 'CommonName' field is used. * You can change the field with 'ssl_cert_username_field = name' setting (parsed using OpenSSL's 'OBJ_txt2nid()' function). 'x500UniqueIdentifier' is a common choice. You may also want to disable the password checking completely. Doing this currently circumvents Dovecot's security model so it's not recommended to use it, but it is possible by making the [PasswordDatabase.txt] allow logins using any password (typically requiring <"nopassword" extra field> [PasswordDatabase.ExtraFields.txt] to be returned). (This file was created from the wiki on 2011-01-13 04:52) wiki/Plugins.Autocreate.txt000066600000001376150501203730011773 0ustar00Autocreate plugin ================= This plugin allows administrator to specify mailboxes that must always exist for all users. They can optionally also be subscribed. The mailboxes are created and subscribed always after user logs in. Namespaces are fully supported, so namespace prefixes need to be used where necessary. Example: ---%<------------------------------------------------------------------------- protocol imap { mail_plugins = $mail_plugins autocreate } plugin { autocreate = Trash autocreate2 = Spam #autocreate3 = ..etc.. autosubscribe = Trash autosubscribe2 = Spam #autosubscribe3 = ..etc.. } ---%<------------------------------------------------------------------------- (This file was created from the wiki on 2011-01-13 04:52) wiki/Design.Arrays.txt000066600000007510150501203730010724 0ustar00Dynamic Arrays ============== 'lib/array.h' and 'lib/array-decl.h' describes Dovecot's type-safe dynamic arrays. Trying to add wrong typed elements gives a compiler warning. Declaring --------- Arrays can be declared in two ways: 1. Directly: 'ARRAY_DEFINE(array_name, array_type);'. For example: 'ARRAY_DEFINE(numbers, int);' or 'ARRAY_DEFINE(foos, struct foo);' 2. Via predefined type: 'ARRAY_DEFINE_TYPE(foo, struct foo); ... ARRAY_TYPE(foo) foos;' The main reason to define a type for an array is to be able to pass the array as a function parameter, like: ---%<------------------------------------------------------------------------- void func(ARRAY_TYPE(foo) *foos) { .. } ---%<------------------------------------------------------------------------- Trying to do the same with 'ARRAY_DEFINE()' will generate a compiler warning. 'lib/array-decl.h' defines several commonly used types. Initializing ------------ Arrays are typically initialized by calling 'i_array_init()', 'p_array_init()' or 't_array_init()' depending on where you want to allocate the memory from. Arrays are internally handled as [Design.Buffers.txt], so the initial size is just multiplied by element size and passed to 'buffer_create_dynamic()'. Example: ---%<------------------------------------------------------------------------- ARRAY_DEFINE(foo, struct foo *); i_array_init(&foo, 32); /* initialize array with 32 elements until it needs to be grown */ ---%<------------------------------------------------------------------------- Arrays can be freed with 'array_free()', but this isn't necessary if the memory gets freed by other means (i.e. it was allocated from alloconly-pool or data stack). Writing ------- * 'array_append(array, data, count)' is the most common way to add data to arrays * 'array_append_array(dest, src)' * 'array_insert(array, idx, data, count)' * 'array_delete(array, idx, count)' * 'array_idx_set(array, idx, data)' replaces (or adds) data to given index * 'array_idx_clear(array, idx)' clears given index by writing NULs to it * 'array_append_space(array, count)' Reading ------- 'array_idx(array, idx)' returns pointer to given index in array. The index must already exist, otherwise the call assert-crashes. This call adds extra overhead for accessing arrays though, so usually it's better to just get list of all elements and access them directly: ---%<------------------------------------------------------------------------- data = array_get(&array, &count); ---%<------------------------------------------------------------------------- You can also iterate through the whole array easily: ---%<------------------------------------------------------------------------- const char *str; array_foreach(&string_array, str) { /* str changes in each iteration */ } ---%<------------------------------------------------------------------------- There's also 'array_foreach_modifiable()' to get the data without const. Unsafe Read/Write ----------------- Functions below have similar problems to [[Design/Buffer|buffer]'s '*_unsafe()' functions. Memory returned by them must not be accessed after calls to other 'array_*()' modifying functions, because they may reallocate the array elsewhere in memory. * 'array_append_space(array)' * 'array_insert_space(array, idx)' * 'array_get_modifiable(array, &count)' * 'array_idx_modifiable(array, idx)' Others ------ * 'array_cmp(array1, array2)' compares two arrays * 'array_reverse(array)' reverses all elements in an array * 'array_sort(array, cmp_func)' is a wrapper for 'qsort()' adding also type safety. The parameters in cmp_func should be the same type as the array, instead of 'const void *'. * 'array_bsearch(array, key, cmp_func)' is a wrapper for 'bsearch()' also adding type safety, just like 'array_sort()'. (This file was created from the wiki on 2011-01-13 04:52) wiki/UserDatabase.Static.txt000066600000002777150501203730012056 0ustar00Static User Database ==================== Static user database can be used when you want to use only single UID and GID values for all users, and their home directories can be specified with a simple template. The syntax is: ---%<------------------------------------------------------------------------- userdb { driver = static args = uid= gid= home= } ---%<------------------------------------------------------------------------- The home is optional. You can also return other [UserDatabase.ExtraFields.txt]. You can use the standard [Variables.txt] everywhere. Deliver ------- Unless your MTA already verifies that the user exists before calling deliver, you'll most likely want deliver itself to verify the user's existence. Since deliver looks up the user only from the userdb, it of course doesn't work with static userdb because there is no list of users. Normally static userdb handles this by doing a passdb lookup instead. This works with most passdbs, with [PasswordDatabase.PAM.txt] being the most notable exception. If you want to avoid this user verification, you can add 'allow_all_users=yes' to the args in which case the passdb lookup is skipped. Example ------- ---%<------------------------------------------------------------------------- userdb { driver = static args = uid=500 gid=500 home=/home/%u } ---%<------------------------------------------------------------------------- (This file was created from the wiki on 2011-01-13 04:52) wiki/Plugins.Compress.txt000066600000000776150501203730011475 0ustar00IMAP COMPRESS plugin ==================== The goal of the extension is to reduce the bandwidth usage of IMAP. The [Plugins.Zlib.txt] has to be enabled for imap_zlib to work Configuration: ---%<------------------------------------------------------------------------- mail_plugins = $mail_plugins zlib protocol imap { mail_plugins = $mail_plugins imap_zlib } ---%<------------------------------------------------------------------------- (This file was created from the wiki on 2011-01-13 04:52) wiki/Design.Indexes.MailIndexApi.txt000066600000004106150501203730013363 0ustar00Mail Index API ============== 'lib-index/mail-index.h' contains the functions to access the index files. 'mail-cache.h' contains the functions to access the cache file. The purpose of the main structures are: * 'struct mail_index': Global state of the index. * 'struct mail_index_view': You can have multiple views to the index. The views see new messages come and expunged messages go only when it's being explicitly synchronized. With mmaped indexes you can't really trust the record data (flags, keywords, extensions) not to change. This doesn't matter with IMAP. * 'struct mail_index_map': Index file is accessed via maps. Views can point to one or more maps. Maps can be shared by different views. Maps can contain either mmap()ed memory areas pointing to the index file, or a in-memory copy of it. * 'struct mail_index_transaction': In-memory list of changes to be written to the transaction log. The writing is done only when the transaction is committed. Views and maps -------------- In general you access all the data in the index files via views. The mails are accessed using sequence numbers, which change only when the view is synchronized. For accessing messages with their UIDs, you'll first need to convert them to sequences with either 'mail_index_lookup_uid()' or 'mail_index_lookup_uid_range()'. 'mail_index_lookup()' can be used to look up a single record's UID and flags. The returned record points to the latest map, so that it contains the latest flag changes. If the message was already expunged from the latest map, it returns 0. 'mail_index_lookup_full()' can be used to get also the map where the message was found. This can be important with extensions. If extension record's state depends on the extension header, they must be looked up from the same map. For this reason there exists 'mail_index_map_get_header_ext()' and 'mail_index_lookup_ext_full()' functions which take the map as parameter. The non-map versions return the data from the latest map if the message hasn't been expunged. (This file was created from the wiki on 2011-01-13 04:52) wiki/Design.Storage.Mailbox.Transaction.txt000066600000005525150501203730014751 0ustar00Mailbox Transactions ==================== Before you can read any mails or do any changes to mails, you need to create a transaction with 'mailbox_transaction_begin()'. It has a few flags: * 'MAILBOX_TRANSACTION_FLAG_HIDE': Mark the changes in a way that when later [Design.Storage.Mailbox.Sync.txt] the mailbox in this session, 'mailbox_sync_next()' won't return sync records for the changes done by this transaction. This is primarily meant for flag and keyword changes, you can't hide expunges. For example IMAP's 'STORE FLAGS.SILENT' command is implemented by setting this flag for the transaction. * 'MAILBOX_TRANSACTION_FLAG_EXTERNAL': Changes done by this transaction should be marked as external changes. Internal changes can be thought of as "change requests" that syncing later finishes, while external changes are done immediately and syncing ignores them. Normally you would use this flag when you want to save or copy messages, nothing else. * 'MAILBOX_TRANSACTION_FLAG_ASSIGN_UIDS': Require assigning UIDs to saved/copied messages immediately. Normally this is done only when it's easy (maildir: if dovecot-uidlist can be locked without waiting, mbox: if mbox is already fully synced). * 'MAILBOX_TRANSACTION_FLAG_REFRESH': Do a quick index refresh, so any recent flag/modseq changes done by other Dovecot sessions will be visible. You shouldn't usually need this, because usually you should have recently done a mailbox sync. Changes for a transaction are kept in memory until the transaction is committed. If you want to cancel the changes, you can call 'mailbox_transaction_rollback()'. Transaction can be commited with 'mailbox_transaction_commit()'. If you want to know a bit more about the results of the transaction, use 'mailbox_transaction_commit_get_changes()' instead. It returns a change structure: * uid_validity: UIDVALIDITY used by returned UIDs * saved_uids: UIDs assigned to saved/copied mails. Typically they're in an ascending order, unless you explicitly requested some specific UIDs for mails while saving them (e.g. dsync does this). * ignored_uid_changes: Number of UIDs that couldn't be changed by 'mail_update_uid()' calls, because the UIDs were less than next_uid's value. * ignored_modseq_changes: Number of modseqs that couldn't be changed by 'mail_update_modseq()' calls, because they would have lowered the modseq. Once you're done with reading the change structure, be sure to free the memory used by it with 'pool_unref(&changes->pool)'. 'mailbox_transaction_set_max_modseq()' can be used to implement atomic conditional flag changes. If message's modseq is higher than the given max_modseq while transaction is being committed, the change isn't done and the message's sequence number is added to the given array. (This file was created from the wiki on 2011-01-13 04:52) wiki/Design.Storage.Plugins.txt000066600000012362150501203730012510 0ustar00Mail Plugins ============ Typically plugins add hooks in their init() function by calling 'mail_storage_hooks_add()', and remove the hooks at deinit() with 'mail_storage_hooks_remove()'. Hooks that are currently supported: * mail_user_created: A new mail user was created. It doesn't yet have any namespaces. * mail_storage_created: A new mail storage was created. It's not connected to any namespaces/mailbox lists yet. * mailbox_list_created: A new mailbox list was created. It's not connected to any storages yet. Because of this, some internal virtual methods haven't been overridden by the storage yet, so plugins rarely want to use this hook. Instead they should use: * mail_namespace_storage_added: Storage was connected to its first namespace/mailbox list. This hook should usually be used if plugin wants to override mailbox_list's methods. * mail_namespaces_created: User's all namespaces have been created. This hook is called only per user at startup. More internal namespaces may be created later when using shared mailboxes. * mailbox_allocated: 'mailbox_alloc()' was called. * mailbox_opened: Mailbox (and its index) was actually opened, either explicitly with 'mailbox_open()' or implicitly by some other function. Overriding methods ------------------ When the hook gets called, you usually want to override some method of the created object. This is the easy part, for example: ---%<------------------------------------------------------------------------- static void plugin_mailbox_allocated(struct mailbox *box) .. box->v.transaction_begin = plugin_transaction_begin; ---%<------------------------------------------------------------------------- The problem is that once 'plugin_transaction_begin()' is called, it should call the original 'transaction_begin()'. There may also be multiple plugins that want to override the same method, so the idea is to just have each plugin call the previous 'transaction_begin()'. The next problem is where do you save the previous value? Most objects have a 'module_contexts' array for storing per-plugin pointers for this purpose. There are several helper functions to make setting and accessing them in a quite safe way. Easiest way to set up the module context is to just copy&paste code from an existing plugin that sets the same context. Here's some documentation about it anyway: First you start by creating register for the plugin. There are different registers for different types of objects: * mail_user_module_register: For mail_user. * mailbox_list_module_register: For mailbox_list. * mail_storage_module_register: For mail_storage, mailbox, mailbox_transaction and mail_search. * mail_module_register: For mail. We'll assume you want to use mail_storage_module_register: ---%<------------------------------------------------------------------------- static MODULE_CONTEXT_DEFINE_INIT(plugin_storage_module, &mail_storage_module_register); ---%<------------------------------------------------------------------------- If you need to make it external, use: ---%<------------------------------------------------------------------------- extern MODULE_CONTEXT_DEFINE(plugin_storage_module, &mail_storage_module_register); struct plugin_storage_module plugin_storage_module = MODULE_CONTEXT_INIT(&mail_storage_module_register); ---%<------------------------------------------------------------------------- Next you'll need to allocate memory for the structure you want to place in the context. If you only want to override some methods, you can use: ---%<------------------------------------------------------------------------- union mailbox_module_context *mbox; struct mailbox_vfuncs *v = box->vlast; mbox = p_new(box->pool, union mailbox_module_context, 1); mbox->super = *v; box->vlast = &mbox->super; v->transaction_begin = plugin_transaction_begin; MODULE_CONTEXT_SET_SELF(box, plugin_storage_module, mbox); ---%<------------------------------------------------------------------------- If you want to store some more plugin-specific data to the object instead of just the super methods, you can do: ---%<------------------------------------------------------------------------- struct plugin_mailbox { /* must be called module_ctx */ union mailbox_module_context module_ctx; }; /* .. */ struct plugin_mailbox *mbox; struct mailbox_vfuncs *v = box->vlast; mbox = p_new(box->pool, struct plugin_mailbox, 1); mbox->module_ctx.super = *v; box->vlast = &mbox->super; v->transaction_begin = plugin_transaction_begin; MODULE_CONTEXT_SET(box, plugin_storage_module, mbox); ---%<------------------------------------------------------------------------- Note that when using union directly you use 'MODULE_CONTEXT_SET_SELF()', while when it's inside a struct you use 'MODULE_CONTEXT_SET()'. Once all this initialization is done, you can look up the module context with: ---%<------------------------------------------------------------------------- #define PLUGIN_CONTEXT(obj) MODULE_CONTEXT(obj, plugin_storage_module) /* .. */ struct plugin_mailbox *mbox = PLUGIN_CONTEXT(box); ---%<------------------------------------------------------------------------- (Yes, this API seems a bit too difficult to use and could use a redesign.) (This file was created from the wiki on 2011-01-13 04:52) wiki/Director.txt000066600000012512150501203730010024 0ustar00Director ======== Director can be used by [PasswordDatabase.ExtraFields.Proxy.txt] to keep a temporary user -> mail server mapping. As long as user has simultaneous connections, the user is always redirected to the same server. Each proxy server is running its own director process, and the directors are communicating the state to each others. Directors are mainly useful for setups where all of the mail storage is seen by all servers, such as with NFS or a cluster filesystem. First test non-director proxying -------------------------------- The director is simply a small add-on for Dovecot proxy. Before configuring director, you should test that a simple proxying setup with static destination server works. See the [PasswordDatabase.ExtraFields.Proxy.txt] page for more information about how to configure it. If you have a simple setup, you can test this easily using a static passdb: ---%<------------------------------------------------------------------------- passdb { driver = static args = proxy=y host=10.2.0.20 nopassword=y } ---%<------------------------------------------------------------------------- Once finished testing, remember to remove the "host" field. Servers ------- You need one or more servers assigned for Dovecot proxies. The same servers could also act as backends handling the mails, but you need to run [RunningDovecot.txt] in different ports. This may get a bit confusing, so it's not recommended. The directors are going to connect to each others in a ring. For example if you have servers called A, B and C, director will create connections A->B, B->C and C->A. Director configuration ---------------------- In example configuration you can configure director from 'conf.d/10-director.conf'. Listeners --------- Configure the listeners that director requires: ---%<------------------------------------------------------------------------- service director { unix_listener login/director { mode = 0666 } fifo_listener login/proxy-notify { mode = 0666 } unix_listener director-userdb { mode = 0600 } inet_listener { port = 9090 } } ---%<------------------------------------------------------------------------- The port 9090 will be used for listening and connecting to other directors. You're free to use any port number you want. Configuring list of director servers ------------------------------------ List all of your directors in 'director_servers' setting separated by spaces. You can use: * IP addresses * hostnames * hostnames that expand to multiple IPs (e.g. you could have a "directors-all" DNS entry that expands to all directors' IPs) You can also add :port after the IP/host. The default port is the same as what director service's inet_listener is using (the port 9090 above). For example if you have 3 directors, you could set: ---%<------------------------------------------------------------------------- director_servers = 10.1.0.2 10.1.0.3 10.1.0.4 ---%<------------------------------------------------------------------------- Configuring list of mail servers -------------------------------- List all of your backend mail servers in 'director_mail_servers' setting separated by spaces. You can use: * IP addresses * IP ranges (e.g. 10.2.0.10-10.2.0.30) * hostnames * hostnames that expand to multiple IPs For example if you had 20 mail servers with consecutive IPs: ---%<------------------------------------------------------------------------- director_mail_servers = 10.2.0.11-10.2.0.30 ---%<------------------------------------------------------------------------- Enabling director ----------------- Enable director for the wanted login services by telling them to connect to director socket instead of the default login socket: ---%<------------------------------------------------------------------------- service imap-login { executable = imap-login director } service pop3-login { executable = pop3-login director } ---%<------------------------------------------------------------------------- If you want to enable director for LMTP, also set: ---%<------------------------------------------------------------------------- lmtp_proxy = yes protocol lmtp { auth_socket_path = director-userdb } ---%<------------------------------------------------------------------------- Passdb configuration -------------------- Your passdb must return "proxy" [PasswordDatabase.ExtraFields.txt], otherwise director doesn't do anything. Director works by adding a "host" extra field to the auth reply, which contains the temporary destination mail server. This "host" field isn't added if the passdb lookup already returns "host". This allows configuring some users to be always proxied to a specific server. If the backend servers verify password, you can use static passdb for director: ---%<------------------------------------------------------------------------- passdb { driver = static args = proxy=y nopassword=y } ---%<------------------------------------------------------------------------- Health monitoring of backend servers ------------------------------------ Brad Davidson has written a small daemon for monitoring backend servers, and disable/enable them on demand. Ref:http://www.dovecot.org/list/dovecot/2010-August/051946.html (This file was created from the wiki on 2011-01-13 04:52) wiki/SharedMailboxes.Shared.txt000066600000023156150501203730012536 0ustar00Mailbox sharing between users ============================= To enable mailbox sharing, you'll need to create a shared namespace. See for more information about ACL-specific settings. ---%<------------------------------------------------------------------------- # User's private mail location. mail_location = maildir:~/Maildir # When creating any namespaces, you must also have a private namespace: namespace { type = private separator = / prefix = #location defaults to mail_location. inbox = yes } namespace { type = shared separator = / prefix = shared/%%u/ location = maildir:%%h/Maildir:INDEX=~/Maildir/shared/%%u subscriptions = no list = children } mail_plugins = acl protocol imap { mail_plugins = $mail_plugins imap_acl } plugin { acl = vfile } ---%<------------------------------------------------------------------------- This creates a shared/ namespace under which each user's mailboxes are. If you have multiple domains and allow sharing between them, you might want to set 'prefix=shared/%%d/%%n/' instead (although %%u works just fine too). If you don't, you might want to drop the domain part and instead use 'prefix=shared/%%n/'. 'list=children' specifies that if no one has shared mailboxes to the user, the "shared" directory isn't listed by the LIST command. If you wish it to be visible always, you can set 'list=yes'. The 'location' setting specifies how to access other users' mailboxes. If you use %%h, the user's home directory is asked from auth process via auth-userdb socket. See for how to configure the socket. If the users' mailboxes can be found using a template, it's faster not to use the %%h. For example: ---%<------------------------------------------------------------------------- location = maildir:/var/mail/%%d/%%n/Maildir:INDEX=~/Maildir/shared/%%u ---%<------------------------------------------------------------------------- % vs %% ------- %var expands to the logged in user's variable, while %%var expands to the other users' variables. For example if your name is "myself" and "someone1" and "someone2" have shared mailboxes to you, the variables could be expanded like: * %u expands to "myself" * %%u expands to "someone1" or "someone2" * %h might expand to "/home/myself" * %%h might expand to "/home/someone1" or "/home/someone2" * ~/ equals %h/ Note that in e.g. mail_location setting you might need both. For example in: ---%<------------------------------------------------------------------------- mail_location = maildir:%%h/Maildir:INDEX=%h/Maildir/shared/%%u ---%<------------------------------------------------------------------------- What it means is: * %%h/Maildir points to the other user's Maildir, e.g. "/home/someone1". * :INDEX=%h/Maildir/shared/%%u points to a per-user directory under your own Maildir, e.g. "/home/myself/Maildir/someone1" or "/home/myself/Maildir/someone2". This is necessary to keep a local copy of the other users' index files. dbox ---- With dbox the index files are a very important part of the mailboxes. You must not try to :INDEX= to have copies of index files. This will only result in mailbox corruption. This also means that with dbox there's currently no way to have private \Seen flags. Filesystem permissions ---------------------- Dovecot assumes that it can access the other users' mailboxes. If you use multiple UNIX UIDs, you may have problems setting up the permissions so that the mailbox sharing works. Dovecot never modifies existing files' permissions. See for more information. Shared mailbox listing ---------------------- With the above configuration it's possible to open shared mailboxes if you know their name, but they won't be visible in the mailbox list. This is because Dovecot has no way of knowing what users have shared mailboxes to whom. Iterating through all users and looking inside their mail directories would be horribly inefficient for more than a couple users. To overcome this problem Dovecot needs a dictionary, which contains the list of users who have shared mailboxes and to whom they have shared. If the users aren't properly listed in this dictionary, their shared mailboxes won't be visible. Currently there's no way to automatically rebuild this dictionary, so make sure it doesn't get lost. If it does, each user having shared mailboxes must use the IMAP SETACL command (see below) to get the dictionary updated for themselves. You could use any dictionary backend, including SQL, but a simple flat file should work pretty well too: ---%<------------------------------------------------------------------------- plugin { acl_shared_dict = file:/var/lib/dovecot/shared-mailboxes.db } ---%<------------------------------------------------------------------------- If you use multiple domains and don't wish users to share their mailboxes to users in other domains, you can use separate dict files for each domain: ---%<------------------------------------------------------------------------- plugin { # assumes mailboxes are in /var/mail/%d/%n: acl_shared_dict = file:/var/mail/%d/shared-mailboxes.db } ---%<------------------------------------------------------------------------- Using SQL dictionary -------------------- 'dovecot.conf': ---%<------------------------------------------------------------------------- plugin { acl_shared_dict = proxy::acl } dict { acl = pgsql:/etc/dovecot/dovecot-dict-sql.conf.ext } ---%<------------------------------------------------------------------------- See for more information, especially about permission issues. Database tables: ---%<------------------------------------------------------------------------- CREATE TABLE user_shares ( from_user varchar(100) not null, to_user varchar(100) not null, dummy char(1) DEFAULT '1', -- always '1' currently primary key (from_user, to_user) ); COMMENT ON TABLE user_shares IS 'User from_user shares folders to user to_user.'; CREATE TABLE anyone_shares ( from_user varchar(100) not null, dummy char(1) DEFAULT '1', -- always '1' currently primary key (from_user) ); COMMENT ON TABLE anyone_shares IS 'User from_user shares folders to anyone.'; ---%<------------------------------------------------------------------------- '/etc/dovecot/dovecot-dict-sql.conf.ext': ---%<------------------------------------------------------------------------- connect = host=localhost dbname=mails user=sqluser password=sqlpass map { pattern = shared/shared-boxes/user/$to/$from table = user_shares value_field = dummy fields { from_user = $from to_user = $to } } map { pattern = shared/shared-boxes/anyone/$from table = anyone_shares value_field = dummy fields { from_user = $from } } ---%<------------------------------------------------------------------------- IMAP ACL commands ----------------- Mailbox sharing is expected to be done using IMAP SETACL command. It is the only way to update the shared mailbox list dictionary. Below is a quick introduction to IMAP ACL commands. See RFC 4314 [http://www.ietf.org/rfc/rfc4314.txt] for more details. * MYRIGHTS : Returns the user's current rights to the mailbox. * GETACL : Returns the mailbox's all ACLs. * SETACL [+|-]: Give the specified rights to the mailbox. * DELETEACL [-]: Delete 's ACL from the mailbox. is one of: * anyone: Matches all users, including anonymous users. * authenticated: Like "anyone", but doesn't match anonymous users. * $group: Matches all users belonging to the group ($ is not part of the group name). * $!group: See group-override in (Dovecot-specific feature). * user: Matches the given user. The $group syntax is not a standard, but it is mentioned in RFC 4314 examples and is also understood by at least Cyrus IMAP. Having '-' before the identifier specifies negative rights. See for list of . Sharing mailboxes to everyone ----------------------------- By default Dovecot doesn't allow using the IMAP "anyone" or "authenticated" identifier, because it would be an easy way to spam other users in the system. If you wish to allow it, set: ---%<------------------------------------------------------------------------- plugin { acl_anyone = allow } ---%<------------------------------------------------------------------------- Note that you can also do this only for some users by using the second table "anyone_shares". Every user listed in this table shares his folders with everyone. See also [UserDatabase.ExtraFields.txt]. IMAP ACL examples ----------------- Let's begin with some simple example that first gives "read" and "lookup" rights, and later adds "write-seen" right: ---%<------------------------------------------------------------------------- 1 SETACL Work user@domain rl 1 OK Setacl complete. 2 SETACL Work user@domain +s 2 OK Setacl complete. 3 GETACL Work * ACL "Work" "user@domain" lrs "myself" lrwstipekxacd 3 OK Getacl completed. ---%<------------------------------------------------------------------------- Let's see how negative rights work by testing it on ourself. See how we initially have "lookup" right, but later we don't: ---%<------------------------------------------------------------------------- 1 MYRIGHTS Work * MYRIGHTS "Work" lrwstipekxacd 1 OK Myrights completed. 2 SETACL Work -myself l 2 OK Setacl complete. 3 GETACL Work * ACL "Work" "-myself" l "user@domain" lr "myself" lrwstipekxacd 3 OK Getacl completed. 4 myrights Work * MYRIGHTS "Work" rwstipekxacd 4 OK Myrights completed. ---%<------------------------------------------------------------------------- (This file was created from the wiki on 2011-01-13 04:52) wiki/Plugins.FTS.Lucene.txt000066600000001451150501203730011537 0ustar00Lucene Full Text Search Indexing ================================ The current Lucene implementation is a bit ugly and simultaneous access may give some errors. Use [Plugins.FTS.Solr.txt] instead if possible. Dovecot builds only a single Lucene index for all mailboxes. In future this would allow support for searching messages quickly from all mailboxes. The Lucene indexes are stored in 'lucene-indexes/' directory under the root index directory (e.g.'~/Maildir/lucene-indexes/'). Configuration ------------- ---%<------------------------------------------------------------------------- mail_plugins = $mail_plugins fts fts_lucene plugin { fts = lucene } ---%<------------------------------------------------------------------------- (This file was created from the wiki on 2011-01-13 04:52) wiki/WhyDoesItNotWork.txt000066600000004761150501203730011463 0ustar00Why doesn't Dovecot work? ========================= * *Dovecot always logs an error message* if anything goes wrong, so make sure you're looking at the correct log files. Debug messages may be written to a different log file than error messages. See . * Try logging in manually by sending IMAP commands. If you're trying with an IMAP client you can't be sure if the problem is with the client's configuration or Dovecot's configuration. Many IMAP clients handle all errors simply by showing you the login password dialog, even if the problem has nothing to do with authentication. See (or ). * Are you using an older version than v1.0? RHEL 5.0 and Debian Etch distribute v1.0.rc15, and there have been a LOT of fixes since then. If you have a problem, see if NEWS [http://dovecot.org/doc/NEWS] file mentions anything related to it. There are also [PrebuiltBinaries.txt] available for newer versions. It says "Authentication failed" ------------------------------- First of all enable 'auth_debug_passwords=yes' and see if the logs show what the problem is. For non-PAM setups it should contain all the information needed to solve the problem. If it's trying to use a wrong password scheme, you can change that. See . If the client isn't even attempting to log in, you most likely have 'disable_plaintext_auth=yes' (default) and the client isn't configured to use SSL/TLS (or you've also set 'ssl=no'). If you're using [PasswordDatabase.PAM.txt]: * Make sure that dovecot-auth is running as root (assuming it's using '/etc/shadow'). * PAM errors aren't written to Dovecot's own logs. Usually they go to '/var/log/auth.log' or something similar. * Unfortunately PAM's error messages aren't always all that helpful in figuring out what exactly the problem is. * You could (temporarily) try to use [PasswordDatabase.Shadow.txt] instead to see if it logs something more understandable. * Usually the problem is that you don't have a correctly named file in '/etc/pam.d/'. See . It's not finding my emails -------------------------- 'mail_debug=yes' makes Dovecot log where it's really looking for mails. Also 'auth_debug=yes' may be helpful in debugging. See for how to configure where the mails are looked up from. (This file was created from the wiki on 2011-01-13 04:52) wiki/LDA.txt000066600000030243150501203730006652 0ustar00Dovecot LDA =========== The Dovecot LDA is a [MDA.txt], which takes mail from an and delivers it to a user's mailbox, while keeping Dovecot index files up to date. An alternative is to use [LMTP.txt], but most of the configuration between them is identical. This page describes the common settings required to make LDA work. You should read it first, and then the MTA specific pages: * * * * * Main features of Dovecot LDA ---------------------------- * [LDA.Indexing.txt], providing faster mailbox access later * [Quota.txt] * [Pigeonhole.Sieve.txt] * Mail filtering * Mail forwarding * Vacation auto-reply Common configuration -------------------- The configuration is done in the /protocol lda/ section in 'dovecot.conf'. The important settings are: * 'postmaster_address' is used as the From: header address in bounce mails * 'hostname' is used in generated Message-IDs and in Reporting-UA: header in bounce mails * 'sendmail_path' is used to send mails. Note that the default is '/usr/lib/sendmail', which doesn't necessarily work the same as '/usr/sbin/sendmail'. * 'auth_socket_path' specifies the UNIX socket to auth-userdb where LDA can lookup userdb information when '-d' parameter is used. See below how to configure Dovecot to configure the socket. Note that 'dovecot.conf' file must be world readable to enable dovecot-lda process read it, while running with user privileges. Parameters ---------- Parameters accepted by dovecot-lda: * '-d ': Destination username. If given, the user information is looked up from dovecot-auth. Typically used with virtual users, but not necessarily with system users. * '-a

': Original envelope recipient address (e.g. user+ext@domain), typically same as SMTP's RCPT TO: value. If not specified, it's taken from header specified by 'lda_original_recipient_header' setting (v2.0.3+). If the header doesn't exist either, defaults to same as username. * '-r

': Final envelope recipient address. Defaults to -a address, but may differ if e.g. aliases are used or when dropping the +ext part. (v2.0.3+) * '-f

': Envelope sender address. If not specified and message data begins with a valid mbox-style "From " line, the address is taken from it. * '-c ': Alternative configuration file path. * '-m ': Destination mailbox (default is INBOX). If the mailbox doesn't exist, it will not be created (unless the lda_mailbox_autocreate setting is set to yes). If message couldn't be saved to the mailbox for any reason, it's delivered to INBOX instead. * If Sieve plugin is used, this mailbox is used as the "keep" action's mailbox. It's also used if there is no Sieve script or if the script fails for some reason. * Deliveries to namespace prefix will result in saving the mail to INBOX instead. For example if you have "Mail/" namespace, this allows you to specify 'dovecot-lda -m Mail/$mailbox' where mail is stored to Mail/$mailbox or to INBOX if $mailbox is empty. * The mailbox name is specified the same as it's visible in IMAP client. For example if you've a Maildir with '.box.sub/' directory and your namespace configuration is 'prefix=INBOX/', 'separator=/', the correct way to deliver mail there is to use '-m INBOX/box/sub' * '-e': If mail gets rejected, write the rejection reason to stderr and exit with EX_NOPERM. The default is to send a rejection mail ourself. * '-k': Don't clear all environment at startup. * '-p ': Path to the mail to be delivered instead of reading from stdin. If using maildir the file is hard linked to the destination if possible. This allows a single mail to be delivered to multiple users using hard links, but currently it also prevents dovecot-lda from updating cache file so it shouldn't be used unless really necessary. Return values ------------- dovecot-lda will exit with one of the following values: * 0 (EX_OK): Delivery was successful * 64 (EX_USAGE): Invalid parameter given. * 77 (EX_NOPERM): -e parameter was used and mail was rejected. Typically this happens when user is over quota and 'quota_full_tempfail=no'. * 75 (EX_TEMPFAIL): A temporary failure. This is returned for almost all failures. See the log file for details. System users ------------ You can use LDA with a few selected system users (ie. user is found from '/etc/passwd' / NSS) by calling dovecot-lda in the user's '~/.forward' file: ---%<------------------------------------------------------------------------- | "/usr/local/libexec/dovecot/dovecot-lda" ---%<------------------------------------------------------------------------- This should work with any MTA which supports per-user '.forward' files. For qmail's per-user setup, see . This method doesn't require the authentication socket explained below since it's executed as the user itself. Virtual users ------------- With a lookup ------------- Give the destination username to dovecot-lda with '-d' parameter, for example: ---%<------------------------------------------------------------------------- dovecot-lda -f $FROM_ENVELOPE -d $DEST_USERNAME ---%<------------------------------------------------------------------------- You'll need to set up a auth-userdb socket for dovecot-lda so it knows where to find mailboxes for the users: ---%<------------------------------------------------------------------------- service auth { unix_listener auth-userdb { mode = 0600 user = vmail # User running dovecot-lda #group = vmail # Or alternatively mode 0660 + dovecot-lda user in this group } } ---%<------------------------------------------------------------------------- The auth-userdb socket can be used to do [UserDatabase.txt] lookups for given usernames or get a list of all users. Typically the result will contain the user's UID, GID and home directory, but depending on your configuration it may return other information as well. So the information is similar to what can be found from eg.'/etc/passwd' for system users. This means that it's probably not a problem to use mode=0666 for the socket, but you should try to restrict it more just to be safe. Without a lookup ---------------- If you have already looked up the user's home directory and you don't need a userdb lookup for any other reason either (such as overriding settings for specific users), you can run dovecot-lda similar to how it's run for system users: ---%<------------------------------------------------------------------------- HOME=/path/to/user/homedir dovecot-lda -f $FROM_ENVELOPE ---%<------------------------------------------------------------------------- This way you don't need to have a master listener socket. Note that you should verify the user's existence prior to running dovecot-lda, otherwise you'll end up having mail delivered to nonexistent users as well. You must have set the proper UID (and GID) before running dovecot-lda. It's not possible to run dovecot-lda as root without '-d' parameter. Multiple UIDs ------------- If you're using more than one UID for users, you're going to have problems running dovecot-lda, as most MTAs won't let you run dovecot-lda as root. Best solution is to use instead, but if you can't do that, there are two ways to work around this problem: 1. Make dovecot-lda setuid-root. 2. Use sudo to wrap the invocation of dovecot-lda. Making dovecot-lda setuid-root: ------------------------------- Beware: *it's insecure to make dovecot-lda setuid-root*, especially if you have untrusted users in your system.*Setuid-root dovecot-lda can be used to gain root privileges*. You should take extra steps to make sure that untrusted users can't run it and potentially gain root privileges. You can do this by making sure only your MTA has execution access to it. For example: ---%<------------------------------------------------------------------------- # chgrp secmail /usr/local/libexec/dovecot/dovecot-lda # chmod 04750 /usr/local/libexec/dovecot/dovecot-lda # ls -l /usr/local/libexec/dovecot/dovecot-lda -rwsr-x--- 1 root secmail 4023932 2010-06-15 16:23 dovecot-lda ---%<------------------------------------------------------------------------- Then start dovecot-lda as a user that belongs to secmail group. Note that you have to recreate these rights after each update of dovecot. Using sudo: ----------- Alternatively, you can use sudo to wrap the invocation of dovecot-lda. This has the advantage that updates will not clobber the setuid bit, but note that *it is just as insecure being able to run dovecot-lda via sudo as setuid-root*. Make sure you only give your MTA the ability to invoke dovecot-lda via sudo. First configure sudo to allow 'dovelda' user to invoke dovecot-lda by adding the following to your '/etc/sudoers': ---%<------------------------------------------------------------------------- Defaults:dovelda !syslog dovelda ALL=NOPASSWD:/usr/local/libexec/dovecot/dovecot-lda ---%<------------------------------------------------------------------------- Then configure your MTA to invoke dovecot-lda as user 'dovelda' and via sudo: ---%<------------------------------------------------------------------------- /usr/bin/sudo /usr/local/libexec/dovecot/dovecot-lda ---%<------------------------------------------------------------------------- instead of just plain '/usr/local/libexec/dovecot/dovecot-lda'. Problems with dovecot-lda ------------------------- * If you are using [UserDatabase.Prefetch.txt], keep in mind that 'dovecot-lda' does not make a password query and thus will not work if '-d' parameter is used. The page explains how to fix this. * See [PasswordDatabase.CheckPassword.txt] for how to make dovecot-lda work with checkpassword. Logging ------- * Normally Dovecot logs everything through its log process, which is running as root. dovecot-lda doesn't, which means that you might need some special configuration for it to log anything at all. * If dovecot-lda fails to write to log files it exits with temporary failure. * If you have trouble finding where Dovecot logs by default, see . * Note that Postfix's 'mailbox_size_limit' setting applies to all files that are written to. So if you have a limit of 50 MB, dovecot-lda can't write to log files larger than 50 MB and you'll start getting temporary failures. If you want dovecot-lda to keep using Dovecot's the default log files: * If you're logging to syslog, make sure the syslog socket (usually '/dev/log') has enough write permissions for dovecot-lda. For example set it world-read/writable:'chmod a+rw /dev/log'. * If you're logging to Dovecot's default log files again you'll need to give enough write permissions to the log files for dovecot-lda. You can also specify different log files for dovecot-lda. This way you don't have to give any extra write permissions to other log files or the syslog socket. You can do this by overriding the 'log_path' and 'info_log_path' settings: ---%<------------------------------------------------------------------------- protocol lda { .. # remember to give proper permissions for these files as well log_path = /var/log/dovecot-lda-errors.log info_log_path = /var/log/dovecot-lda.log } ---%<------------------------------------------------------------------------- For using syslog with dovecot-lda, set the paths empty: ---%<------------------------------------------------------------------------- protocol lda { .. log_path = info_log_path = # You can also override the default syslog_facility: #syslog_facility = mail } ---%<------------------------------------------------------------------------- Plugins ------- * Most of the [Plugins.txt] work with dovecot-lda. * Virtual quota can be enforced using [Quota.txt]. * Sieve language support can be added with the [Pigeonhole.Sieve.txt]. (This file was created from the wiki on 2011-01-13 04:52) wiki/SharedMailboxes.txt000066600000001242150501203730011321 0ustar00Shared mailboxes ================ Dovecot can support mailbox sharing in several different ways: * [SharedMailboxes.Public.txt]: Shared mailboxes created by administrators (ACLs can still restrict who sees them) * [SharedMailboxes.Shared.txt]: Users sharing their mailboxes to other users (using IMAP ACL commands) (v1.2+) * [SharedMailboxes.Symlinks.txt]: Quick and dirty way of sharing a few mailboxes. See for common filesystem related permission problems that are common with all the sharing methods. (This file was created from the wiki on 2011-01-13 04:52) wiki/HowTo.EximAndDovecotSASL.txt000066600000002064150501203730012645 0ustar00Exim and Dovecot SASL ===================== Exim v4.64+ users can use Dovecot SASL instead of Cyrus SASL for authenticating SMTP clients. conf.d/10-master.conf --------------------- ---%<------------------------------------------------------------------------- service auth { ... #SASL unix_listener auth-client { mode = 0660 user = mail } ... } ---%<------------------------------------------------------------------------- exim.conf --------- Create authenticators for Dovecot: ---%<------------------------------------------------------------------------- dovecot_login: driver = dovecot public_name = LOGIN server_socket = /var/run/dovecot/auth-client # setting server_set_id might break several headers in mails sent by authenticated smtp. So be careful. server_set_id = $auth1 dovecot_plain: driver = dovecot public_name = PLAIN server_socket = /var/run/dovecot/auth-client server_set_id = $auth1 ---%<------------------------------------------------------------------------- (This file was created from the wiki on 2011-01-13 04:52) wiki/Quota.Maildir.txt000066600000004473150501203730010731 0ustar00Maildir++ quota =============== Maildir++ is the most commonly used quota backend with Maildir format. Note that *Maildir++ quota works only with Maildir format*. With other mailbox formats you should use [Quota.Dict.txt]. Dovecot implements the standard Maildir++ specification [http://www.inter7.com/courierimap/README.maildirquota.html], so it's compatible with Courier [http://www.courier-mta.org/], maildrop [http://www.courier-mta.org/maildrop/], Exim [http://www.exim.org], etc. There are two ways to configure Maildir++ quota limits: 1. Configure the limits in Dovecot. You most likely want to do this. See [Quota.txt] for how to do this configuration. 2. Make Dovecot get the limits from existing 'maildirsize' files. Only Maildir++-specific settings are described below. See for more generic configuration. Maildir++ quota relies on 'maildirsize' file having correct information, so if your users can modify the file in some way (e.g. shell access), you're relying on the goodwill of your users for the quota to work. You can't rely on Dovecot noticing external changes to Maildir and updating maildirsize accordingly. This happens eventually when quota is being recalculated, but it may take a while. Quota recalculation also currently doesn't trigger quota warning executions. Maildirsize file ---------------- The 'maildirsize' file in the Maildir root directory contains both the quota limit information and the current quota status. It contains a header in format: ---%<------------------------------------------------------------------------- S,C ---%<------------------------------------------------------------------------- If you don't configure any quota limits in Dovecot ('quota=maildir' with no other settings), Dovecot takes the limits from the header. If the file does not exist, quota isn't enforced. If you configure quota limits in Dovecot, Dovecot makes sure that this header is kept up to date. If the file does not exist, it's simply rebuilt. Once the 'maildirsize' reaches 5120 bytes, the quota is recalculated and the file is recreated. This makes sure that if quota happens to be broken (e.g. externally deleted files) it won't stay that way forever. (This file was created from the wiki on 2011-01-13 04:52) wiki/PasswordDatabase.ExtraFields.AllowNets.txt000066600000001107150501203730015616 0ustar00Allow_nets extra field ---------------------- This is a comma separated list of IPs and/or networks where the user is allowed to log in from. If the user tries to log in elsewhere, the authentication will fail the same way as if a wrong password was given. Example: 'allow_nets=127.0.0.0/8,192.168.0.0/16,1.2.3.4,4.5.6.7'. IPv6 addresses are also allowed. IPv6 mapped IPv4 addresses (eg. '::ffff:1.2.3.4') are converted to standard IPv4 addresses before matching. Example:'allow_nets=::1,2001:abcd:abcd::0:0/80,1.2.3.4' (This file was created from the wiki on 2011-01-13 04:52) wiki/OSCompatibility.txt000066600000003140150501203730011321 0ustar00Operating System Compatibility ============================== Dovecot is commonly used with Linux, Solaris, FreeBSD, OpenBSD, NetBSD and Mac OS X. The following operating systems have also worked at least at some point in Dovecot's existence: * BSD/OS 4.2 * AIX 4.3 * HP-UX 11i (Dovecot v1.1+) * Tru64 ?.? * 7.1.4 * IRIX 6.5 compiles, but SCM_RIGHTS seems to be broken. * Cygwin apparently doesn't work nowadays. * MidnightBSD (ports) If there are compiling problems with any OS, please send a bug report that includes the error messages. SCM_RIGHTS ---------- There is one slightly problematic feature that Dovecot requires for implementing privilege separation:*SCM_RIGHTS*. If it doesn't work correctly, you'll get errors on 'fd_send()' or 'fd_read()' such as: ---%<------------------------------------------------------------------------- imap-login: fd_send(X) failed: Bad file number ---%<------------------------------------------------------------------------- If this happens, you can still use inetd and mailfront [http://untroubled.org/mailfront/], which executes Dovecot's post-login IMAP or POP3 binary. You can also try defining 'BUGGY_CMSG_MACROS' in 'src/lib/fdpass.c' to see if it helps. Compilers --------- Dovecot should compile with any ANSI-C99 compiler. Dovecot has been known to compile (at least once in its lifetime) with the following compilers: * GCC * Intel CC [http://www.intel.com/software/products/compilers/clin/] * Tiny CC [http://www.tinycc.org] * Sun Studio 11 * Sun Studio 12 * AIX xlC * HP-UX cc (This file was created from the wiki on 2011-01-13 04:52) wiki/SharedMailboxes.Permissions.txt000066600000014371150501203730013642 0ustar00Filesystem permissions in shared mailboxes ========================================== IMAP processes need filesystem level permissions to access shared/public mailboxes. This means that: * If you use more than one [UserIds.txt] for your mail users (e.g. you use system users), you'll need to make sure that all users can access the mailboxes on filesystem level. ( [ACL.txt] won't help you with this.) * You can remove write permissions on purpose from public namespace root directory to prevent users from creating new mailboxes under it. Dovecot never modifies permissions for existing mail files or directories. When users share mailboxes between each others, the system must have been set up in a way that filesystem permissions don't get in the way. The easiest way to do that is to use only a single UID. Another possibility would be to use one or more groups for all the mail files that may be shared to other users belonging to the same group. For example if you host multiple domains, you might create a group for each domain and allow mailbox sharing (only) between users in the same domain. System user UNIX groups ----------------------- There's no requirement to use UNIX groups (i.e. typically defined in '/etc/group') for anything. If you don't care about them, you can safely ignore this section. If you use [AuthDatabase.Passwd.txt] userdb, the IMAP process has access to all the UNIX groups defined for that user. You may use these groups when granting filesystem permissions. If you wish to use UNIX groups defined in '/etc/group' but don't use passwd userdb, you can still do this by returning 'system_groups_user' [UserDatabase.ExtraFields.txt], which contains the UNIX user name whose groups are read from the group file. You can also set up extra UNIX groups by listing them in 'mail_access_groups' setting. To have per-user UNIX groups, return 'mail_access_groups' as userdb extra field. The advantage of using this method is that only Dovecot mail processes have access to the group, but nothing else, such as user's SSH session. For example a simple way to set up shared mailbox access for all system users is to make all mail dirs/files 0770/0660 mode and owned by group "sharedmail" and then set 'mail_access_groups=sharedmail'. Using more fine grained groups of course leaks less mail data in case there's a security hole in Dovecot. Permissions for new mailboxes ----------------------------- When creating a new mailbox, Dovecot copies the permissions from the mailbox root directory. For example with mboxes if you have directories: ---%<------------------------------------------------------------------------- drwx--xr-x 8 user group 4096 2009-02-21 18:31 /home/user/mail/ drwxrwxrwx 2 user group 4096 2009-02-21 18:32 /home/user/mail/foo/ ---%<------------------------------------------------------------------------- When creating a new foo/bar/ directory, Dovecot gives it permissions: ---%<------------------------------------------------------------------------- drwx--xr-x 2 user group 4096 2009-02-21 18:33 /home/usermail/foo/bar/ ---%<------------------------------------------------------------------------- As you can see, the file mode was copied from mail/ directory, not mail/foo/. The group is also preserved. If this causes problems (e.g. different users having different groups create mailboxes, causing permission denied errors when trying to preserve the group) you can set the setgid bit for the root directory: ---%<------------------------------------------------------------------------- chmod g+s /home/user/mail ---%<------------------------------------------------------------------------- This will cause the group to be automatically copied by the OS for all created files/directories under it, even if the user doesn't belong to the group. Permissions for new files in mailboxes -------------------------------------- When creating new files inside a mailbox, Dovecot copies the read/write permissions from the mailbox's directory. For example if you have: ---%<------------------------------------------------------------------------- drwx--xr-x 5 user group 4096 2009-02-21 18:53 /home/user/Maildir/.foo/ ---%<------------------------------------------------------------------------- Dovecot creates files under it with modes: ---%<------------------------------------------------------------------------- drwx--xr-x 2 user group 4096 2009-02-21 18:54 cur/ drwx--xr-x 2 user group 4096 2009-02-21 18:54 new/ drwx--xr-x 2 user group 4096 2009-02-21 18:54 tmp/ -rw----r-- 1 user group 156 2009-02-21 18:54 dovecot.index.log -rw----r-- 1 user group 17 2009-02-21 18:54 dovecot-uidlist ---%<------------------------------------------------------------------------- Note how the g+x gets copied to directories, but for files it's simply ignored. The group is copied the same way as explained in the previous section. When mails are copied between Maildirs, it's usually done by hard linking. If the source and destination directory permissions are different, Dovecot create a new file and copies data the slow way so that it can assign the wanted destination permissions. The source and destination permission lookups are done only by looking at the mailbox root directories' permissions, not individual mail files. This may become a problem if the mail files' permissions aren't as Dovecot expects. Mail Delivery Agent permissions ------------------------------- When using Dovecot , it uses all the same configuration files as IMAP/POP3, so you don't need to worry about it. When using an external MDA to deliver to a shared mailbox, you need to make sure that the resulting files have proper permissions. For example with Procmail + Maildir, set 'UMASK=007' in '.procmailrc' to make the delivered mail files group-readable. To get the file to use the proper group, set the group to the Maildir's 'tmp/' directory and also set its setgid bit ('chmod g+s'). Dictionary files ---------------- Created dictionary files (e.g. 'acl_shared_dict = file:...') also base their initial permissions on parent directory's permissions. After the initial creation, the permissions are permanently preserved. So if you want to use different permissions, just chown/chmod the file. (This file was created from the wiki on 2011-01-13 04:52) wiki/Quota.txt000066600000006120150501203730007340 0ustar00Quota ===== Quota backend specifies the method how Dovecot keeps track of the current quota usage. They don't (usually) specify users' quota limits, that's done by returning extra fields from userdb. There are different quota backends that Dovecot can use: * [Quota.FS.txt]: Filesystem quota. * [Quota.Dirsize.txt]: The simplest and slowest quota backend, but it works quite well with mboxes. * [Quota.Dict.txt]: Store quota usage in a dictionary (e.g. SQL). * [Quota.Maildir.txt]: Store quota usage in Maildir++ maildirsize files. This is the most commonly used quota for virtual users. Enabling quota plugins ---------------------- There are two quota related plugins: * quota: Implements the actual quota handling and includes also all the quota backends. * imap_quota: For reporting quota information via IMAP. Enable them in configuration files, e.g.: conf.d/10-mail.conf: ---%<------------------------------------------------------------------------- # Space separated list of plugins to load for all services. Plugins specific to # IMAP, LDA, etc. are added to this list in their own .conf files. mail_plugins = $mail_plugins quota ---%<------------------------------------------------------------------------- conf.d/20-imap.conf: ---%<------------------------------------------------------------------------- protocol imap { # Space separated list of plugins to load (default is global mail_plugins). mail_plugins = $mail_plugins imap_quota } ---%<------------------------------------------------------------------------- Configuration ------------- See for backend-independent quota configuration. Quota recalculation ------------------- If your quotas are out of sync, you can use 'doveadm quota recalc' command to recalculate them. Quota and Trash mailbox ----------------------- Standard way to expunge messages with IMAP works by: 1. Marking message with \Deleted flag 2. Actually expunging the message using EXPUNGE command Both of these commands can be successfully used while user's quota is full. However many clients use a "move-to-Trash" feature, which works by: 1. COPY the message to Trash mailbox 2. Mark the message with \Deleted 3. Expunge the message from the original mailbox. 4. (Maybe later expunge the message from Trash when "clean trash" feature is used) If user is over quota (or just under it), the first COPY command will fail and user may get an unintuitive message about not being able to delete messages because user is over quota. The possible solutions for this are: * Disable move-to-trash feature from client * You can create a separate quota rule ignoring Trash mailbox's quota. Note that this would allow users to store messages infinitely to the mailbox. * You can create a separate quota rule giving Trash mailbox somewhat higher quota limit (but not unlimited). To make sure users don't start keeping messages permanently in Trash you can use a nightly [Plugins.Expire.txt] to expunge old messages from Trash mailbox. (This file was created from the wiki on 2011-01-13 04:52) wiki/Design.Storage.MailboxList.txt000066600000017240150501203730013316 0ustar00Mailbox List ============ 'src/lib-storage/mailbox-list.h' and 'mailbox-list-private.h' describes mailbox list. The purpose of mailbox list is to manage mailbox storage name<-> physical directory path mapping. Its most important functions are: * listing existing mailboxes, * creating directories for new mailboxes (but not the mailboxes themselves, that's storage's job), * deleting mailboxes, * renaming mailboxes and * managing mailbox subscriptions. Mailbox list code also internally creates and updates mailbox changelog (in 'dovecot.mailbox.log' file), which keeps track of mailbox deletions, renames and subscription changes. This is primarily useful for dsync utility. Mailbox list is configured by [MailLocation.txt] setting, which fills 'struct mailbox_list_settings': * root_dir: The root mail directory (e.g. with 'mail_location=maildir:~/Maildir' it would be the '~/Maildir'). * index_dir: Directory under which index files are written to. Empty string means in-memory indexes. Defaults to root_dir. * control_dir: Directory under which control files are written to. Control files are files that contain some important metadata information about mailbox so (unlike index files) they should never be deleted. For example subscriptions file is a control file. Defaults to root_dir. * alt_dir: This is currently [MailboxFormat.dbox.txt]-specific setting. * inbox_path: Path to INBOX mailbox. This exists mainly because with mbox format INBOX is often in a different location than other mailboxes. * subscription_fname: Filename used by subscriptions file. * dir_guid_fname: Filename used to store directories' (not mailboxes') global UIDs. Directory GUIDs are mainly useful for dsync. * maildir_name: Directory name under which the actual mailboxes are stored in, such as dbox-Mails/ with dbox. See the .h file for more detailed description. * mailbox_dir_name: If non-empty, store all mailboxes under root_dir/mailbox_dir_name/. Listing mailboxes ----------------- First the list operation is initialized with one of the init functions: * 'mailbox_list_iter_init()' lists mailboxes that match the given pattern. * 'mailbox_list_iter_init_multiple()' lists mailboxes that match any of the given patterns list. * 'mailbox_list_iter_init_namespaces()' lists matching mailboxes from all namespaces. * 'MAILBOX_LIST_ITER_SKIP_ALIASES' flag skips namespaces that have 'alias_for' set. You usually want to set this flag to avoid processing the same mailbox multiple times. The patterns are IMAP-style patterns with '%' and '*' wildcards as described by RFC 3501: '%' matches only up to next hierarchy separator, while '*' matches the rest of the string. These flags control what mailboxes are returned: * 'MAILBOX_LIST_ITER_NO_AUTO_INBOX' doesn't list INBOX unless it physically exists. Normally INBOX is listed, because INBOX doesn't need to be (and cannot be) explicitly created. It can always be opened and messages can be saved to it, it's just automatically created when it doesn't exist. * 'MAILBOX_LIST_ITER_SELECT_SUBSCRIBED' lists only subscribed mailboxes. * 'MAILBOX_LIST_ITER_SELECT_RECURSIVEMATCH' is currently only useful when combined with '_SELECT_SUBSCRIBED' flag. Then it adds 'MAILBOX_CHILD_SUBSCRIBED' flags for mailboxes whose children are subscribed. It also lists mailboxes that aren't themselves subscribed, but have children that do. These flags control what is returned for matching mailboxes: * 'MAILBOX_LIST_ITER_RETURN_NO_FLAGS' can be set when you don't care about mailbox flags. They're then set only if it can be done without any additional disk I/O. * 'MAILBOX_LIST_ITER_RETURN_SUBSCRIBED' returns mailbox's subscription state. * 'MAILBOX_LIST_ITER_RETURN_CHILDREN' sets "has child mailboxes" or "doesn't have child mailboxes" flag. Other flags: * 'MAILBOX_LIST_ITER_RAW_LIST' should usually be avoided. It ignores ACLs and just returns everything. * 'MAILBOX_LIST_ITER_VIRTUAL_NAMES' enables listing to use virtual names instead of storage names in patterns and returned mailbox names. Once listing is initialized, 'mailbox_list_iter_next()' can be called until it returns NULL. The returned mailbox_info struct contains: * 'name': Mailbox's name, either virtual or storage name depending on '_VIRTUAL_NAMES' flag. * 'ns': Mailbox's namespace. This is useful only when mailboxes are listed using 'mailbox_list_iter_init_namespaces()'. * 'flags': Mailbox flags: * 'MAILBOX_NOSELECT': Mailbox exists, but can't be selected. It's possible that it can be created and then it becomes selectable. For example with mbox and FS layout the directories aren't selectable mailboxes. * 'MAILBOX_NONEXISTENT': Mailbox doesn't exist. It's listed only because it has child mailboxes that do exist but don't match the pattern. * Example: "foo/bar" exists, but "foo" doesn't. "%", "foo" or "*o" pattern would list "foo", because it matches the pattern but its child doesn't. Then again "*", "*bar" or "%/%" wouldn't list "foo", because "foo/bar" matches the pattern (and is also listed). Something like "*asd*" wouldn't match either "foo" or "foo/bar" so neither is returned. * 'MAILBOX_CHILDREN' and 'MAILBOX_NOCHILDREN': Mailbox has or doesn't have children. If neither of these flags are set, it's not known if mailbox has children. * 'MAILBOX_NOINFERIORS': Mailbox doesn't have children and none can ever be created. For example with mbox and FS layout the mailboxes have this flag set, because files can't be created under files. * 'MAILBOX_MARKED' and 'MAILBOX_UNMARKED': Mailbox has or doesn't have messages with \Recent flags. If neither is set, the state is unknown. Because this check is done in a very cheap way, having 'MAILBOX_MARKED' doesn't always mean that there are \Recent flags. However, if 'MAILBOX_UNMARKED' is returned it is guaranteed to be correct. (False positives are ok, false negatives are not ok.) * 'MAILBOX_SUBSCRIBED': Mailbox is subscribed. * 'MAILBOX_CHILD_SUBSCRIBED': Mailbox has a child that is subscribed (and '_SELECT_RECURSIVEMATCH' flag was set). Finally the listing is deinitalized with 'mailbox_list_iter_deinit()'. If it returns -1, it means that some mailboxes perhaps weren't listed due to some internal error. If you wish to get mailbox_info flags only for a single mailbox, you can use 'mailbox_list_mailbox()'. Directory permissions --------------------- 'mailbox_list_get_permissions()' and 'mailbox_list_get_dir_permissions()' can be used to get wanted permissions for newly created files and directories. * For global files, give NULL as the mailbox name. The permissions are then based on the root_dir. If root_dir doesn't exist, it returns 0700/0600 mode. * For per-mailbox files, give the mailbox name. The permissions are then based on the mailbox's directory. The returned permissions are: * mode: Creation mode, like 0600. * gid: Group that should be set, unless it's '(gid_t)-1'. There are 3 reasons why it could be that: * directory has g+s bit set, so the wanted group is set automatically * group is the same as process's effective GID, so it gets set automatically * mode's group permissions are the same as world permissions, so group doesn't matter. * gid_origin: This string points to the directory where the group (and permissions in general) was based on, or "defaults" for internal defaults. If changing the group fails with EPERM, 'eperm_error_get_chgrp()' can be used to log a nice and understandable error message. (This file was created from the wiki on 2011-01-13 04:52) wiki/LDA.Postfix.txt000066600000022702150501203730010306 0ustar00Dovecot LDA with Postfix ======================== This page contains only information specific to using LDA with Postfix, see for more information about using the LDA itself. System users ------------ If you wish you use 'dovecot-lda' for all system users on a single domain mail host you can do it by editing 'mailbox_command' parameter in '/etc/postfix/main.cf' (postconf(5) [http://www.postfix.org/postconf.5.html]): ---%<------------------------------------------------------------------------- mailbox_command = /usr/local/libexec/dovecot/dovecot-lda # or mailbox_command = /usr/libexec/dovecot/dovecot-lda # or mailbox_command = /usr/lib/dovecot/dovecot-lda # or wherever it was installed in your system. ---%<------------------------------------------------------------------------- Then run 'postfix reload'. * Be sure that /var/mail or wherever you deliver mail is writable by the dovecot-lda process, which does not run as root. * Postfix's 'mailbox_size_limit' setting applies to all files that are written via dovecot-lda. The default is 50 MB, so dovecot-lda can't write *any* files larger than that, including mbox files or log files. This shows up only in Dovecot's logs: ---%<---------------------------------------------------------------------- dovecot-lda(user): write() failed with mbox file /home/user/mail/foo: File too large (process was started with ulimit -f limit) ---%<---------------------------------------------------------------------- * If you have trouble seeing anything in Dovecot's logs, see [LDA.txt]. Some detailed config files and examples at http://heinous.org/wiki/Virtual_Domains%2C_Postfix%2C_Dovecot_LDA%2C_and_LDAP [http://heinous.org/wiki/Virtual_Domains,_Postfix,_Dovecot_LDA,_and_LDAP] Virtual users ------------- Dovecot LDA is very easy to use on large scale installations with Postfix virtual domains support, just add a 'dovecot' service in '/etc/postfix/master.cf' (master(5) [http://www.postfix.org/master.5.html]): ---%<------------------------------------------------------------------------- dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/local/libexec/dovecot/dovecot-lda -f ${sender} -d ${recipient} ---%<------------------------------------------------------------------------- An example using address extensions (ie user+extension@domain.com (don't forget to define the proper recipient_delimiter in Postfix's main.cf)) to deliver to the folder 'extension' in your maildir (If you wish to preserve the case of ${extension}, remove the 'hu'flags [http://www.postfix.org/pipe.8.html], and be sure to utilize [Variables.txt] in your dovecot.conf for mail locations and other configuration parameters that are expecting lower case): ---%<------------------------------------------------------------------------- dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/local/libexec/dovecot/dovecot-lda -f ${sender} -d ${user}@${nexthop} -m ${extension} # or if you have a INBOX/ namespace prefix: dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/local/libexec/dovecot/dovecot-lda -f ${sender} -d ${user}@${nexthop} -m INBOX/${extension} ---%<------------------------------------------------------------------------- This example ignores address extensions (ie user+extension@domain.com delivers just like user@domain.com ), but still shows the original address for Sieve: ---%<------------------------------------------------------------------------- dovecot unix - n n - - pipe flags=DRhu user=vmail:vmail argv=/usr/lib/dovecot/dovecot-lda -f ${sender} -a ${recipient} -d ${user}@${nexthop} ---%<------------------------------------------------------------------------- Replace 'vmail' above with your virtual mail user account. Then set 'virtual_transport' to 'dovecot' in '/etc/postfix/main.cf': ---%<------------------------------------------------------------------------- dovecot_destination_recipient_limit = 1 virtual_mailbox_domains = your.domain.here virtual_transport = dovecot ---%<------------------------------------------------------------------------- And remember to run ---%<------------------------------------------------------------------------- postfix reload ---%<------------------------------------------------------------------------- Virtual users with multiple uids/gids ------------------------------------- If you need multiple uids/gids you'll need to set dovecot-lda setuid root or invoke it through sudo. See [LDA.txt] for how to do this securely. Postfix with a NFS mail store ----------------------------- If you are experiencing problems with dovecot-lda processes hanging when delivering to an NFS mail store, it's likely that the dovecot-lda process is hanging while waiting for free locks. The occurrence of this can be greatly reduced, if not eradicated, by forcing Postfix to only deliver to the same recipient one at a time. ---%<------------------------------------------------------------------------- dovecot_destination_concurrency_limit = 1 ---%<------------------------------------------------------------------------- Prevent backscatter ------------------- To prevent backscatter you should configure Postfix to reject mail for non existent recipients. This is the default behaviour (smtpd_reject_unlisted_recipient = yes) so there's no need to set "reject_unlisted_recipient" in any of your restriction. But: Postfix must know if a recipient exists. Depending on how you've configured Dovecot and Postfix this can be done several ways. System users ------------ If you only use local system users this is no problem - all valid recipients can be found in the local password or alias database. Virtual users (static) ---------------------- When you use virtual users and domains you should maintain a list of valid recipients. The relevant settings settings are: *virtual_alias_maps, virtual_mailbox_maps* For static verification you can maintain the content of the files yourself. For every recipient or alias you need one entry. Example: *virtual_alias_maps* ---%<------------------------------------------------------------------------- name_recipient@example.com external@example.net ---%<------------------------------------------------------------------------- *virtual_mailbox_maps* ---%<------------------------------------------------------------------------- name@example.com OK recipient@example.com available ---%<------------------------------------------------------------------------- Don't forget to run "postmap" afterwards. *Info:* if you use the Dovecot LDA or LMTP it doesn't matter what you use behind the recipient address. Use "OK", the full name of the user or else. Virtual users (dynamic) ----------------------- Do you already use a database (MySQL, PostgreSQL) for Dovecot? Use the same source for Postfix. You only have to to define a valid sql query for Postfix. Example: ---%<------------------------------------------------------------------------- virtual_mailbox_maps = proxy:mysql:/etc/postfix/virtual_mailbox_maps.cf ---%<------------------------------------------------------------------------- *virtual_mailbox_maps.cf* ---%<------------------------------------------------------------------------- user = mysql-user password = mysql-password hosts = unix:/var/run/mysql/mysqld.sock dbname = mailserver query = SELECT name FROM mailbox WHERE email='%s' ---%<------------------------------------------------------------------------- This query will return the value of the filed "name" from table "mailbox" if the email address of the recipient matches the email from the field "email". This is enough for Postfix because Postfix must only know if the recipient exists. The value doesn't matter. When you use a database (or LDAP) there's no need to manually maintain a file with valid recipients. *Info:* If you use "relay_domains" instead of "virtual_mailbox_domains" you have to use "relay_recipient_maps" instead of "virtual_mailbox_maps". Dynamic address verfication with LMTP ------------------------------------- With Dovecot 2.0 you can also use LMTP and the Postfix setting "reject_unverified_recipient" for dynamic address verification. It's really nice because Postfix doesn't need to query an external datasource (MySQL, LDAP...). Postfix maintain a local database with existing/non existing addresses (you can configure how long positive/negative results should be cached). To use LMTP and dynamic address verification you must first get Dovecot working. Then you can configure Postfix to use LMTP and set "reject_unverified_recipient" in the smtpd_recipient_restrictions. On every incoming email Postfix will probe if the recipient address exists. You will see similar entries in your logfile: ---%<------------------------------------------------------------------------- Recipient address rejected: undeliverable address: host tux.example.com[private/dovecot-lmtp] said: 550 5.1.1 < tzknvtr@example.com > User doesn't exist: tzknvtr@example.com (in reply to RCPT TO command); from=< cnrilrgfclra@spammer.org > to=< tzknvtr@example.com > ---%<------------------------------------------------------------------------- If the recipient address exists (status=deliverable) Postfix accepts the mail. *Info:* you can not use "reject_unverified_recipient" with "pipe" so this doesn't work with the Dovecot LDA "deliver". (This file was created from the wiki on 2011-01-13 04:52) wiki/Design.Indexes.Cache.txt000066600000014156150501203730012070 0ustar00Cache file ========== Cache file is used for storing immutable data. It supports several different kinds of fields: MAIL_CACHE_FIELD_FIXED_SIZE: The field size doesn't need to be stored in the cache file. It's always the same. MAIL_CACHE_FIELD_BITMASK: A fixed size bitmask field. It's possible to add new bits by updating this field. All the added fields are ORed together. MAIL_CACHE_FIELD_VARIABLE_SIZE: Variable sized binary data. MAIL_CACHE_FIELD_STRING: Variable sized string. MAIL_CACHE_FIELD_HEADER: Variable sized message header. The data begins with a 0-terminated 'uint32_t line_numbers[]'. The line number exists only for each header, header continuation lines in multiline headers don't get listed. After the line numbers comes the list of headers, including the "header-name: " prefix for each line, LFs and the TABs or spaces for continued lines. The last 3 variable sized fields are treated identically by the cache file code. Their main purpose is to make it easier for "dump cache file's contents" programs ('src/util/idxview') to do their job. Locking ------- Because cache file is typically used in potentially long-running operations, such as with IMAP command 'FETCH 1:* (BODY.PEEK[] ENVELOPE BODYSTRUCTURE)' it's important that updating the cache file doesn't block out any other readers. Also because the readers are often also writers (if something isn't cached, it's added there), it's important that they don't block writers either. Reading cache files requires no locking. Writing is done by first locking the file, reserving some space to write to, and immediately after that unlocking the file. This way the transaction can keep writing to the cache file as long as it wants to without blocking other writers. When the transaction is committed, the updated cache offsets are written to the transaction log which makes them visible to other processes. This also means that it's possible for two processes to write the same cached fields twice to the cache file. Because the data written to the cache file are really just cached data, the fields' contents are identical. Having the data exist twice (or even more times) means wasting some disk space, but otherwise it isn't a problem. The duplicates are dropped the next time the file is compressed. Cache decisions --------------- Dovecot tries to be smart about what it keeps in the cache file. If the client never fetches the cached data, it's just waste of disk space and disk I/O. The caching decisions are: MAIL_CACHE_DECISION_NO: This field isn't cached currently. MAIL_CACHE_DECISION_TEMP: This field is cached for new mails. MAIL_CACHE_DECISION_YES: This field is cached for all mails. Normally Dovecot changes the decisions based on what fields are fetched and for what messages. A specific decision can be forced by ORing it with 'MAIL_CACHE_DECISION_FORCED'. 'mail-cache-decisions.c' file contains the rules how Dovecot changes the decisions. The following is copied from the file: Users can be divided to three groups: 1. Most users will use only a single IMAP client which caches everything locally. For these users it's quite pointless to do any kind of caching as it only wastes disk space. That might also mean more disk I/O. 2. Some users use multiple IMAP clients which cache everything locally. These could benefit from caching until all clients have fetched the data. After that it's useless. 3. Some clients don't do permanent local caching at all. For example Pine and webmails. These clients would benefit from caching everything. Some locally caching clients might also access some data from server again, such as when searching messages. They could benefit from caching only these fields. After thinking about these a while, I figured out that people who care about performance most will be using Dovecot optimized LDA anyway which updates the indexes/cache immediately. In that case even the first user group would benefit from caching the same way as second group. LDA reads the mail anyway, so it might as well extract some information about it and store them into cache. So, group 1. and 2. could be optimally implemented by keeping things cached only for a while. I thought a week would be good. When cache file is compressed, everything older than week will be dropped. But how to figure out if user is in group 3? One quite easy rule would be to see if client is accessing messages older than a week. But with only that rule we might have already dropped useful cached data. It's not very nice if we have to read and cache it twice. Most locally caching clients always fetch new messages (all but body) when they see them. They fetch them in ascending order. Noncaching clients might fetch messages in pretty much any order, as they usually don't fetch everything they can, only what's visible in screen. Some will use server side sorting/threading which also makes messages to be fetched in random order. Second rule would then be that if a session doesn't fetch messages in ascending order, the fetched field type will be permanently cached. So, we have three caching decisions: 1. Don't cache: Clients have never wanted the field 2. Cache temporarily: Clients want this only once 3. Cache permanently: Clients want this more than once Different mailboxes have different decisions. Different fields have different decisions. There are some problems, such as if a client accesses message older than a week, we can't know if user just started using a new client which is just filling its local cache for the first time. Or it might be a client user hasn't just used for over a week. In these cases we shouldn't have marked the field to be permanently cached. User might also switch clients from non-caching to caching. So we should re-evaluate our caching decisions from time to time. This is done by checking the above rules constantly and marking when was the last time the decision was right. If decision hasn't matched for two months, it's changed. I picked two months because people go to at least one month vacations where they might still be reading mails, but with different clients. (This file was created from the wiki on 2011-01-13 04:52) wiki/Design.Dsync.txt000066600000011550150501203730010542 0ustar00Dsync Design ============ Two-way synchronization ----------------------- dsync attempts to preserve all changes done by both sides of the synced mailboxes. Mailbox list ------------ Mailboxes have 128 bit globally unique IDs, which are used for figuring out when two mailboxes should actually be synchronized. This solves two major problems: * If mailbox has been renamed in one side, dsync finds it because its GUID hasn't changed. * If mailbox has been deleted and recreated, dsync doesn't attempt to sync it because it's a different mailbox. Then there's the problem of how to correctly sync mailbox renames and deletions. How do you know which side of the sync has the most recent name for the mailbox? How do you know if one side had deleted mailbox, or if the other side had created it? To solve these problems, Dovecot v2.0 created a "mailbox log", which adds a record with mailbox GUID and timestamp whenever mailbox is renamed or deleted. So: * If mailbox has different names on two sides, its "last renamed" timestamp is looked up from the mailbox list index. The side with the most recent timestamp is assumed to contain the newer name and the other side's mailbox is renamed to it. * If neither side has a "last renamed" timestamp, one side is picked. This shouldn't happen, except when mailbox log is deleted for some reason or if the renaming is done outside Dovecot. * If mailbox exists only on one side, the other side checks if mailbox log contains a delete record for its GUID. If there is one, the mailbox is deleted from the other side. If there's not, the mailbox is created and synced. * Subscriptions and unsubscriptions are synced in a similar way. But because it's possible to be subscribed to nonexistent mailboxes, mailbox log can't contain mailbox GUIDs for them. Instead the first 128 bits of SHA1 of mailbox name are used. Collisions for mailbox names are highly unlikely, but even if one happens, the worst that can happen is that user gets unsubscribed from wrong mailbox. dsync writes timestamps to changelog using the original timestamps, so that dsync's changes won't override changes done by user during sync. Mailbox ------- When saving new mails, dsync preserves all of their immutable state: * GUID * Received date * Save date * Message contents It also attempts preserve IMAP UID. This works as long as the other side hasn't already used the UID for another mail. If it has, dsync doesn't attempt to preserve the UID, because an IMAP client might have already seen the UID and cached another mail's contents for it. IMAP requires that message's contents must never change, so UIDs can't be reused. So whenever an UID conflict happens, dsync gives messages in both sides a new UID, because it can't know which message the client had seen, or perhaps user used two clients and both saw a different message. (This assumes a master/slave replication use case for dsync.) The mutable metadata that dsync preserves is: * Message flags and keywords * Modification sequences (modseqs) Flags and keywords are synced based on modseqs. Whichever side has a higher modseq for the message, its flags and keywords are synced to the other side. Currently there's no per-flag or per-keyword synchronization, so that if one side had added \Seen flag and other side had added \Answered flag, one of them would be dropped. Finding what to sync -------------------- dsync can run in full mode or fast mode. Full mode means it goes through all messages in all mailboxes, making sure everything is fully synchronized. In fast mode it relies on uidvalidity, uid-next and highest-modseq values to find out changes. If any of the values changed, the mailbox is included in sync. FIXME: A superfast mode should still be implemented, where once a mailbox is selected for syncing, it should sync only mails whose modseq is higher than a given one. This would improve performance and network traffic with large mailboxes. Copy optimizations ------------------ Before dsync actually starts syncing anything, it first fetched a list of all to-be-synced messages and adds them to a GUID -> message hash table. Whenever dsync needs to sync a new message to the other side, it first checks if the message's GUID already exists on the other side. If it does, it starts a message copy operation instead of a full save. It's possible that this copy operation fails if the message just gets expunged from the other side, so there needs to be fallback handling for this. If the message exists in multiple mailboxes, a copy from the next mailbox is attempted. If all of them fail, dsync fallbacks to saving the message. FIXME: This optimization currently works only in full sync mode. If this were to work in fast sync mode, the full mailbox list would have to be looked up from local side. And this would slow it down.. (This file was created from the wiki on 2011-01-13 04:52) wiki/Authentication.Kerberos.txt000066600000012654150501203730013012 0ustar00Kerberos ======== Dovecot supports Kerberos 5 using GSSAPI. The Kerberos authentication mechanism doesn't require having a [PasswordDatabase.txt], but you do need a [UserDatabase.txt] so Dovecot can lookup user-specific information, such as where their mailboxes are stored. *Note:* If you only wish to authenticate clients using their Kerberos /passphrase/ (as opposed to ticket authentication), you will probably want to use [PasswordDatabase.PAM.txt] authentication with 'pam_krb5.so' instead. Pre-requisites -------------- This document assumes that you already have a Kerberos Realm up and functioning correctly at your site, and that each host in your realm also has a host /keytab/ installed in the appropriate location. For Dovecot, you will need to install the appropriate /service/ keys on your server. By default, Dovecot will look for these in the host's keytab file, typically '/etc/krb5.keytab', but you can specify an alternate path using the 'auth_krb5_keytab' configuration entry in dovecot.conf. If you wish to provide an IMAP service, you will need to install a service ticket of the form 'imap/hostname@REALM'. For POP3, you will need a service ticket of the form 'pop/hostname@REALM'. When using Dovecot's [Sasl.txt] with MTA, you will need to install service ticket of the form 'smtp/hostname@REALM'. Example dovecot.conf configurations ----------------------------------- If you only want to use Kerberos ticket-based authentication: ---%<------------------------------------------------------------------------- auth_mechanisms = gssapi userdb { driver = static args = uid=vmail gid=vmail home=/var/vmail/%u } ---%<------------------------------------------------------------------------- (In this virtual-hosting example, all mail is stored in /var/vmail/$username with uid and gid set to 'vmail') If you also want to support plaintext authentication in addition to ticket-based authentication, you will need something like: ---%<------------------------------------------------------------------------- auth_mechanisms = plain gssapi passdb { driver = pam } userdb { driver = passwd } ---%<------------------------------------------------------------------------- (Note that in this example, you will also need to configure PAM to use whichever authentication backends are appropriate for your site.) Enable plaintext authentication to use Kerberos ----------------------------------------------- This is needed when some of your clients don't support GSSAPI and you still want them to authenticate against Kerberos. Install pam_krb5 module for PAM, and create '/etc/pam.d/dovecot': ---%<------------------------------------------------------------------------- auth sufficient pam_krb5.so account sufficient pam_krb5.so ---%<------------------------------------------------------------------------- Then enable PAM passdb: ---%<------------------------------------------------------------------------- passdb { driver = pam } ---%<------------------------------------------------------------------------- Check '/var/log/auth.log' if you have any problems logging in. The problem could be that PAM is still trying to use pam_unix.so rather than pam_krb5.so. Make sure pam_krb5.so is the first module for account or just change pam_unix.so to sufficient. Client support -------------- Mail clients that support Kerberos GSSAPI authentication include: * Evolution * Mozilla Thunderbird * SeaMonkey * Mutt * UW Pine * Apple Mail Testing ------- *FIXME*: This section requires cleanup. Test that the server can access the keytab ------------------------------------------ This test demonstrates that the server can acquire its private credentials. First telnet directly to the server ---%<------------------------------------------------------------------------- $ telnet localhost 143 * OK Dovecot ready. ---%<------------------------------------------------------------------------- or, if you are using IMAPS then use openssl instead of telnet to connect: ---%<------------------------------------------------------------------------- $ openssl s_client -connect localhost:993 CONNECTED(00000003) ... * OK Dovecot ready. ---%<------------------------------------------------------------------------- Check that GSSAPI appears in the authentication capabilities: ---%<------------------------------------------------------------------------- a capability * CAPABILITY ... AUTH=GSSAPI ---%<------------------------------------------------------------------------- Attempt the first round of GSS communication. The '+' indicates that the server is ready ---%<------------------------------------------------------------------------- a authenticate GSSAPI + ---%<------------------------------------------------------------------------- Abort the telnet session by typing control-] and then 'close' ---%<------------------------------------------------------------------------- ^] telnet> close ---%<------------------------------------------------------------------------- The test: * Setup mutt in /etc/Muttrc to use kerberos using gssapi and imap configuration * this is done with 'set imap_authenticators="gssapi"' * run kinit (type in password for kerb) * run command mutt * If you get error No Authentication Method * run command klist (list all kerberos keys) should show imap/HOSTNAME * /etc/hosts has to be set properly so that kerberos can find server. (This file was created from the wiki on 2011-01-13 04:52) wiki/Variables.txt000066600000020757150501203730010173 0ustar00Variables ========= You can use special variables in several places: * [MailLocation.txt] setting and [Namespaces.txt] locations * [UserDatabase.Static.txt] and [AuthDatabase.PasswdFile.txt] template strings * [AuthDatabase.LDAP.txt] and [AuthDatabase.SQL.txt] userdb query strings * log prefix for imap/pop3 process * [Plugins.txt] settings The variables that work everywhere are: * +------------+----------+-----------------------------------------------------+ | *Variable* | *Long | *Description* | | | name* | | +------------+----------+-----------------------------------------------------+ | %% | | '%' character. See for | | | | further information about %% variables | +------------+----------+-----------------------------------------------------+ | %u | user | full username (e.g. user@domain) | +------------+----------+-----------------------------------------------------+ | %n | username | user part in user@domain, same as %u if there's no | | | | domain | +------------+----------+-----------------------------------------------------+ | %d | domain | domain part in user@domain, empty if user there's no| | | | domain | +------------+----------+-----------------------------------------------------+ | %s | service | imap, pop3, smtp, lda (and doveadm, dsync, etc.) | +------------+----------+-----------------------------------------------------+ | %p | pid | PID of the current process (login or imap/pop3 | | | | process) | +------------+----------+-----------------------------------------------------+ | %l | lip | local IP address | +------------+----------+-----------------------------------------------------+ | %r | rip | remote IP address | +------------+----------+-----------------------------------------------------+ These variables work almost everywhere else except in Dovecot-auth (userdb queries/templates): * +------------+----------+-----------------------------------------------------+ | *Variable* | *Long | *Description* | | | name* | | +------------+----------+-----------------------------------------------------+ | %h | home | home directory. Use of ~/ is better whenever | | | | possible. | +------------+----------+-----------------------------------------------------+ | %i | uid | UNIX UID of the user | +------------+----------+-----------------------------------------------------+ These variables work only in Dovecot-auth: * +------------+----------+-----------------------------------------------------+ | *Variable* | *Long | *Description* | | | name* | | +------------+----------+-----------------------------------------------------+ | %w | password | plaintext password from plaintext authentication | | | | mechanism | +------------+----------+-----------------------------------------------------+ | %m | mech | | | | | [Authentication.Mechanisms.txt], e.g. PLAIN | +------------+----------+-----------------------------------------------------+ | %a | lport | Local port | +------------+----------+-----------------------------------------------------+ | %b | rport | Remote port | +------------+----------+-----------------------------------------------------+ | %c | secured | "secured" string with SSL, TLS and localhost | | | | connections. Otherwise empty. | +------------+----------+-----------------------------------------------------+ | %k | cert | "valid" if client had sent a valid client | | | | certificate, otherwise empty. | +------------+----------+-----------------------------------------------------+ These variables work only in 'login_log_format_elements' setting: * +------------+--------------+-------------------------------------------------+ | *Variable* | *Long name* | *Description* | +------------+--------------+-------------------------------------------------+ | %m | mech | | | | | [Authentication.Mechanisms.txt], e.g. PLAIN | +------------+--------------+-------------------------------------------------+ | %a | lport | Local port | +------------+--------------+-------------------------------------------------+ | %b | rport | Remote port | +------------+--------------+-------------------------------------------------+ | %c | secured | SSL, TLS or empty | +------------+--------------+-------------------------------------------------+ | %k | ssl_security | SSL protocol and cipher information, e.g. "TLSv1| | | | with cipher DHE-RSA-AES256-SHA (256/256 bits)" | +------------+--------------+-------------------------------------------------+ | %e | mail_pid | Mail process (imap/pop3) PID that handles the | | | | post-login connection | +------------+--------------+-------------------------------------------------+ Long variable names can be used like '%{long_name} ' or with L modifier: '%L{long_name}'. Environment variables can be accessed with '%{env:ENVIRONMENT_VARIABLE}'. Additionally, the (self-explanatory) variables '%{pid}' and '%{hostname} ' are available. Modifiers --------- You can apply a modifiers for each variable (e.g. %Us = POP3): * %L - lowercase * %U - uppercase * %E - escape '"', "'" and '\' characters by inserting '\' before them. Note that variables in SQL queries are automatically escaped, you don't need to use this modifier for them. * %X - parse the variable as a base-10 number, and convert it to base-16 (hexadecimal) * %R - reverse the string * %H - take a 32bit hash of the variable and return it as hex. You can also limit the hash value. For example %256Hu gives values 0..ff. You might want padding also, so %2.256Hu gives 00..ff. This can be useful for example in dividing users automatically to multiple partitions. * %H hash function is a bit bad if all the strings end with the same text, so if you're hashing usernames being in user@domain form, you probably want to reverse the username to get better hash value variety, e.g. %3RHu. * %M - return the string's MD5 sum as hex. * %D - return "sub.domain.org" as "sub,dc=domain,dc=org" (for LDAP queries) * %T - Trim trailing whitespace You can take a substring of the variable by giving optional offset followed by '.' and width after the '%' character. For example %2u gives first two characters of the username. %2.1u gives third character of the username. If the offset is negative, it counts from the end, for example %-2.2i gives the UID mod 100 (last two characters of the UID printed in a string). If a positive offset points outside the value, empty string is returned, if a negative offset does then the string is taken from the start. If the width is prefixed with zero, the string isn't truncated, but only padded with '0' character if the string is shorter. For example %04i may return "0001", "1000" and "12345". %1.04i for the same string would return "001", "000" and "2345". The modifiers are applied from left-to-right order, except the substring is always taken from the final string. (This file was created from the wiki on 2011-01-13 04:52) wiki/Migration.MailFormat.txt000066600000025562150501203730012245 0ustar00Converting between mailbox formats ================================== If you want a transparent migration, the biggest problem is preserving message UIDs. See for the problems this may cause. If you do the conversion with dsync, it preserves the UIDs. dsync ----- With dsync you can convert between any two mailbox formats that Dovecot supports. As much of the mailbox state is preserved as possible. Typically it's everything. See for full documentation, here are only a couple of examples: * mbox -> maildir migration. Set 'mail_location=maildir:~/Maildir' and run 'doveadm mirror -u username mbox:~/mail:INBOX=/var/mail/username' * maildir -> mdbox migration. Set 'mail_location=mdbox:~/mdbox' and run 'doveadm mirror -u username maildir:~/Maildir' If you can successfully use dsync, you can skip the rest of this page. Converting from mbox to Maildir ------------------------------- * mb2md.pl with Dovecot modifications [http://dovecot.org/tools/mb2md.pl] can convert mails, preserving UIDs and keywords. * See also [attachment:migrateuser.sh script to drive the full migration for a user]. * This script requires patched 'mailutil' that supports Maildir. One working 'mailutil' binary is RHEL4 PINE RPM [http://dag.wieers.com/rpm/packages/pine/pine-4.64-3.el4.rf.i386.rpm] from the DAG RPM Repository [http://dag.wieers.com/rpm/]. You could also patch any version of pine/c-client and patch it with the Maildir patches from http://staff.washington.edu/chappa/pine/info/maildir.html. * mb2md.py [http://dovecot.org/list/dovecot/2008-March/029736.html] can convert also message UIDs. * Yet another way to fix the UIDL migration problem. If you can generate an uildlist ("messagenumber uidl" pairs), use my new -U uidllist option to inject X-UILD: headers in the converted Maildir-file. The modified mb2md script is avalable here:mb2md.xuidl.pl.gz [http://www.chaos.dk/~sch/mb2md.xuidl/mb2md.xuidl.pl.gz]. I used this to convert a cucipop installation to dovecot. pop3_reuse_xuidl=yes will do the rest./-- 2009-02-13/ Check also the *User-Contributed Maildir Support* section on the qmail community site [http://www.qmail.org/top.html#usersoft] for more choices. Example (user's mail in '~someuser/mail' and INBOX in '/var/mail/someuser'): ---%<------------------------------------------------------------------------- cd ~someuser mb2md-3.20.pl -s mail -R mb2md-3.20.pl -m -s /var/mail/someuser mv mail mail.old ---%<------------------------------------------------------------------------- Now the mail will be in '~someuser/Maildir'. Do not forget to migrate the subscriptions as well, otherwise the new maildir will seem to have only an inbox when viewed through a mail client that supports them. This can be as simple as copying the old '~someuser/mail/.subscriptions' file to '~someuser/Maildir/subscriptions' (warning: I have not tested this extensively, my subscription list and folder hierarchy was very simplistic). Hierarchy separator change -------------------------- The default hierarchy separator with Maildir is '.' instead of '/' which is common with mboxes. To keep the migration transparent to users, you can keep the '/' separator by using [Namespaces.txt]. In any case you need to replace the '/' with '.' in the subscriptions file: * ---%<---------------------------------------------------------------------- sed 's:/:.:g' subscriptions > subscriptions.new mv subscriptions.new subscriptions ---%<---------------------------------------------------------------------- UW-IMAP's subscriptions file is in '~/.mailboxlist'. Dovecot's mbox subscriptions is in '/.subscriptions'. Dovecot's Maildir subscriptions is in '/subscriptions'. Also if you're migrating from UW-IMAP, you probably had "mail/" prefixes in the mailbox names. You can again use [Namespaces.txt] to let clients use the prefix, or you can tell your users to remove the namespace prefix from their clients and change the subscriptions file: ---%<------------------------------------------------------------------------- sed 's/^mail\.//' subscriptions > subscriptions.new mv subscriptions.new subscriptions ---%<------------------------------------------------------------------------- Note that because Maildir uses '.' as the hierarchy separator in filesystem, it's not possible to have mailbox names containing '.' characters, even if you changed the separator in namespaces. If you really want to have dots, the only way to do this is by modifying the filesystem separator in 'MAILDIR_FS_SEP' and 'MAILDIR_FS_SEP_S' defines in 'src/lib-storage/index/maildir/maildir-storage.h' file in the sources. Do not be tempted to change 'MAILDIR_FS_SEP' et al to '/'; it won't work. Converting from Maildir to mbox ------------------------------- This is especially helpful if you want to archive your mail to a single file for storage on a CD, a PC, etc. But it can also be helpful if you want to use [MailboxFormat.mbox.txt] with Dovecot. Use the reformail program that comes with maildrop [http://www.courier-mta.org/maildrop/]. You can also use the formail program that comes with procmail [http://www.procmail.org/]. Here is a simple script showing how this works. To use it, adjust the script to invoke the right command according to your system. Then 'cd' to the user's home directory (one level above 'Maildir') and run the script with two arguments: the mailbox name (You can use "." for the top-level folder), and the output mbox filename, for example: ---%<------------------------------------------------------------------------- cd ~hans perl dw-maildirtombox.pl . >/tmp/hans-inbox perl dw-maildirtombox.pl Sent >/tmp/hans-sent ---%<------------------------------------------------------------------------- ---%<------------------------------------------------------------------------- #!/usr/bin/env perl # dw-maildirtombox.pl # dw = Dovecot Wiki :-) # NOTE! The output file must not contain single quotes (')! # figure out which program to run $cmd="reformail -f1"; system("$cmd /dev/null 2>/dev/null") == 0 or $cmd="formail"; system("$cmd /dev/null 2>/dev/null") == 0 or die "cannot find reformail or formail on your \$PATH!\nAborting"; $dir=$ARGV[0]; $outputfile=$ARGV[1]; if (($outputfile eq '') || ($dir eq '')) { die "Usage: ./archivemail.pl mailbox outputfile\nAborting"; } if (!stat("Maildir/$dir/cur") || !stat("Maildir/$dir/new")) { die "Maildir/$dir is not a maildir.\nAborting"; } @files = (,); foreach $file (@files) { next unless -f $file; # skip non-regular files next unless -s $file; # skip empty files next unless -r $file; # skip unreadable files $file =~ s/'/'"'"'/; # escape ' (single quote) $run = "cat '$file' | $cmd >>'$outputfile'"; system($run) == 0 or warn "cannot run \"$run\"."; } ---%<------------------------------------------------------------------------- Converting from MBX to Maildir ------------------------------ See the uw2dovecot.pl [http://wiki.dovecot.org/Migration/UW?action=AttachFile&do=view&target=uw2dovecot.pl] as mentioned on the page. Converting from MBX to mbox --------------------------- If you are using UW-IMAP and using the [MailboxFormat.mbx.txt], you will need to convert it to [MailboxFormat.mbox.txt] format. The conversion process isn't pretty, but here is a script that works. You will need to get and compile the mailutil program from the UW-IMAP web site [http://www.washington.edu/imap/]. ---%<------------------------------------------------------------------------- #! /bin/sh # Written by Marc Perkel - public domain # overhauled by Matthias Andree, 2006 # Usage: mbx-convert # This code assumes there a user named "marc" with the primary group "marc". # Change to any real user on your system. # Yes - it look bizzare - but it gets the job done # abort on error set -e user=marc group=marc homedir=/home/$user if [ $# -ne 1 ] ; then echo >&2 "Usage: $0 " exit 1 fi # set up automatic cleanup trap 'rm -f "${homedir}"/in.$$ "${homedir}"/out.$$' 0 # First copy to users home dir and make the user the owner cp "$1" "${homedir}/in.$$" chown "$user":"$group" "${homedir}/in.$$" # Run mailutil to convert as the user other than root # mailutil requires this su "$user" -c "mailutil copy in.$$ \#driver.unix/out.$$" # create new file with same permissions/owner as old cp -p "$1" "${1}.new" # cat instead of copy leaves the original owner and permissions alone if cat "${homedir}/out.$$" >"${1}.new" ; then # cat succeeded, rename file into place mv "${1}.new" "$1" else # cat failed, remove temp file rm -f "${1}.new" exit 1 fi ---%<------------------------------------------------------------------------- Make a copy of some folders and test it first. Once you are satisfied that it works then: * Write a script to convert all your files. * Shut down your email system so files can't be modified * Copy all your email to a backup location in case you have to revert * Run the script * Turn on Dovecot and test to verify it is working * Important - make sure that you changed your SMTP configuration to write mbox and not MBX. * Turn on SMTP and verify it is all working *User comments:* * Is this hassle actually necessary? I have run mailutil as root like this (mailutil as provided by the PINE 4.61 package for SUSE Linux 10.0):'mailutil copy /tmp/foo.mbx.orig '#driver.unix//tmp/foo.test'' and did not encounter any problems./-- , 2006-05-18/ * I did the same (using 'mailutil') but it doesn't maintain UIDs or UIDVALIDITY. So I hacked[attachment:mbx2mbox.tgz this] together to do the migration./-- , 2008-08-02/ * Tried to do 'mailutil -v copy /tmp/foo '#driver.unix'/tmp/foo.unix', but mailutil argues 'Can't open mailbox /tmp/foo: no such mailbox'. (mailutil as from the Debian package uw-mailutils in Debian 5.0 Lenny) strace shows that it searches for /tmp/foo/cur, i.e. a Maildir format mailbox. (WTF?) No idea yet how to get this working. And I'm really glad when I got rid of MBX. -- , 2009-06-19 * As Mark Crispin always says: Don't use UW-IMAP with buggy (Maildir) patches, compile your own. * Same problem here - just reinstalled uw-imapd, used thunderbird to create a "normal" mailfolder and copied the old folder contents - F.Fernandez * At least with ubuntu mailutil from uw-mailutils it starts search from root *ALWAYS* and if you start with slash it tries to convert from maildir format 'mailutil -v copy /tmp/foo '#driver.unix'/tmp/foo.unix' = convert maildir formated folder /tmp/foo/cur to mbox and 'mailutil -v copy tmp/foo '#driver.unix'/tmp/foo.unix' = convert /tmp/foo file to mbox - Manwe, 2010-03-09 (This file was created from the wiki on 2011-01-13 04:52) wiki/Dict.txt000066600000002661150501203730007140 0ustar00Dict Proxy Process ================== Dict can be used by: * [Quota.Dict.txt] * [SharedMailboxes.Shared.txt] * [Plugins.Expire.txt] When using these, the mail processes need to have access the dict socket. By default only "dovecot" user has access to dict socket, which doesn't typically work in any installations. However, giving too wide permissions by default might allow untrusted users to access the dict and cause problems. If all users share a single UNIX UID (e.g. "vmail"), you could make the dict socket accessible only to it: ---%<------------------------------------------------------------------------- service dict { unix_listener dict { mode = 0600 user = vmail } } ---%<------------------------------------------------------------------------- If you use multiple UNIX UIDs, you can add an extra group for all Dovecot mail processes. This works even if you have untrusted system users who have shell access to the server: ---%<------------------------------------------------------------------------- mail_access_groups = dovecot service dict { unix_listener dict { mode = 0660 group = dovecot } } ---%<------------------------------------------------------------------------- However, it works with only if it's started as root. If this isn't possible, look into using instead. (This file was created from the wiki on 2011-01-13 04:52) wiki/SystemUsers.txt000066600000011765150501203730010570 0ustar00System Users ============ System users are typically defined in '/etc/passwd' file, but this isn't necessary. Using NSS [http://en.wikipedia.org/wiki/Name_Service_Switch] you can configure the lookups to be done from for example LDAP. See [AuthDatabase.Passwd.txt] userdb configuration for how to set this up. Especially if you're using nss_ldap you must set 'blocking=yes'. System users usually have their own separate user IDs (UIDs). This is good from security point of view, because it means that the kernel will also prevent users from accessing each others' mails. If the users have direct write access to the mail files (eg. the users have shell access), they can easily cause all sorts of mailbox corruptions. That may generate all kinds of error messages to Dovecot's error logs, so it may be sometimes difficult to tell if there really is a problem or if user is just doing something stupid. If users are going to access the mailboxes with other software than Dovecot, it's important to make sure that their mailbox accesses are compatible. This mostly means that with mboxes you must make sure that everyone uses the [MailboxFormat.mbox.txt]. Authentication -------------- Admins often wish to use different passwords for IMAP and POP3 than for other services (eg. SSH), because IMAP and POP3 clients often send the password unencrypted over the internet without even bothering to give users any warnings. Dovecot can easily support non-system passwords for system users. If you wish to use system passwords, you'll want to use one of these passdbs: * [PasswordDatabase.PAM.txt]: Most commonly used in Linux and BSDs nowadays. * [PasswordDatabase.BSDAuth.txt]: BSD authentication is used by OpenBSD. * [AuthDatabase.Passwd.txt]: System users (NSS, '/etc/passwd', or similiar). This may work instead of PAM (mostly in some BSDs). * [PasswordDatabase.Shadow.txt]: Shadow passwords for system users (NSS,'/etc/shadow' or similiar). Deprecated by PAM nowadays, but it should work with Linux and Solaris. If you wish to use non-system passwords, you can use pretty much any of the Dovecot's [PasswordDatabase.txt], but for simple installations you'll probably want to use [AuthDatabase.PasswdFile.txt]. [UserDatabase.txt] for system users is always [AuthDatabase.Passwd.txt]. Mail Location ------------- Usually UNIX systems are configured by default to deliver mails to '/var/mail/username' or '/var/spool/mail/username' mboxes. You may decide to use these, or use [MailboxFormat.Maildir.txt] format instead. Dovecot detects the mailbox format and location automatically if 'mail_location' setting isn't set, but it's still a good idea to explicitly tell Dovecot where to find the mails. This makes it sure that Dovecot behaves correctly also when the user's mailbox doesn't exist at the moment (eg. a new user). If Dovecot can't figure out where the existing mails are, it simply gives an error message and quits. It never tries to create a missing mailbox when autodetection is used. See for more information how to configure the mailbox location. Below are the highlights for mbox and maildir. mbox ---- The '/var/mail/username' mbox is called user's INBOX. IMAP protocol supports multiple mailboxes however, so Dovecot needs some directory where to store the other mailboxes. Typically they're stored in '~/mail/' or '~/Mail/' directory. All of these locations are included in mailbox location autodetection. You can specify them manually with: ---%<------------------------------------------------------------------------- mail_location = mbox:~/mail:INBOX=/var/mail/%u ---%<------------------------------------------------------------------------- Remember that the first path after 'mbox:' is the mailbox root directory, never try to give 'mbox:/var/mail/%u' because that just isn't going to work (unless you really want to store mails under '/var/mail/%u/' directory). If you're also using other software than Dovecot to access mboxes, you should try to figure out what locking methods exactly they're using and update 'mbox_read_locks' and 'mbox_write_locks' settings accordingly. See locking section in [MailboxFormat.mbox.txt] for more information. Maildir ------- Maildir is typically stored in '~/Maildir' directory. You can specify this manually with: ---%<------------------------------------------------------------------------- mail_location = maildir:~/Maildir ---%<------------------------------------------------------------------------- Chrooting --------- Dovecot, including several other software, allow using "/./" in home directory path to specify the chroot path. For example '/home/./user' would chroot to '/home'. If you want to enable this for Dovecot, add the chroot path to 'valid_chroot_dirs' setting ('/home' in the previous example). If this isn't done, Dovecot just ignores the "/./". (This file was created from the wiki on 2011-01-13 04:52) wiki/PasswordDatabase.BSDAuth.txt000066600000000315150501203730012727 0ustar00BSDAuth ======= This is similar to [PasswordDatabase.PAM.txt], but used by OpenBSD. It supports 'cache_key' parameter the same way as PAM. (This file was created from the wiki on 2011-01-13 04:52) wiki/HowTo.PopBSMTPAndDovecot.txt000066600000033614150501203730012631 0ustar00Contents 1. POP3 (IMAP) before SMTP 1. Are you sure you want this? 1. Problems with POP-before-SMTP 2. Advantages of POP-before-SMTP over SMTP AUTH 2. Pop-before-smtp.pl 3. DRAC 4. SQL 1. Example for postgresql, postfix 2. Example for MySQL, postfix 5. relay-ctrl POP3 (IMAP) before SMTP ======================= /sometimes also called SMTP-after-POP3 or SMTP-after-IMAP/ Are you sure you want this? --------------------------- POP-before-SMTP is generally considered a kludge, originally invented to make up for the lack of authentication in the original SMTP [http://en.wikipedia.org/wiki/Smtp] specification for clients on dynamic IP addresses.ESMTP [http://en.wikipedia.org/wiki/Extended_SMTP] resolved that shortcoming long ago, and all modern mail clients and servers support it by now. You should consider implementing ESMTP AUTH [http://en.wikipedia.org/wiki/SMTP-AUTH] in your mail transport/submission agent, and using it in your clients, rather than using POP-before-SMTP. See also or . Problems with POP-before-SMTP ----------------------------- * *Shared IP addresses* are in widespread use. You are opening your server not only to your user, but to anyone else who might be sharing the same IP address, other users, other computers in the same NAT. If you lose the connection, the next one who is assigned your IP also inherits your relay permit. This might include virus-infected spambot machines. Or consider a public wireless hotspot or an Internet cafe: both types of establishments are known to be frequented by spammers. * *Not properly implemented* in all mail clients: it only works right if the client checks for new mail immediately before attempting to send. And it can be very unsafe if longer timeouts are used, such that the user has time to write an email. * Probably others. [RobMcGee.txt] ( [RobMcGee.txt]) just thought it was wrong to have a HOWTO page here without a warning about why /not/ to. Know what you are doing. If you are setting up a new mail service from scratch, by all means, do it right! Advantages of POP-before-SMTP over SMTP AUTH -------------------------------------------- * Likely to be relatively easier to implement in your mail submission agent. What's easier is a matter of opinion, and it varies, of course, but probably all MTA/MSA servers support some form of access lists without patching or recompiling. * Simple non-technical instructions for users: /"Remember to check for new mail before you try to send mail."/ Pop-before-smtp.pl ================== If you want to use pop-before-smtp.pl (from http://popbsmtp.sourceforge.net/) together with Dovecot, you can use this regular expression to match successful POP3 and IMAP logins: ---%<------------------------------------------------------------------------- $pat = '^(... .. ..:..:..) \S+ (?:pop3|imap)-login: Login: .+ \[(\d+\.\d+\.\d+\.\d+)\]'; ---%<------------------------------------------------------------------------- v1.0RC2 seems to need this format to work properly: ---%<------------------------------------------------------------------------- $pat = '^dovecot: (... .. ..:..:..) \S+ (?:pop3|imap)-login: Login: \S+ \S+ \S+ lip=(\d+\.\d+\.\d+\.\d+)'; ---%<------------------------------------------------------------------------- Note: This only works with IPv4, anyone who wants to fix it for IPv6, please do so:) worked for me on Fedora: ---%<------------------------------------------------------------------------- $pat = '(?:pop3|imap)-login: (... .. ..:..:..) Info: Login: \S+ \[(\d+\.\d+\.\d+\.\d+)\]'; ---%<------------------------------------------------------------------------- With v1.0 Alpha 4, the following pattern works: ---%<------------------------------------------------------------------------- $pat = '^(... .. ..:..:..) \S+ (?:dovecot: )?(?:imap|pop3)-login: Login: \S+ \S+ rip=(\d+\.\d+\.\d+\.\d+)' ---%<------------------------------------------------------------------------- This works with RHEL 4.3 (at least until IPv6 really catches): ---%<------------------------------------------------------------------------- $pat = '(?:pop3|imap)-login: (... .. ..:..:..) Info: Login: \S+ \[::ffff:(\d+\.\d+\.\d+\.\d+)\]'; ---%<------------------------------------------------------------------------- DRAC ==== There is a Dovecot plugin for DRAC [http://mail.cc.umanitoba.ca/drac/] at drac.c [http://www.dovecot.org/patches/1.1/drac.c]. DRAC runs as a separate daemon, maintaining a BerkeleyDB database of IPs that have successfully authenticated via POP3 or IMAP, expiring them after 30 minutes. Installing it therefore requires that both your POP3/IMAP server and your SMTP daemon (Postfix/Sendmail/qmail) be set up to support it. DRAC is a small C program, and accessing BerkeleyDB databases is efficient so it works pretty well. The file drac.c has instructions on how to compile it in a comment at the top. By following the instructions you will install a file drac.so in your dovecot 'lib/' directories for IMAP and/or POP3 loadable modules. To turn on the new DRAC plugin in dovecot, you must set up these lines in your dovecot.conf. There is a separate section for ''protocol imap'' and another under ''protocol pop3''; make sure you enable both. ---%<------------------------------------------------------------------------- # Support for dynamically loadable modules mail_plugin_dir = /usr/lib/dovecot/imap mail_plugins = drac # provide a list of all plugins you want to load here ---%<------------------------------------------------------------------------- Permissions note: the directory containing the drac.so file has to be readable by ordinary users. Check your Dovecot error log for help. To get DRAC working on your machine, download the main DRAC [http://mail.cc.umanitoba.ca/drac/] daemon, edit the makefile as directed in the instructions, and make and install it. You will also want to ensure that you register the rpcs by executing rpcgen. See the Makefile for more details. SQL === Advantage: you do not have a multi-megabyte Perl daemon reading your logs Disadvantage: for each login you need the time and space to execute this script 1. tell your MTA to look up IPs authorized to relay in an SQL table 2. delete old IPs from the table regularly (cron job for example, or a modification to the script below) 3. tell dovecot to update the SQL table upon successful login Dovecot 1.0 (and probably 0.99) can update a SQL table with the script below. /!\ *Note* that *you* must set up a script that deletes old IPs separately, and *you* also must configure your MTA properly. The script *only* performs the 'update on successful login' step, which alone is insecure without expiring older IPs!/Add your working examples to this section. This Wiki depends on your help!/ ---%<------------------------------------------------------------------------- #!/bin/sh # This script created 2005-08-21 by Lorens Kockum # Released into the Public Domain # Changes: # 2006-06-06 Matthias Andree # - changed $* to "$@" for more robust argument quoting # Action: when called by dovecot 1.0 as described below, updates an SQL table # with logged-in IP and current time, and then executes the relevant process. # Output: normally nothing # dovecot.conf should be modified with these lines (where # /usr/lib/dovecot/popbsmtp.sh represents this script): # protocol pop3 { # mail_executable = /usr/lib/dovecot/popbsmtp.sh /usr/lib/dovecot/pop3 # } # protocol imap { # mail_executable = /usr/lib/dovecot/popbsmtp.sh /usr/lib/dovecot/imap # } # The HOME= lines are necessary to find $HOME/.my.cnf containing login info, # because mail_executable is executed as root, but without a home directory. # Of course this script must not be writable by anyone else than root. ( # drop out IPs from local networks that can relay anyway IP=`echo $IP | grep -v '^192\.168\.'` if [ -n "$IP" ] then export HOME=/root/ echo "replace into popbsmtp VALUES('$IP',now());" | mysql mail export HOME=/ fi ) >> /var/log/dovecot3 2>&1 exec "$@" ---%<------------------------------------------------------------------------- Example for postgresql, postfix ------------------------------- /usr/lib/dovecot/popbsmtp.sh ---%<------------------------------------------------------------------------- #!/bin/sh ( if [ -n "$IP" ] then /usr/bin/psql -U popbsmtp -d popbsmtp -c "begin;update auth set accessed=now() where host=substring('$IP' from 8);commit;insert into auth(host, accessed) values(substring('$IP' from 8),now());" fi ) >> /var/log/dovecot3 2>&1 exec "$@" ---%<------------------------------------------------------------------------- The substring call was necessary because $IP has '::ffff:' or something like that in front of the IP address on my system. The update followed by an insert, with the update in a transaction is necessary to replicate mysql's REPLACE INTO functionality. The INSERT will produce an error if the IP already exists but it doesn't matter as the UPDATE will have committed by then. /etc/postfix/main.cf ---%<------------------------------------------------------------------------- smtpd_recipient_restrictions = permit_mynetworks permit_sasl_authenticated permit_tls_clientcerts check_client_access pgsql:/etc/postfix/popbsmtp.cf reject_unauth_destination check_policy_service unix:private/policy ---%<------------------------------------------------------------------------- /etc/postfix/popbsmtp.cf ---%<------------------------------------------------------------------------- hosts = localhost user = username password = secret dbname = popbsmtp query = SELECT 'OK' as result FROM auth WHERE host = '%s' ---%<------------------------------------------------------------------------- /etc/cron.hourly/popbsmtp_purge ---%<------------------------------------------------------------------------- #!/bin/bash /usr/bin/psql -U popbsmtp -d popbsmtp -c "DELETE FROM auth WHERE (now() - accessed) > '30 minutes'::interval" ---%<------------------------------------------------------------------------- Example for MySQL, postfix -------------------------- Note that you can use this even if pop/imap and smtp are not on the same host as it is the case in my setup. First you have to create a table (in this example named "popbsmtp") with 2 fields: * address (varchar 39, primary) * last_seen (datetime) varchar size 39 is for IPv6 addresses.You should definitely consider adding IPv6 support to your popbsmtp solution because postfix and dovecot do well with IPv6. /!\ *address field* must be *primary* for "REPLACE into" to work. /opt/dovecot-popbsmtp.sh ---%<------------------------------------------------------------------------- #!/bin/sh ( if [ -n "$IP" ] then echo "REPLACE INTO virtual_mail.popbsmtp (address,last_seen) VALUES ('$IP', NOW( ))" \ | mysql -u user -p secret -h host > /dev/null 2>&1 fi ) exec "$@" ---%<------------------------------------------------------------------------- mail_executable in dovecot.conf looks something like this: ---%<------------------------------------------------------------------------- mail_executable = /opt/dovecot-popbsmtp.sh /usr/libexec/dovecot/imap ---%<------------------------------------------------------------------------- postfix map (/etc/postfix/mysql_popbsmtp_access_maps.cf): ---%<------------------------------------------------------------------------- hosts = mysqlhost user = user password = secret dbname = virtual_mail query = SELECT 'OK' FROM popbsmtp WHERE last_seen >= DATE_SUB(NOW(),INTERVAL 30 MINUTE) AND address = '%s' ---%<------------------------------------------------------------------------- In postfix main.cf add the following access map to your recipient restrictions (/!\ *before* "reject_unauth_destination"): ---%<------------------------------------------------------------------------- check_client_access mysql:$config_directory/mysql_popbsmtp_access_maps.cf ---%<------------------------------------------------------------------------- The 30 minute relay access period is handled by the INTERVAL in DATE_SUB. So it's safe anyway, but you should definitely run a cron job daily that deletes older records. That's to keep the table clean and speed up lookups. You might also need to run "OPTIMIZE TABLE" via the cron job to free up allocated space. relay-ctrl ========== relay-ctrl [http://untroubled.org/relay-ctrl/] consists of a few small programs designed to fit in qmail-like command chains. The most important: * 'relay-ctrl-allow' runs after a successful POP/IMAP login, recording the client IP and timestamp * 'relay-ctrl-check' runs before the SMTP server, enabling relaying if the client IP has authenticated recently 'relay-ctrl-allow' expects to find the client IP in the environment as '$TCPREMOTEIP'. Dovecot provides it as '$IP', so you'll need this tiny 'dovecot-settcpremoteip' wrapper script: ---%<------------------------------------------------------------------------- #!/bin/sh # # Wrapper for relay-ctrl-allow that sets TCPREMOTEIP. TCPREMOTEIP="${IP}"; export TCPREMOTEIP exec "$@" ---%<------------------------------------------------------------------------- Edit 'dovecot.conf' and set 'mail_executable' appropriately, e.g., for IMAP (this is one long line): ---%<------------------------------------------------------------------------- mail_executable = /usr/local/bin/envdir /etc/relay-ctrl /usr/local/bin/relay-ctrl-chdir /usr/local/bin/dovecot-settcpremoteip /usr/local/bin/relay-ctrl-allow /usr/local/libexec/dovecot/imap Dove ---%<------------------------------------------------------------------------- Restart Dovecot. Verify that your IMAP client still works. Verify that relay-ctrl has recorded your client IP. Hook 'relay-ctrl-check' into your SMTP service, as documented in the relay-ctrl README, and you're done. (This file was created from the wiki on 2011-01-13 04:52) wiki/mutt.txt000066600000006136150501203730007247 0ustar00Contents 1. Using mutt with IMAP 1. configuration 2. problems 1. Move read messages to mbox 2. New mail in a folder just left 3. Authentication realm 2. References Using mutt with IMAP ==================== configuration ------------- First, mutt needs to be told to use IMAP. To achieve this, edit '~/.muttrc' (or a file it 'source's) to contain: ---%<------------------------------------------------------------------------- set spoolfile=imap://user@hostname/INBOX set folder=imap://user@hostname/ ---%<------------------------------------------------------------------------- problems -------- Move read messages to mbox -------------------------- If mutt asks 'Move read messages to /home/$user/mbox? ([no]/yes):' Your alternatives (to set in '~/.muttrc') are (pick one and ignore the others): 1. don't ask about moving messages, just do it: ---%<--------------------------------------------------------------------- set move=yes ---%<--------------------------------------------------------------------- 2. don't ask about moving messages and _don't_ do it: ---%<--------------------------------------------------------------------- set move=no ---%<--------------------------------------------------------------------- 3. ask about moving message, default answer 'yes': ---%<--------------------------------------------------------------------- set move=ask-yes ---%<--------------------------------------------------------------------- 4. ask about moving message, default answer 'no': ---%<--------------------------------------------------------------------- set move=ask-no ---%<--------------------------------------------------------------------- New mail in a folder just left ------------------------------ There is a bug with dovecot in <=0.99. It can sometimes make mutt think there is new email in a mailbox you just 'left'...http://www.dovecot.org/list/dovecot/2004-February/002950.html Authentication realm -------------------- If you are using IMAP with SASL and get error messages like 'Invalid realm' in your dovecot log files, try putting this in your '.muttrc' or '/etc/Muttrc]': ---%<------------------------------------------------------------------------- set imap_authenticators="plain" ---%<------------------------------------------------------------------------- It might be easier however to just put the following in your 'dovecot.conf': For Dovecot 1.0: ---%<----------------------------------------------------------------------- auth default { mechanisms = plain } ---%<----------------------------------------------------------------------- For Dovecot 0.99: ---%<----------------------------------------------------------------------- auth_mechanisms = plain ---%<----------------------------------------------------------------------- References ========== * Official mutt homepage [http://www.mutt.org/] * mutt with IMAP documentation [http://mutt.sourceforge.net/imap/] * http://jamespo.org.uk/blog/archives/000271.html (This file was created from the wiki on 2011-01-13 04:52) wiki/Pigeonhole.Sieve.Configuration.txt000066600000030663150501203730014231 0ustar00Pigeonhole Sieve Configuration ============================== Basic Configuration ------------------- To use Sieve, you will first need to make sure you are using Dovecot or for delivering incoming mail to users' mailboxes. Then, you need to enable the Pigeonhole Sieve plugin in your configuration: ---%<------------------------------------------------------------------------- protocol lda { mail_plugins = $mail_plugins sieve } protocol lmtp { mail_plugins = $mail_plugins sieve } ---%<------------------------------------------------------------------------- The sieve plugin recognizes the following configuration options in the 'plugin' section of the config file (default values are shown if applicable): sieve = ~/.dovecot.sieve : The path to the user's main active script. sieve_global_path : A path to a global sieve script file, which gets executed ONLY if user's private Sieve script doesn't exist, e.g.'/var/lib/dovecot/default.sieve'. Check the [Pigeonhole.Sieve.Configuration.txt] for instructions on running global Sieve scripts before and after the user's personal script. Be sure to pre-compile the specified script manually using the 'sievec' command line tool, as explained [Pigeonhole.Sieve.Usage.txt]. sieve_global_dir = : Directory for :global include scripts for the include extension [http://tools.ietf.org/html/draft-ietf-sieve-include-05]. The Sieve interpreter only recognizes files that end with a '.sieve' extension, so the include extension expects a file called 'name.sieve' to exist in the 'sieve_global_dir' directory for the following example: ---%<------------------------------------------------------------------------- require "include"; include :global "name"; ---%<------------------------------------------------------------------------- sieve_dir = ~/: Directory for :personal include scripts for the include extension [http://tools.ietf.org/html/draft-ietf-sieve-include-05]. The Sieve interpreter only recognizes files that end with a '.sieve' extension, so the include extension expects a file called 'name.sieve' to exist in the 'sieve_dir' directory for: ---%<------------------------------------------------------------------------- require "include"; include :personal "name"; ---%<------------------------------------------------------------------------- sieve_extensions = : Which Sieve language extensions are available to users. By default, all supported extensions are available, except for deprecated extensions or those that are still under development. Some system administrators may want to disable certain Sieve extensions or enable those that are not available by default. All supported extensions are listed [Pigeonhole.Sieve.txt]. Normally, all enabled extensions must be listed for this setting, but starting with Sieve version 0.1.7, this setting can use '+' and '-' to specify differences relative to the default. For example 'sieve_extensions = +imapflags' will enable the deprecated imapflags extension [http://tools.ietf.org/html/draft-melnikov-sieve-imapflags-03] in addition to all extensions enabled by default. sieve_plugins = : The Pigeonhole Sieve interpreter can have plugins of its own. Using this setting, the used plugins can be specified. Check the [Pigeonhole.Sieve.Plugins.txt] for available plugins. recipient_delimiter = +: The separator that is expected between the :user and :detail address parts introduced by the subaddress extension [http://tools.ietf.org/html/rfc5233/]. This may also be a sequence of characters (e.g. '--'). The current implementation looks for the separator from the left of the localpart and uses the first one encountered. The :user part is left of the separator and the :detail part is right. This setting is also used by Dovecot's service with identical semantics. For example: ---%<------------------------------------------------------------------------- plugin { ... # The location of the user's active script: sieve = ~/.dovecot.sieve # If the user has no personal active script (i.e. if the file # indicated in sieve= does not exist), use this one: sieve_global_path = /var/lib/dovecot/sieve/default.sieve # The include extension fetches the :personal scripts from this # directory. When ManageSieve is used, this is also where scripts # are uploaded. sieve_dir = ~/sieve # The include extension fetches the :global scripts from this # directory. sieve_global_dir = /var/lib/dovecot/sieve/global/ } ---%<------------------------------------------------------------------------- Per-user Sieve script location ------------------------------ By default, the Dovecot Sieve plugin looks for the user's Sieve script file in the user's home directory ('~/.dovecot.sieve'). This requires that the [VirtualUsers.txt] is set for the user. If you want to store the script elsewhere, you can override the default using the 'sieve' setting, which specifies the path to the user's script file. This can be done in two ways: 1. Define the 'sieve' setting in the plugin section of 'dovecot.conf'. 2. Return 'sieve' extra field from [UserDatabase.ExtraFields.txt]. For example, to use a Sieve script file named '.sieve' in '/var/sieve-scripts', use: ---%<------------------------------------------------------------------------- plugin { ... sieve = /var/sieve-scripts/%u.sieve } ---%<------------------------------------------------------------------------- You may use templates like %u, as shown in the example. See all [Variables.txt]. A relative path (or just a filename) will be interpreted to point under the user's home directory. Executing Multiple Scripts Sequentially --------------------------------------- The Dovecot Sieve plugin allows executing multiple Sieve scripts sequentially. The extra scripts can be executed before and after the user's private script. For example, this allows executing global Sieve policies before the user's script. This is not possible using the 'sieve_global_path' setting, because that is only used when the user's private script does not exist. The following settings in the 'plugin' section of the Dovecot config file control the execution sequence: sieve_before = : Path to a script file or a directory containing script files that need to be executed before the user's script. If the path points to a directory, all the Sieve scripts contained therein (with the proper '.sieve' extension) are executed. The order of execution is determined by the file names, using a normal 8bit per-character comparison. sieve_after = : Identical to 'sieve_before', only the specified scripts are executed after the user's script (only when keep is still in effect!). The script execution ends when the currently executing script in the sequence does not yield a "keep" result: when the script terminates, the next script is only executed if an implicit or explicit "keep" is in effect. Thus, to end all script execution, a script must not execute keep and it must cancel the implicit keep, e.g. by executing "'discard; stop;'". This means that the command "'keep;'" has different semantics when used in a sequence of scripts. For normal Sieve execution, "'keep;'" is equivalent to "'fileinto "INBOX";'", because both cause the message to be stored in INBOX. However, in sequential script execution, it only controls whether the next script is executed. Storing the message into INBOX (the default folder) is not done until the last script in the sequence executes (implicit) keep. To force storing the message into INBOX earlier in the sequence, the fileinto command can be used (with "':copy'" or together with "'keep;'"). Apart from the keep action, all actions triggered in a script in the sequence are executed before continuing to the next script. This means that when a script in the sequence encounters an error, actions from earlier executed scripts are not affected. The sequence is broken however, meaning that the script execution of the offending script is aborted and no further scripts are executed. An implicit keep is executed in stead. Just as for executing a single script the normal way, the Dovecot Sieve plugin takes care never to duplicate deliveries, forwards or responses. When vacation actions are executed multiple times in different scripts, the usual error is not triggered: the subsequent duplicate vacation actions are simply discarded. For example: ---%<------------------------------------------------------------------------- plugin { ... # Scripts executed before the user's script. # E.g. handling messages marked as dangerous sieve_before = /var/lib/dovecot/sieve/discard-virusses.sieve # Scripts executed after the user's script (if keep is still in effect) # E.g. default mail filing rules. sieve_after = /var/lib/dovecot/sieve/after.d/ } ---%<------------------------------------------------------------------------- *IMPORTANT*: Be sure to manually pre-compile the scripts specified by 'sieve_before' and 'sieve_after' using the 'sievec' tool, as explained [Pigeonhole.Sieve.Usage.txt]. Migration --------- General Dovecot 2.0 changes --------------------------- * Note that the Dovecot v2.0 does not create mailfolders automatically by default anymore. If your configuration relies on this, you need to enable the 'lda_mailbox_autocreate' setting for or start using the Sieve mailbox extension's ':create' tag for *fileinto* commands. * Dovecot v2.0 adds support for . Much like the [LDA.txt], it can make use of the Pigeonhole Sieve plugin. Since the service has its own 'prototocol lmtp' section in the config file, you need to add the Sieve plugin to the 'mail_plugins' setting there too when you decide to use . From CMUSieve (Dovecot v1.0/v1.1) --------------------------------- For the most part, migration from CMUSieve to Pigeonhole Sieve is just a matter of changing the used plugin name from *cmusieve* to *sieve* in the 'mail_plugins' option in the 'protocol lda' section of the config file (as explained [Pigeonhole.Sieve.Configuration.txt]). However, there are a few important differences in the supported Sieve language features: * The *imapflags* extension is now called *imap4flags*. The CMUSieve implementation is based on an old draft specification [http://tools.ietf.org/html/draft-melnikov-sieve-imapflags-03] that is not completely compatible with the new version [http://tools.ietf.org/html/rfc5232/]. Particularly, the *mark* and *unmark* commands were removed from the new specification. For backwards compatibility, support for the old imapflags extension can be enabled using the 'sieve_extensions' setting (as explained [Pigeonhole.Sieve.Configuration.txt]). This is disabled by default. * The *notify* extension is now called *enotify*. The CMUSieve implementation is based on an old draft specification [http://tools.ietf.org/html/draft-martin-sieve-notify-01] that is not completely compatible with the new version [http://tools.ietf.org/html/rfc5435/]. Particularly, the *denotify* command and *$text$* substitutions were removed from the new specification. For backwards compatibility, support for the old imapflags extension can be enabled using the 'sieve_extensions' setting (as explained [Pigeonhole.Sieve.Configuration.txt]). This is disabled by default. * The include extension [http://tools.ietf.org/html/draft-ietf-sieve-include] now requires your script file names to end with ".sieve" :'include :personal "myscript";' won't work unless you rename "myscript" to "myscript.sieve" * Be sure to use *UTF8* for the mailbox argument of the *fileinto* command. Older CMUSieve installations used modified UTF7 (as IMAP does) for the mailbox parameter. If not adjusted, the Pigeonhole Sieve plugin will use the wrong folder name for storing the message. From Dovecot Sieve v0.1.x (Dovecot v1.2) ---------------------------------------- * The 'sieve_subaddress_sep' setting for the Sieve subaddress extension [http://tools.ietf.org/html/rfc5233/] is now known as 'recipient_delimiter'. Although 'sieve_subaddress_sep' is still recognized for backwards compatibility, it is recommended to update the setting to the new name, since the service also uses the 'recipient_delimiter' setting. (This file was created from the wiki on 2011-01-13 04:52) wiki/PasswordDatabase.ExtraFields.Proxy.txt000066600000015446150501203730015042 0ustar00Proxying ======== Dovecot supports proxying IMAP, POP3 and connections to other hosts. The proxying can be done for all users, or only for some specific users. There are two ways to do the authentication: 1. Forward the password to the remote server and let it perform the actual authentication. This requires that the client uses only plaintext authentication. 2. Let Dovecot proxy perform the authentication and login to remote server using the proxy's [MasterPassword.txt]. This allows client to use also non-plaintext authentication. The proxy is configured pretty much the same way as [PasswordDatabase.ExtraFields.Host.txt], with the addition of 'proxy' field. The common fields to use for both proxying ways are: * 'proxy' and 'proxy_maybe': Enables the proxying. Either one of these fields is required. * 'proxy_maybe' can be used to implement "automatic proxying". If the proxy destination matches the current connection, the user gets logged in normally instead of being proxied. If the same happens with 'proxy', the login fails with "Proxying loops" error. * 'host=s': The destination server's *IP address*. This field is required. * 'port=s': The destination server's port. The default is 143 with IMAP and 110 with POP3. * 'destuser=s': Tell client to use a different username when logging in. * 'proxy_timeout': Abort connection after this many seconds. You can use SSL/TLS connection to destination server by returning: * ssl=yes: Use SSL and require a valid verified remote certificate. *WARNING: Unless used carefully, this is an insecure setting!* Currently the host name isn't checked in any way against the certificate's CN. The only way to use this securely is to only use and allow your own private CA's certs, anything else is exploitable by a man-in-the-middle attack. * ssl=any-cert: Use SSL, but don't require a valid remote certificate. * starttls: Use STARTTLS command instead of doing SSL handshake immediately after connected. * starttls=any-cert: Combine starttls and ssl=any-cert. The destination servers don't need to be running Dovecot, but you should make sure that the Dovecot proxy doesn't advertise more capabilities than the destination server can handle. For IMAP you can do this by changing 'imap_capability' setting. For POP3 you'll have to modify Dovecot's sources for now ('src/pop3/capability.h'). Dovecot also automatically sends updated untagged CAPABILITY reply if it detects that the remote server has different capabilities than what it already advertised to the client, but some clients simply ignore the updated CAPABILITY reply. Password forwarding ------------------- Make sure that the authentication succeeds with any given password. You can do this by returning empty password and 'nopassword' field. Master password --------------- This way of forwarding requires the destination server to support master user feature. The users will be normally authenticated in the proxy and the common proxy fields are returned, but you'll need to return two fields specially: * 'master=s': This contains the master username (e.g. "proxy"). It's used as SASL auhentication ID. * Alternatively you could return 'destuser=user*master' and set 'auth_master_user_separator = *'. * 'pass=s': This field contains the master user's password. See for more information how to configure this. Example password forwarding SQL configuration --------------------------------------------- Create the SQL table: ---%<------------------------------------------------------------------------- CREATE TABLE proxy ( user varchar(255) NOT NULL, host varchar(16) default NULL, destuser varchar(255) NOT NULL default '', PRIMARY KEY (user) ); ---%<------------------------------------------------------------------------- Insert data to SQL corresponding your users. Working data could look like this: +------+-------------+-----------------+ | user | host | destuser | +------+-------------+-----------------+ | john | 192.168.0.1 | | +------+-------------+-----------------+ | joe | 192.168.0.2 | joe@example.com | +------+-------------+-----------------+ The important parts of 'dovecot.conf': ---%<------------------------------------------------------------------------- # If you want to trade a bit of security for higher performance, change these settings: service imap-login { service_count = 0 } service pop3-login { service_count = 0 } # If you are not moving mailboxes between hosts on a daily basis you can # use authentication cache pretty safely. auth_cache_size = 4096 auth_mechanisms = plain passdb { driver = sql args = /usr/local/etc/dovecot/dovecot-sql.conf.ext } ---%<------------------------------------------------------------------------- The important parts of 'dovecot-sql.conf.ext': ---%<------------------------------------------------------------------------- driver = mysql connect = host=sqlhost1 host=sqlhost2 dbname=mail user=dovecot password=secret password_query = SELECT NULL AS password, 'Y' as nopassword, host, destuser, 'Y' AS proxy FROM proxy WHERE user = '%u' ---%<------------------------------------------------------------------------- Example proxy_maybe SQL configuration ------------------------------------- Create the SQL table: ---%<------------------------------------------------------------------------- CREATE TABLE users ( user varchar(255) NOT NULL, domain varchar(255) NOT NULL, password varchar(100) NOT NULL, host varchar(16) NOT NULL, home varchar(100) NOT NULL, PRIMARY KEY (user) ); ---%<------------------------------------------------------------------------- The important parts of 'dovecot.conf': ---%<------------------------------------------------------------------------- # user/group who owns the message files: mail_uid = vmail mail_gid = vmail auth_mechanisms = plain passdb { driver = sql args = /usr/local/etc/dovecot/dovecot-sql.conf.ext } userdb sql { driver = sql args = /usr/local/etc/dovecot/dovecot-sql.conf.ext } ---%<------------------------------------------------------------------------- The important parts of 'dovecot-sql.conf.ext': ---%<------------------------------------------------------------------------- driver = mysql password_query = \ SELECT concat(user, '@', domain) AS user, password, host, 'Y' AS proxy_maybe \ FROM users WHERE user = '%n' AND domain = '%d' user_query = SELECT user AS username, domain, home \ FROM users WHERE user = '%n' AND domain = '%d' ---%<------------------------------------------------------------------------- Example proxy LDAP configuration -------------------------------- see: for more information, and a worked out example (This file was created from the wiki on 2011-01-13 04:52) wiki/MailLocation.mbox.txt000066600000017170150501203730011575 0ustar00mbox configuration ================== See for a complete description of how Dovecot has implemented mbox support. Mail location ------------- In many systems the user's mails are by default stored in '/var/mail/username' file. This file is called INBOX in IMAP world. Since IMAP supports multiple mailboxes, you'll need to have a directory for them as well. Usually '~/mail' is a good choice for this. For installation such as this, the mail location is specified with: ---%<------------------------------------------------------------------------- # %u is replaced with the username that logs in mail_location = mbox:~/mail:INBOX=/var/mail/%u ---%<------------------------------------------------------------------------- It's in no way a requirement to have the INBOX in '/var/mail/' directory. In fact this often just brings problems because Dovecot might not be able to write dotlock files to the directory (see below). You can avoid this completely by just keeping everything in '~/mail/': ---%<------------------------------------------------------------------------- # INBOX exists in ~/mail/inbox mail_location = mbox:~/mail ---%<------------------------------------------------------------------------- Index files ----------- See [MailLocation.txt] for full explanation of how to change the index path. For example: ---%<------------------------------------------------------------------------- mail_location = mbox:~/mail:INBOX=/var/mail/%u:INDEX=/var/indexes/%u ---%<------------------------------------------------------------------------- Locking ------- Make sure that all software accessing the mboxes are using the same locking methods in the same order. The order is important to prevent deadlocking. From Dovecot's side you can change these from 'mbox_read_locks' and 'mbox_write_locks' settings. See for more information. /var/mail/ dotlocks ------------------- Often mbox write locks include dotlock, which means that Dovecot needs to create a new ".lock" file to the directory where the mbox file exists. If your INBOXes are in '/var/mail/' directory you may have to give Dovecot write access to the directory. There are two ways the '/var/mail/' directory's permissions have traditionally been set up: * World-writable with sticky bit set, allowing anyone to create new files but not overwrite or delete existing files owned by someone else (ie. same as /tmp). You can do this with 'chmod a+rwxt /var/mail' * Directory owned by a mail group and the directory set to group-writable You can give Dovecot access to mail group by setting: ---%<------------------------------------------------------------------------- mail_privileged_group = mail ---%<------------------------------------------------------------------------- NOTE: With the 'mail_privileged_group' setting unfortunately doesn't work, so you'll have to use the sticky bit, disable dotlocking completely or use LMTP server instead. Optimizations ------------- The settings below are related to mbox performance. See for more complete description of what they do. * 'mbox_lazy_writes=yes' (default): Metadata updates, such as writing X-UID headers or flag changes, aren't written to mbox file until the mailbox is closed or CHECK or EXPUNGE IMAP commands are sent by the client. The mbox rewrites can be costly, so this may avoid a lot of disk writes. * 'mbox_dirty_syncs=yes' (default): Dovecot assumes that external mbox file changes only mean that new messages were appended to it. Without this setting Dovecot re-reads the whole mbox file whenever it changes. There are various safeguards in place to make this setting safe even when other changes than appends were done to the mbox. The only downside to this setting is that external message flag modifications may not be visible immediately. * 'mbox_very_dirty_syncs=yes' (not default): When opening mbox file that has been changed externally, don't re-read it. Otherwise similar to 'mbox_dirty_syncs=yes'. * 'mbox_min_index_size=n': If mbox file is smaller than n kilobytes, don't update its index files. If an index file exists for it, it's still read however. Only /var/mail/ mboxes ---------------------- With POP3 it's been traditional that users have their mails only in the '/var/mail/' directory. IMAP however supports having multiple mailboxes, so each user has to have a private directory where the mailboxes are stored. Dovecot also needs a directory for its index files unless you disable them completely. If you *really* want to use Dovecot as a plain POP3 server without index files, you can work around the problem of not having the per-user directory: * Set users' home directory in userdb to some empty non-writable directory, for example '/var/empty' * Modify 'mail_location' setting so that the mail root directory is also the empty directory and append ':INDEX=MEMORY' to it. For example: 'mail_location = mbox:/var/empty:INBOX=/var/mail/%u:INDEX=MEMORY' * Note that if you have IMAP users, they'll see the '/var/empty' as the directory containing other mailboxes than INBOX. If the directory is writable, all the users will have their mailboxes shared. Directory layout ---------------- By default Dovecot uses filesystem layout under mbox. This means that mail is stored in mbox files under hierarchical directories, for example: * '~/mail/inbox' - mbox file containing mail for INBOX * '~/mail/foo' - mbox file containing mail for mailbox "foo" * '~/mail/bar/baz' - mbox file containing mail for mailbox "bar/baz" One upshot of this is that it is not normally possible to have mailboxes which are subfolders of mailboxes containing messages. As an alternative, it is possible to configure Dovecot to store all mailboxes in a single directory with hierarchical levels separated by a dot. This can be configured by adding ':LAYOUT=maildir++' to the mail location. There are, however, some further considerations when doing this; see for some examples. Control files ------------- Under mbox format, Dovecot maintains the subscribed mailboxes list in a file '.subscriptions' which by default is stored in the mail location root. So in the example configuration this would be at '~/mail/.subscriptions'. If you want to put this somewhere else, you can change the directory in which the '.subscriptions' file is kept by using the 'CONTROL' parameter. So for example, if we configured the mail location using: ---%<------------------------------------------------------------------------- mail_location = mbox:~/mail:CONTROL=~/mail-control ---%<------------------------------------------------------------------------- then the subscribed mailboxes list would be maintained at '~/mail-control/.subscriptions'. One practical application of the 'CONTROL' parameter is described at . Message file name ----------------- By default, Dovecot stores messages for INBOX in an mbox file called "inbox", and messages for all other mailboxes in an mbox file whose relative path is equivalent to the name of the mailbox. Under this scheme, it is not possible to have mailboxes which contain both messages and child mailboxes. However, the behaviour (for mailboxes other than INBOX) can be changed using the 'DIRNAME' parameter. If the 'DIRNAME' parameter is specified with a particular value, then Dovecot will store messages in a file with a name of that value, in a directory with a name equivalent to the mailbox name. There are, however, some further considerations when doing this; see for an example. (This file was created from the wiki on 2011-01-13 04:52) wiki/PasswordDatabase.Static.txt000066600000001503150501203730012724 0ustar00Static Password Database ======================== Static password database is typically used only for testing, proxying setups and perhaps some other special kind of setups. Static passdb allows all users to log in with any username. For password you return either: * password=secret: All users have "secret" as password. * nopassword: Users can log in with any password. You can return any other [PasswordDatabase.ExtraFields.txt]. You can use the standard [Variables.txt] everywhere. Example ------- Using : ---%<------------------------------------------------------------------------- userdb { driver = static args = proxy=y nopassword=y } ---%<------------------------------------------------------------------------- (This file was created from the wiki on 2011-01-13 04:52) wiki/Debugging.Authentication.txt000066600000006453150501203730013131 0ustar00Debugging Authentication ======================== The most important thing to do is to set 'auth_debug=yes', and preferrably also 'auth_debug_passwords=yes'. After that you'll see in the logs exactly what dovecot-auth is doing, and that should help you to fix the problem. PLAIN SASL mechanism -------------------- With IMAP and POP3 it's easy to log in manually using the IMAP's LOGIN command or POP3's USER and PASS commands (see and for details), but with SMTP AUTH you'll need to use PLAIN authentication mechanism, which requires you to build a base64-encoded string in the correct format. The PLAIN authentication is also used internally by both IMAP and POP3 to authenticate to dovecot-auth, so you see it in the debug logs. The PLAIN mechanism's authentication format is: NUL NUL . Authorization ID is the username who you want to log in as, and authentication ID is the username whose password you're giving. If you're not planning on doing a [Authentication.MasterUsers.txt], you can either set both of these fields to the same username, or leave the authorization ID empty. Encoding with mmencode ---------------------- printf(1) and mmencode(1) should be available on most Unix or GNU/Linux systems. (If not, check with your distribution. GNU coreutils includes printf(1), and metamail includes mmencode(1). In Debian, mmencode is called mimencode(1).) ---%<------------------------------------------------------------------------- $ printf 'username\0username\0password' | mmencode dXNlcm5hbWUAdXNlcm5hbWUAcGFzc3dvcmQ= ---%<------------------------------------------------------------------------- This string is what a client would use to attempt PLAIN authentication as user "username" with password "password." With ''auth_debug_passwords=yes', it would appear in your logs. Decoding with mmencode ---------------------- You can use mmencode -u to interpret the encoded string pasted into stdin as follows: ---%<------------------------------------------------------------------------- # mmencode -u bXl1c2VybmFtZUBkb21haW4udGxkAG15dXNlcm5hbWVAZG9tYWluLnRsZABteXBhc3N3b3Jk myusername@domain.tldmyusername@domain.tldmypassword # ---%<------------------------------------------------------------------------- You should see the correct user address (twice) and password. The null bytes won't display. Encoding with Perl ------------------ Unfortunately, mmencode on FreeBSD chokes on "\0". As an alternate, if you have MIME::Base64 on your system, you can use a perl statement to do the same thing: ---%<------------------------------------------------------------------------- perl -MMIME::Base64 -e 'print encode_base64("myusername\@domain.tld\0myusername\@domain.tld\0mypassword");' ---%<------------------------------------------------------------------------- As mmencode -u doesn't encounter any "\0" you can still do: ---%<------------------------------------------------------------------------- perl -MMIME::Base64 -e 'print encode_base64("myusername\@domain.tld\0myusername\@domain.tld\0mypassword");' | mmencode -u ---%<------------------------------------------------------------------------- to check that you have encoded correctly. (This file was created from the wiki on 2011-01-13 04:52) wiki/PasswordDatabase.txt000066600000006123150501203730011501 0ustar00Password Databases ================== Dovecot authenticates users against password databases. It can also be used to configure things like [PasswordDatabase.ExtraFields.Proxy.txt]. You can use multiple databases, so if the password doesn't match in the first database, Dovecot checks the next one. This can be useful if you want to easily support having both virtual users and also local system users (see ). Success/failure databases ------------------------- These databases simply verify if the given password is correct for the user. Dovecot doesn't get the correct password from the database, it only gets a "success" or a "failure" reply. This means that these databases can't be used with non-plaintext [Authentication.Mechanisms.txt]. Databases that belong to this category are: * [PasswordDatabase.PAM.txt]: Pluggable Authentication Modules. * [PasswordDatabase.BSDAuth.txt]: BSD authentication. * [AuthDatabase.CheckPassword.txt]: External checkpassword program. Lookup databases ---------------- Dovecot does a lookup based on the username and possibly other information (e.g. IP address) and verifies the password validity itself. Fields that the lookup can return: * [Authentication.PasswordSchemes.txt]: User's password. * password_noscheme: Like "password", but if a password begins with "{", assume it belongs to the password itself instead of treating it as a [Authentication.PasswordSchemes.txt] prefix. This is usually needed only if you use plaintext passwords. * [PasswordDatabase.ExtraFields.User.txt]: Returning a user field can be used to change the username. Typically used only for case changes (e.g. "UseR" -> "user"). * username: Like user, but doesn't drop existing domain name (e.g. "username=foo" for "user@domain" gives "foo@domain"). * domain: Updates the domain part of the username. * Other special [PasswordDatabase.ExtraFields.txt]. Databases that support looking up only passwords, but no user or extra fields: * [AuthDatabase.Passwd.txt]: System users (NSS, '/etc/passwd', or similiar). * [PasswordDatabase.Shadow.txt]: Shadow passwords for system users (NSS,'/etc/shadow' or similiar). * Dovecot supports reading all [Authentication.PasswordSchemes.txt] from passwd and shadow databases (if prefix is specified), but that is of course incompatible with all other tools using/modifying the passwords. * [AuthDatabase.VPopMail.txt]: External software used to handle virtual domains. Databases that support looking up everything: * [AuthDatabase.PasswdFile.txt]: '/etc/passwd'-like file in specified location. * [AuthDatabase.LDAP.txt]: Lightweight Directory Access Protocol. * [AuthDatabase.SQL.txt]: SQL database (PostgreSQL, MySQL, SQLite). * [PasswordDatabase.Static.txt]: Static passdb for simple configurations (This file was created from the wiki on 2011-01-13 04:52) wiki/Authentication.PasswordSchemes.txt000066600000017537150501203730014355 0ustar00Password Schemes ================ Password scheme means the format in which the password is stored in [PasswordDatabase.txt]. The most commonly used password schemes are: * *PLAIN*: Password is in plaintext. * *CRYPT*: Traditional DES-crypted password in '/etc/passwd' (e.g. "pass" = 'vpvKh.SaNbR6s') * Dovecot uses libc's 'crypt()' function, which means that CRYPT is usually able to decrypt also MD5-CRYPT and possibly also other password schemes. * Only the first 8 characters of the password are used, the rest are ignored. * *MD5-CRYPT*: MD5 based salted password hash nowadays commonly used in '/etc/shadow'. (e.g. "pass" = '$1$ozdpg0V0$0fb643pVsPtHVPX8mCZYW/') * *MD5*: Alias for MD5-CRYPT. Dovecot versions earlier than v1.0.rc16 need to use this instead of MD5-CRYPT. This name is deprecated because MD5-CRYPT isn't an actual MD5 hash. * *PLAIN-MD5*: An actual MD5 hash of the password. (e.g. "pass" = '1a1dc91c907325c69271ddf0c944bc72') Password databases have a default password scheme: * [AuthDatabase.SQL.txt]: See 'default_pass_scheme' setting in 'dovecot-sql.conf.ext' * [AuthDatabase.LDAP.txt]: See 'default_pass_scheme' setting in 'dovecot-ldap.conf.ext' * [AuthDatabase.PasswdFile.txt]: CRYPT is used by default, but can be changed with 'scheme' parameter in passdb args. * [AuthDatabase.Passwd.txt], [PasswordDatabase.Shadow.txt], [AuthDatabase.VPopMail.txt]: CRYPT is used by default and can't be changed currently. * [PasswordDatabase.PAM.txt], [PasswordDatabase.BSDAuth.txt], [PasswordDatabase.CheckPassword.txt]: Dovecot never even sees the password with these databases, so Dovecot has nothing to do with what password scheme is used. The password scheme can be overridden for each password by prefixing it with {SCHEME}, for example:'{PLAIN}pass'. You can generate passwords for wanted scheme easily with "doveadm pw" utility. For example: ---%<------------------------------------------------------------------------- doveadm pw -s ssha256 ---%<------------------------------------------------------------------------- What scheme to use? ------------------- With most installations it doesn't really matter what scheme you're using. If you already have users with existing passwords, it's easiest to just keep using the same scheme. Otherwise just pick something strong enough, for example SSHA256. The main idea behind storing passwords in non-plaintext scheme is that if an attacker gets access to your server, he can't easily just get all users' passwords and start using them. With stronger schemes it takes more time to crack the passwords. Non-plaintext authentication mechanisms --------------------------------------- See for explanation of auth mechanisms. Most installations use only plaintext mechanisms, so you can skip this section unless you know you want to use them. The problem with non-plaintext auth mechanisms is that the password must be stored either in plaintext, or using a mechanism-specific scheme that's incompatible with all other non-plaintext mechanisms. For example if you're going to use CRAM-MD5 authentication, the password needs to be stored in either PLAIN or CRAM-MD5 scheme. If you want to allow both CRAM-MD5 and DIGEST-MD5, the password must be stored in plaintext. In future it's possible that Dovecot could support multiple passwords in different schemes for a single user. Supported schemes ----------------- *PLAIN*, *CRYPT* and *MD5-CRYPT* schemes were explained above. Non-plaintext mechanism specific schemes: * *LANMAN*: DES-based encryption. Used sometimes with NTLM mechanism. * *NTLM*: MD4 sum of the password stored in hex. Used with NTLM mechanism. * *RPA*: Used with RPA mechanism. * *CRAM-MD5*: Used with CRAM-MD5 mechanism. * *DIGEST-MD5*: Used with [Authentication.Mechanisms.DigestMD5.txt]. The username is included in the hash, so it's not possible to use the hash for different usernames. MD5 based schemes: * *PLAIN-MD5*: MD5 sum of the password stored in hex. * *LDAP-MD5*: MD5 sum of the password stored in base64. * *SMD5*: Salted MD5 sum of the password stored in base64. SHA based schemes (also see below for libc's SHA* support): * *SHA*: SHA1 sum of the password stored in base64. * *SSHA*: Salted SHA1 sum of the password stored in base64. * *SHA256*: SHA256 sum of the password stored in base64. (v1.1 and later). * *SSHA256*: Salted SHA256 sum of the password stored in base64. (v1.2 and later). * *SHA512*: SHA512 sum of the password stored in base64. (v2.0 and later). * *SSHA512*: Salted SHA512 sum of the password stored in base64. (v2.0 and later). For some schemes (e.g. PLAIN-MD5, SHA) Dovecot is able to detect if the password hash is base64 or hex encoded, so both can be used.'doveadm pw' anyway generates the passwords using the encoding mentioned above. SHA256 and SHA512 in libc ------------------------- glibc v2.7+ supports SHA256 and SHA512 based password schemes. The passwords look like: * SHA256: $5$salt$data * SHA512: $6$salt$data These passwords are completely different than what Dovecot generates. They use multiple rounds of SHA, so they're also safer against brute forcing (but also requiring more CPU from your server). You can use these simply by using CRYPT scheme, assuming your libc can handle these kinds of passwords. Encoding -------- The base64 vs. hex encoding that is mentioned above is simply the default encoding that is used. You can override it for any scheme by adding a ".hex", ".b64" or ".base64" suffix. For example: * '{SSHA.b64}986H5cS9JcDYQeJd6wKaITMho4M9CrXM' contains the password encoded to base64 (just like {SSHA}) * '{SSHA.HEX}3f5ca6203f8cdaa44d9160575c1ee1d77abcf59ca5f852d1' contains the password encoded to hex This can be especially useful with plaintext passwords to encode characters that would otherwise be illegal. For example in passwd-file you couldn't use a ":" character in the password without encoding it to base64 or hex. For example:'{PLAIN}{\}:!"' is the same as '{PLAIN.b64}e1x9OiEiCg=='. You can also specify the encoding with doveadm pw. For example: 'doveadm pw -s plain.b64' Salting ------- For most of the salted password schemes (SMD5, SSHA*) the salt is stored after the password hash and its length can vary. When hashing the password, append the salt after the plaintext password, e.g.: SSHA256(pass, salt) = SHA256(pass + salt) + salt. For example with SSHA256 you know that the hash itself is 32 bytes (256 bits/8 bits per byte). Everything after that 32 bytes is the salt. For example if you have a password: ---%<------------------------------------------------------------------------- {SSHA256}SoR/78T5q0UPFng8UCXWQxOUKhzrJZlwfNtllAupAeUT+kQv ---%<------------------------------------------------------------------------- After base64 decoding it you'll see that its length is 36 bytes, so the first 32 bytes are the hash and the following 4 bytes are the salt: * length: 'echo SoR/78T5q0UPFng8UCXWQxOUKhzrJZlwfNtllAupAeUT+kQv|base64 -d|wc -c' -> 36 * hash: 'echo SoR/78T5q0UPFng8UCXWQxOUKhzrJZlwfNtllAupAeUT+kQv|base64 -d|dd bs=1 count=32|hexdump -C' -> 4a 84 7f ef c4 f9 ab 45 0f 16 78 3c 50 25 d6 43 13 94 2a 1c eb 25 99 70 7c db 65 94 0b a9 01 e5 * salt: 'echo SoR/78T5q0UPFng8UCXWQxOUKhzrJZlwfNtllAupAeUT+kQv|base64 -d|dd bs=1 skip=32|hexdump -C' -> 13 fa 44 2f Other common hash sizes are: * MD5: 16 bytes * SHA: 20 bytes * SHA256: 32 bytes * SHA512: 64 bytes The web management gui VBoxAdm [http://developer.gauner.org/vboxadm/] has some code dealing with creation and verification of salted hashes in Perl. However not all password schemes provided by dovecotpw are supported. Have a look at the module VBoxAdm::DovecotPW for more details. (This file was created from the wiki on 2011-01-13 04:52) wiki/Design.Indexes.MainIndex.txt000066600000022515150501203730012737 0ustar00Main index ========== The main index can be used to quickly look up messages' UIDs, flags, keywords and extension-specific data, such as cache file or mbox file offsets. Reading, writing and locking ---------------------------- Reading 'dovecot.index' file requires locking, unfortunately. Shared read locking is done using the standard index locking method specified in 'lock_method' setting ('lock_method' parameter for 'mail_index_open()'). Writing to index files requires transaction log to be exclusively locked first. This way the index locking only has to worry about existing read locks. The locking works by first trying to lock the index with the standard locking method, but if it couldn't acquire the lock in two seconds, it'll fallback to copying the index file to a temporary file, and when unlocking it'll 'rename()' the temporary file over the 'dovecot.index' file. Note that this is safe only because of the exclusive transaction log lock. This way the writers are never blocked by readers who are allowed to keep the shared lock as long as they want. The copy-locking is used always when doing anything that could corrupt the index file if it crashed in the middle of an operation. For example if the header or record size changes, or if messages are expunged. New messages can be appended however, because the message count in the header is updated last. Expunging the last messages would probably be safe also (because only the header needs updating), but it's not done currently. The index file should never be directly modified. Everything should go through the transaction log, and the only time the index needs to be write-locked is when transactions are written to it. Currently the index file is updated whenever the backend mailbox is synchronized. This isn't necessary, because an old index file can be updated using the transaction log. In future there could be some smarter decisions about when writing to the index isn't worth the extra disk writes. Header ------ Fields that won't change without recreating the index: major_version: If this doesn't match 'MAIL_INDEX_MAJOR_VERSION', don't try to read the index. Dovecot recreates the index file then. minor_version: If this doesn't match 'MAIL_INDEX_MINOR_VERSION' there are some backwards compatible changes in the index file (typically header fields). Try to preserve the headers and the minor version when updating the index file. base_header_size: Extension headers begin after the base headers. This is normally the same as 'sizeof(struct mail_index_header)'. header_size: Records begin after base and extension headers. record_size: Size of each record and its extensions. Initially the same as 'sizeof(struct mail_index_record)'. compat_flags: Currently there is just one compatibility flag: 'MAIL_INDEX_COMPAT_LITTLE_ENDIAN'. Dovecot doesn't try to bother to read different endianess files, they're simply recreated. indexid: Unique index file ID. This is used to make sure that the main index, transaction log and cache file are all part of the same index. Header flags: MAIL_INDEX_HDR_FLAG_CORRUPTED: Set whenever the index file is found to be corrupted. If the reader notices this flag, it shouldn't try to continue using the index. MAIL_INDEX_HDR_FLAG_HAVE_DIRTY: This index has records with 'MAIL_INDEX_MAIL_FLAG_DIRTY' flag set. MAIL_INDEX_HDR_FLAG_FSCK: Call 'mail_index_fsck()' as soon as possible. This flag isn't actually set anywhere currently. Message UIDs and counters: uid_validity: IMAP UIDVALIDITY field. Initially can be 0, but after it's set we don't currently try to even handle the case of UIDVALIDITY changing. It's done by marking the index file corrupted and recreating it. That's a bit ugly, but typically the UIDVALIDITY never changes. next_uid: UID given to the next appended message. Only increases. messages_count: Number of records in the index file. recent_messages_count: Number of records with 'MAIL_RECENT' flag set. seen_messages_count: Number of records with 'MAIL_SEEN' flag set. deleted_messages_count: Number of records with 'MAIL_DELETED' flag set. first_recent_uid_lowwater: There are no UIDs lower than this with 'MAIL_RECENT' flag set. first_unseen_uid_lowwater: There are no UIDs lower than this *without* 'MAIL_SEEN' flag set. first_deleted_uid_lowwater: There are no UIDs lower than this with 'MAIL_DELETE' flag set. The lowwater fields are used to optimize searching messages with/without a specific flag. Fields related to syncing: log_file_seq: Log file the log_*_offset fields point to. log_file_int_offset, log_file_ext_offset: All the internal/external transactions before this offset in the log file are synced to the index. External transactions are synced more often than internal, so 'log_file_int_offset' <= 'log_file_ext_offset'. sync_size, sync_stamp: Used by the mailbox backends to store their synchronization information. Some day these should be removed and replaced with extension headers. Then there are day fields: day_stamp: UNIX timestamp to the beginning of the day when new records were last added to the index file. day_first_uid[8]: These fields are updated when 'day_stamp' < today. The [0..6] are first moved to [1..7], then [0] is set to the first appended UID. So they contain the first UID of the day for last 8 days when messages were appended. The 'day_first_uid[]' fields are used by cache file compression to decide when to drop 'MAIL_CACHE_DECISION_TEMP' data. Extension headers ----------------- After the base header comes a list of extensions and their headers. The first extension begins from 'mail_index_header.base_header_size' offset. The second begins after the first one's 'data[]' and so on. The extensions always begin 64bit aligned however, so you may need to skip a few bytes always. Read the extensions as long as the offset is smaller than 'mail_index_header.header_size'. ---%<------------------------------------------------------------------------- struct mail_index_ext_header { uint32_t hdr_size; /* size of data[] */ uint32_t reset_id; uint16_t record_offset; uint16_t record_size; uint16_t record_align; uint16_t name_size; /* unsigned char name[name_size] */ /* unsigned char data[hdr_size] (starting 64bit aligned) */ }; ---%<------------------------------------------------------------------------- 'reset_id', record offset, size and alignment is explained in 's 'struct mail_transaction_ext_intro'. Records ------- There are 'hdr.messages_count' records in the file. Each record contains at least two fields: Record UID and flags. The UID is always increasing for the records, so it's possible to find a record by its UID with binary search. The record size is specified by 'mail_index_header.record_size'. The flags are a combination of 'enum mail_flags' and 'enum mail_index_mail_flags'. There exists only one index flag currently: 'MAIL_INDEX_MAIL_FLAG_DIRTY'. If a record has this flag set, it means that the mailbox syncing code should ignore the flag in the mailbox and use the flag in the index file instead. This is used for example with mbox and 'mbox_lazy_writes=yes'. It also allows having modifiable flags for read-only mailboxes. The rest data is stored in record extensions. Keywords -------- The keywords are stored in record extensions, but for better performance and lower disk space usage in transaction logs, they are quite tightly integrated to the index file code. The list of keywords is stored in "keywords" extension header: ---%<------------------------------------------------------------------------- struct mail_index_keyword_header { uint32_t keywords_count; /* struct mail_index_keyword_header_rec[] */ /* char name[][] */ }; struct mail_index_keyword_header_rec { uint32_t unused; /* for backwards compatibility */ uint32_t name_offset; /* relative to beginning of name[] */ }; ---%<------------------------------------------------------------------------- The unused field originally contained 'count' field, but while writing this documentation I noticed it's not actually used anywhere. Apparently it was added there accidentally. It'll be removed in later versions. So there exists 'keywords_count' keywords, each listed in a NUL-terminated string beginning from 'name_offset'. Since crashing in the middle of updating the keywords list pretty much breaks the keywords, adding new keywords causes the index file to be always copied to a temporary file and be replaced. The keywords in the records are stored in a "keywords" extension bitfield. So the nth bit in the bitfield points to the nth keyword listed in the header. It's not currently possible to safely remove existing keywords. Extensions ---------- The extensions only specify their wanted size and alignmentation, the index file syncing code is free to assign any offset inside the record to them. The extensions may be reordered at any time. Dovecot's current extension ordering code works pretty well, but it's not perfect. If the extension size isn't the same as its alignmentation, it may create larger records than necessary. This will be fixed later. The records size is always divisible by the maximum alignmentation requirement. This isn't strictly necessary either, so it could be fixed later as well. (This file was created from the wiki on 2011-01-13 04:52) wiki/ACL.txt000066600000021707150501203730006656 0ustar00Access Control Lists ==================== This page talks mainly about how ACLs work, for more general description of how shared mailboxes work, see . Dovecot v1.0 and v1.1 supports administrator-configured ACL files. v1.2+ supports also IMAP ACL extension, which allows users to change ACLs themselves. The ACL code was written to allow multiple ACL backends, but currently Dovecot supports only virtual ACL files. Note that using ACLs doesn't grant mail processes any extra filesystem permissions that they already don't have. [SharedMailboxes.Permissions.txt] to be able to access the mailboxes. When testing you could first try accessing shared/public mailboxes without ACL plugin even enabled. ACLs can be enabled in dovecot.conf with: ---%<------------------------------------------------------------------------- mail_plugins = acl protocol imap { mail_plugins = $mail_plugins imap_acl } plugin { # Without global ACLs: acl = vfile # With global ACLs in /etc/dovecot/acls/ directory: #acl = vfile:/etc/dovecot/acls } ---%<------------------------------------------------------------------------- ACL groups support works by returning a comma-separated 'acl_groups' [UserDatabase.ExtraFields.txt] from userdb, which contains all the groups the user belongs to. User's UNIX groups have no effect on ACLs (you can "enable" them by using a special [PostLoginScripting.txt]). The default ACL for mailboxes is to give the mailbox owner all permissions and other users none. Mailboxes in public namespaces don't have owners, so by default no one can access them. ACL vfile backend ----------------- vfile backend supports per-mailbox ACLs and global ACLs. Per-mailbox ACLs are stored in 'dovecot-acl' named file, which exists in: * maildir: The Maildir's mail directory (eg. '~/Maildir', '~/Maildir/.folder/') * mbox: Control directory. You should explicitly specify ':CONTROL=' in mail location. * dbox: dbox's mail directory (eg. '~/dbox/INBOX/dbox-Mails/') ACL Inheritance --------------- Every time you create a new mailbox, it gets its ACLs from the parent mailbox. If you're creating a root-level mailbox, it uses the namespace's default ACLs. There is no actual inheritance, however: If you modify parent's ACLs, the child's ACLs stay the same. There is currently no support for ACL inheritance. Namespace's default ACLs are read from "dovecot-acl" file in the namespace's mail root directory (e.g.'/var/public/Maildir'). Note that currently these default ACLs are used only when creating new mailboxes, they aren't used for mailboxes without ACLs. Global ACLs ----------- Global ACLs can be used to apply ACLs globally to all user's specific mailboxes. They are used mainly for two purposes: 1. Removing some permissions from users' personal mailboxes. For example each user might have an "Invoices" mailbox which will be read-only. 2. Giving permissions to master user logins. Master user is not the mailbox owner, so by default it has no permissions to any of the mailboxes. Global ACL directory is specified as a parameter to vfile backend in 'acl' setting ('/etc/dovecot/acls/' in the above example). They are looked up using the mailbox's real name. For example: * INBOX: '/etc/dovecot/acls/INBOX' * archives.2007: '/etc/dovecot/acls/archives.2007' * archives/2007: '/etc/dovecot/acls/archives/2007' The mailbox names are looked up using the real separators, not the virtual ones that can be configured in namespaces. So even if you specify 'separator=/' with Maildir, the ACL files still use the '.' separator, just as the Maildir directory names are also stored using '.'. There is no leading '.' however. There is an extra problem with mailbox formats that use '/' as the separator (e.g. mbox, dbox): For example if you have mailboxes "foo" and "foo/bar" and you wish to give ACLs to both of them, you can't create both '/etc/dovecot/acls/foo' and '/etc/dovecot/acls/foo/bar' files. The 'foo' has to be either a directory or a file, it can't be both. To solve this problem, you can instead create a .DEFAULT file for "foo": * foo: '/etc/dovecot/acls/foo/.DEFAULT' * foo/bar: '/etc/dovecot/acls/foo/bar' The filenames must start with namespace prefix (if it has one). For example with namespace 'prefix=INBOX/' containing mailbox "foo" use '/etc/dovecot/acls/INBOX/foo'. If a mailbox has both global and per-mailbox ACL file, both of them are read and the ACLs are merged. If there are any conflicts, the global ACL file overrides per-mailbox ACL file. This is because users can modify their own per-mailbox ACL files via IMAP ACL extension. Global ACLs can only be modified by administrator, so users shouldn't be able to override them. ACL files --------- The files themselves are in format: ---%<------------------------------------------------------------------------- [:] ---%<------------------------------------------------------------------------- Where *identifier* is one of: * group-override=*group name* * user=*user name* * owner * group=*group name* * authenticated * anyone (or anonymous, which is alias for anyone) The ACLS are processed in the order given above, so for example if you have given read-access to a group, you can still remove that from specific users inside the group. Group-override identifier allows you to override users' ACLs. Probably the most useful reason to do this is to temporarily disable access for some users. For example: ---%<------------------------------------------------------------------------- user=timo rw group-override=tempdisabled ---%<------------------------------------------------------------------------- Now if /timo/ is in /tempdisabled/ group, he has no access to the mailbox. This wouldn't be possible with a normal group identifier, because the 'user=timo' would override it. The currently supported ACLs and their corresponding named ACLs are: +---+---------------+---------------------------------------------------------+ | l | lookup | Mailbox is visible in mailbox list. Mailbox can be | | | | subscribed to. | +---+---------------+---------------------------------------------------------+ | r | read | Mailbox can be opened for reading. | +---+---------------+---------------------------------------------------------+ | w | write | Message flags and keywords can be changed, except \Seen | | | | and \Deleted | +---+---------------+---------------------------------------------------------+ | s | write-seen | \Seen flag can be changed | +---+---------------+---------------------------------------------------------+ | t | write-deleted | \Deleted flag can be changed | +---+---------------+---------------------------------------------------------+ | i | insert | Messages can be written or copied to the mailbox | +---+---------------+---------------------------------------------------------+ | p | post | Messages can be posted to the mailbox by , e.g.| | | | from [Pigeonhole.Sieve.txt] scripts | +---+---------------+---------------------------------------------------------+ | e | expunge | Messages can be expunged | +---+---------------+---------------------------------------------------------+ | k | create | Mailboxes can be created (or renamed) directly under | | | | this mailbox (but not necessarily under its children, | | | | see ACL Inheritance section above) (renaming also | | | | requires delete rights) | +---+---------------+---------------------------------------------------------+ | x | delete | Mailbox can be deleted | +---+---------------+---------------------------------------------------------+ | a | admin | Administration rights to the mailbox (currently: ability| | | | to change ACLs for mailbox) | +---+---------------+---------------------------------------------------------+ The ACLs are compatible with RFC 4314 (IMAP ACL extension, updated version). Unknown ACL letters are complained about, but unknown named ACLs are ignored. Named ACLs are mostly intended for future extensions. Example ACL file: ---%<------------------------------------------------------------------------- owner lrwstipekxa user=timo rl ---%<------------------------------------------------------------------------- List cache ---------- 'dovecot-acl-list' file lists all mailboxes that have "l" rights assigned. If you manually add/edit 'dovecot-acl' files, you may need to delete the 'dovecot-acl-list' to get the mailboxes visible. (This file was created from the wiki on 2011-01-13 04:52) wiki/Quota.Dict.txt000066600000011745150501203730010233 0ustar00Dictionary quota ================ The /dictionary/ quota backend supports both *storage* and *messages* quota limits. The current quota is kept in a dictionary. The available dictionaries are: * MySQL * PostgreSQL * Flat file The quota root format is: ---%<------------------------------------------------------------------------- quota = dict::[: