Configuration File

• Table of Contents
• Overview
  • String Encoding
  • Boolean Encoding
  • Duration Encoding
• Grouping Contexts
• Configuration Directives
  • Performance Tuning
    • GreenArrow Studio
    • HTTP Server
    • Database Server
    • System Resources
  • SMTP Delivery
    • DNS
    • Automatic Back-off
    • Outgoing Connections
    • Encryption
    • Overrides
    • Error Handling
    • DKIM
  • Logging
    • DNS
    • SMTP
  • VirtualMTAs
• Syntax Checker
• Reload Configuration in Active Services
  • Detecting and Correcting Invalid Configuration
• Dynamic Defaults
• Review Per-Domain Configuration
• Example File

Overview

GreenArrow Engine originally used multiple configuration files, one for each setting in the /var/hvmail/control directory. Now, as new features are added, new configuration parameters are added as directives in /var/hvmail/control/greenarrow.conf. Eventually, all existing GreenArrow configuration files will be replaced with directives in greenarrow.conf, and it will serve as a single master configuration file.

The greenarrow.conf configuration file is a plain-text file (using UTF-8) inspired by the NGINX configuration file format.

A single directive inside of a grouping block looks like this:

general {
  system_max_smtp_connections 5000
}

Some blocks have parameters. For example, this applies a configuration to the “yahoo.com” domain name:

domain yahoo.com {
  reuse_connections yes
}

All lines whose first non-whitespace character is a pound sign (#) are treated as comments.

String Encoding

String values that consist of only letters (a-zA-Z), numbers (0-9), dashes (-), underscores (_), plus signs (+), periods (.), asterisks (*), slashes (/), backslashes (\), square brackets ([ and ]), dollar signs ($), colons (:), semicolons (;), and percentages (%) do not require quotation.

All other string values must be surrounded by quotation marks (") and may contain the same escape codes as JSON string values.

The escape codes are:

The following are all valid string values:

example
example.com
other-example123
other_example.900
5000.12
09000
"example with spaces"
"example\nwith\ncontrol\codes\n"

Boolean Encoding

Booleans may be specified as yes, true, no, and false.

Duration Encoding

A time interval may be specified as a sequence of decimal numbers, each with a unit suffix. For example 25h, 300s, or 4h15m. Valid suffixes are s, m, and h.

Grouping Contexts

The greenarrow.conf configuration file is organized hierarchically as described below:

This top-level group contains settings that do not belong in any other group.

This grouping contains settings that are specific to one or more IP addresses.

This configuration grouping does not create an IP address VirtualMTA unless define_virtual_mtas_in_config_file is true. Rather it applies settings to existing IP address VirtualMTAs and the ip_address_name must case-insensitively match the name of an IP address VirtualMTA configured through the web-interface or API.

The special IP address of a single asterisk (*) means that this context block defines settings that apply to all IP addresses in the system. Records defined for a specific IP address name will take precedence over records defined for the wildcard * IP address.

For example:

ip_address smtp2, smtp3 {
  ...
}

And:

ip_address * {
  ...
}

This grouping contains settings that are specific to one or more SMTP relay servers.

This configuration grouping does not create a relay server unless define_virtual_mtas_in_config_file is true. Rather it applies settings to existing relay server VirtualMTAs and the relay_server_name must case-insensitively match the name of a relay server configured through the web-interface or API.

For example:

relay_server esp1, esp2 {
  ...
}

Wildcard (*) relay server configurations are not supported.

This context exists within an IP address (including the * wildcard entry). It contains settings for one or more remote domains. Specifying an * indicates that the settings shall be used as the default for any remote domain that does not otherwise contain a particular setting.

Domains can be defined with the following wildcards:

• A domain that is prefixed with [*.] will match the given domain and all subdomains. For example, [*.]example.com will match example.com, subdomain.example.com, and www.subdomain.example.com.
• A domain that is prefixed with *. will match only subdomains. For example, *.example.com will match subdomain.example.com and www.subdomain.example.com, but will not match example.com.

Settings to use for a domain are checked in this order. The first match is used:

1. An exact IP name match and an exact domain match
2. An exact IP name match and the wildcard domain entry
3. The wildcard IP entry and an exact domain match
4. The wildcard IP entry and the wildcard domain entry

For example, when looking up delivery to foo.example.com domain strings are searched for in the following order:

foo.example.com
[*.]foo.example.com
*.example.com
[*.]example.com
*.com
[*.]com
*

All name or domain matches are case-insensitive.

If define_virtual_mtas_in_config_file is enabled, the wildcard * may not be combined with any other domain names or wildcards.

ip_address ipaddr-1, ipaddr-2 {
  domain yahoo.com, ymail.com {
    ...
  }
}
ip_address * {
  domain hotmail.com {
    ...
  }
  domain gmail.com, googlemail.com {
    ...
  }
  domain * {
    ...
  }
}

This context defines a Routing Rule.

The define_virtual_mtas_in_config_file directive must be enabled for this context to take effect.

routing_rule my-routing-rule-name {
  ...
}

This context defines a domain rule within a Routing Rule.

The define_virtual_mtas_in_config_file directive must be enabled for this context to take effect.

Each routing_rule is required to have at least a routing_rule_domain for the * domain.

The wildcard * may not be combined with any other domain names or wildcards.

routing_rule my-routing-rule-name {
  routing_rule_domain first.example.com, second.example.com, third.example.com {
    routing_rule_destination foo
  }
  routing_rule_domain alpha.example.com, beta.example.com {
    routing_rule_destination bar
  }
  routing_rule_domain * {
    routing_rule_destination baz
  }
  ...
}

This context is for configuration directives that apply to an email based off of the VirtualMTA name used to inject the message. This is the VirtualMTA name provided in the X-GreenArrow-VirtualMTA header or the GREENARROW_MTAID environment variable. This is not necessarily the same as the IP Address or Relay Server used to actually deliver the message.

For example, in the case where:

• An email is injected with the VirtualMTA name of shared_pool
• shared_pool is a Routing Rule that splits email through ip_a and ip_b
• The actual email delivery is done on ip_a

These settings will apply:

• virtual_mta_injected "shared_pool" { }
  • This matches the injected VirtualMTA name of shared_pool.
• virtual_mta_injected "*" { }
  • This is the default and, unless overridden by a more precise directive, applies to all messages.
• virtual_mta "ip_a" { }
  • This is the IP on which the email was delivered.

These settings will not apply:

• virtual_mta_injected "ip_a" { }
  • The message was injected on shared_pool, not ip_a.
• virtual_mta "shared_pool" { }
  • The message was delivered on ip_a, not shared_pool.

All name matching is done case-insensitivity.

If email is injected using the numeric primary key of an IP Address, Relay Server, or Routing Rule, then a virtual_mta_injected block with a name matching the name of the IP Address, Relay Server, or Routing Rule will be used.

virtual_mta_injected smtp-1, smtp-2 {
  ...
}

Configuration Directives

Directives are configuration settings that apply a specific option within a context.

This directive creates a macro that may be referenced from the strings/domains provided to a domain directive. See the example below to see how they are used.

domain_macro google gmail.com, googlemail.com
domain_macro microsoft msn, hotmail
domain_macro yahoo yahoo, ymail, rocketmail
domain_macro tlds com, net
ip_address * {
  domain $google {
    ...
  }
  domain $microsoft.com {
    ...
  }
  domain $yahoo.$tlds {
    ...
  }
}

The above example expands to the equivalent configuration below.

ip_address * {
  domain gmail.com, googlemail.com {
    ...
  }
  domain msn.com, hotmail.com {
    ...
  }
  domain yahoo.com, yahoo.net, ymail.com, ymail.net, rocketmail.com, rocketmail.net {
    ...
  }
}

Domain macros must be defined before they are used. If a macro’s definition is changed later in a configuration file, the new definition is used for subsequent directives.

This directive includes the given file to be parsed as part of the configuration file. This is useful for separating complex configurations into multiple files.

The filename given must be an absolute path (i.e. it must begin with /).

This directive can only be specified at the top-level of a file.

include /usr/local/etc/ga/ip_addresses.conf
include /usr/local/etc/ga/general.conf

Performance Tuning

GreenArrow Studio

The settings in this section pertain to GreenArrow Studio.

The number of message generation processes each campaign should start. Increasing this setting is advisable for systems that tend to send single, large campaigns.

For systems that are sending multiple simultaneous campaigns, this should remain at its default.

general {
  studio_message_generation_processes_per_campaign 2
}

The number of “shared” workers that are created to send campaigns. If there are no campaigns to send, these workers can also perform imports, exports, and other background jobs – excluding transactional email. Using this along with zero values for studio_workers_general and studio_workers_imports means that if there are enough campaigns running, all imports/exports and general tasks will have to wait until the campaigns are done – which is not desirable.

We only recommend using shared workers on very low memory systems, because it is better to have a dedicated number of workers for each type of task.

general {
  studio_workers_shared 2
}

The number of workers that are created exclusively to send campaigns. This should generally be used in favor of studio_workers_shared.

general {
  studio_workers_campaigns 2
}

The number of workers that are created to run other background jobs. These workers handle most background activity in GreenArrow Studio. The things that are not handled by these workers include imports, exports, transactional email, and campaigns.

Examples of general tasks include recording a click to the database and calculating the estimated number of recipients in a campaign.

general {
  studio_workers_general 2
}

The number of workers that are created to handle imports and exports.

general {
  studio_workers_imports 2
}

The number of workers that are created to deliver transactional email - this includes autoresponders, campaign previews, web form confirmation, and any other non-campaign emails sent from GreenArrow Studio.

general {
  studio_workers_transactional 2
}

HTTP Server

The settings in this section are related to the Apache HTTP Server and the Passenger application server.

This sets the PassengerMaxRequestQueueSize directive in Passenger, the maximum number of requests that can sit in Passenger’s queue waiting for an application instance.

To apply changes to this directive, run greenarrow_config reload_apache.

general {
  passenger_max_requests_in_queue 2
}

This sets the PassengerMinInstances directive in Passenger, the minimum number of instances of the application server which handles HTTP/HTTPS requests that should be kept running at any time.

To apply changes to this directive, run greenarrow_config reload_apache.

general {
  passenger_min_application_instances 2
}

This sets the PassengerMaxPoolSize directive in Passenger, the maximum number of instances of the application server which handles HTTP/HTTPS requests that should be kept running at any time. On servers that are not low on memory, we recommend keeping the minimum and maximum equal.

To apply changes to this directive, run greenarrow_config reload_apache.

general {
  passenger_max_application_instances 2
}

This sets the MaxClients and ServerLimit directives in Apache. These limit the number of HTTP/HTTPS sessions that Apache can handle simultaneously.

To apply changes to this directive, run greenarrow_config reload_apache.

general {
  apache_max_clients 2
}

Database Server

The maximum number of concurrent connections allowed to be made to the PostgreSQL database server.

To apply changes to this directive, run greenarrow_config reload_postgres.

general {
  postgres_max_connections 200
}

By default, the following equation is used, rounded up to the next 100:

1.25 * (
  100 +
  (6 + studio_message_generation_processes_per_campaign) *
  (studio_workers_shared + studio_workers_campaigns) + 2 *
  (studio_workers_general + studio_workers_imports + studio_workers_transactional) +
  passenger_max_application_instances
)

System Resources

The system resources directives are normally only used to decrease GreenArrow’s view of what resources are available. For example, it could make sense to do that if you’re running non-GreenArrow applications on the same server.

This directive overrides GreenArrow’s view of how much RAM exists in this system. This causes dynamic defaults to be set based upon this amount of RAM, instead of the actual amount of RAM.

This does not cause GreenArrow to use more or less RAM, except as an effect of changed dynamic defaults.

general {
  system_memory_gigabytes 8.5
}

This directive overrides GreenArrow’s view of how many CPU cores exist in this system. This causes dynamic defaults to be set based upon this number of CPU cores, instead of the actual number of CPU cores.

This does not cause GreenArrow to use more or less CPU, except as an effect of changed dynamic defaults.

general {
  system_cpu_core_count 4
}

SMTP Delivery

The settings in this section control how GreenArrow delivers email via SMTP.

DNS

The maximum number of concurrent DNS queries that may be executed. It’s extremely uncommon that this value should be changed. Set this to 0 to have no limit.

general {
  max_dns_queries 100
}

Automatic Back-off

Whenever a throttle program begins or ends automatic back-off, these email addresses will receive a notification.

general {
  notify_on_backoff "[email protected]", "[email protected]"
}

The email message when backoff mode begins will look like:

Backoff mode started

Backoff mode will last until 2018-03-21 12:39:07 CDT.

This throttle:

    https://example.com/ga/eui/virtual_mta/eng_throttles/45

IP Address:

    127.0.0.101
    ip-1.example.com
    https://example.com/ga/eui/virtual_mta/ip_addresses/74

Domains:

    first.example.com
    second.example.com
    example.com

Logged information:

    backoff mode began
    failure rate (60/120; 50.00%) exceeds limit of 20%
    combined failure and deferral rate (60/120 failures; 0/120 deferrals; 50.00%) exceeds deferral limit of 30%

The email message when backoff mode ends will look like:

Backoff mode ended

This throttle:

    https://example.com/ga/eui/virtual_mta/eng_throttles/45

IP Address:

    127.0.0.101
    ip-1.example.com
    https://example.com/ga/eui/virtual_mta/ip_addresses/74

Domains:

    first.example.com
    second.example.com
    example.com

Logged information:

    backoff mode ended

Outgoing Connections

The maximum number of concurrent SMTP connections that may be established. Set this to 0 to have no limit.

general {
  system_max_smtp_connections 10000
}

By default, when sending emails, GreenArrow Engine attempts to deliver one message per SMTP session. This means that if an email provider imposes a limit on how many messages they will accept per SMTP session, you don’t need to worry about exceeding that limitation.

For high volume email recipients, you can choose to enable connection reuse to send multiple messages in a single SMTP session. This can offer a significant performance boost when applied to your most common recipient domains.

With this setting enabled, connections will be used so long as the previous delivery attempt succeeded. When a delivery attempt ends in any kind of non-acceptance (deferral, failure, or connection error), the connection is closed.

See reuse_connections_timeout and reuse_connections_max_messages to adjust for how long connections are reused.

# Reuse connections to Google on all IP addresses.

ip_address * {
  domain gmail.com, googlemail.com {
    reuse_connections yes
  }
}

The duration for which the connection should be held open without any delivery attempts.

This value is ignored unless reuse_connections is set to yes.

# Reuse connections on "ip-addr-3" to all domains, allowing the connection to
# idle for up to 2 seconds between deliveries.

ip_address ip-addr-3 {
  domain * {
    reuse_connections yes
    reuse_connections_timeout 2s
  }
}

The maximum number of deliveries that should be attempted on a single open connection before closing.

This value is ignored unless reuse_connections is set to yes.

# Reuse connections on all IPs to all Domains, with a maximum of 500 messages
# per connection.

ip_address * {
  domain * {
    reuse_connections yes
    reuse_connections_max_messages 500
  }
}

Encryption

When this directive is enabled for a domain (including the * wildcard domain), GreenArrow will issue the STARTTLS command to servers that offer it.

If the server replies to STARTTLS with an error, the existing connection is used without TLS - unless starttls_require is enabled.

# Override the default to always try to use STARTTLS if it's available.
ip_address * {
  domain * {
    starttls_use yes
  }
}

# However - on this IP, we don't want to use STARTTLS to Gmail.
ip_address smtp-7 {
  domain gmail.com, googlemail.com {
    starttls_use no
  }
}

When this directive and starttls_use are both enabled, email will not be delivered unless a STARTTLS connection can be established:

• If the STARTTLS command is not offered by the remote server, the type of response is determined by the starttls_require_action directive.

• If an error occurs while establishing a STARTTLS connection, a temporary failure is returned. This differs from GreenArrow’s standard behavior, which would result in the message being delivered without TLS.

# Override the default to always try to use STARTTLS if it's available.
ip_address * {
  domain * {
    starttls_use yes
  }

  # Don't deliver to Google unless we're in TLS.
  domain gmail.com, googlemail.com {
    starttls_require yes
  }
}

When starttls_require is enabled, this directive determines what is done if STARTTLS is unavailable or unsuccessful.

Connected to {IP} but STARTTLS is not available, delivery attempt not made. (#5.7.10)

Connected to {IP} but STARTTLS is not available, delivery attempt not made. (#4.7.10)

Connected to {IP} but STARTTLS is not available, delivery attempt not made. (#2.7.10)

# Override the default to always try to use STARTTLS if it's available.
ip_address * {
  domain * {
    starttls_use yes
  }
}

# Discard email for Yahoo that can't be sent with TLS.
ip_address * {
  domain yahoo.com {
    starttls_require yes
    starttls_require_action discard
  }
}

Overrides

Override the delivery.

Delivery attempt prevented by local configuration. (#5.7.1)

Delivery attempt prevented by local configuration. (#4.7.1)

This message was discarded by local configuration. (#2.7.1)

# Discard all email for Yahoo. Do not attempt delivery.
ip_address * {
  domain yahoo.com {
    delivery_override discard
  }
}

Override the destination to which email is delivered. The argument may include a port number following a colon. If the port number is not specified, port 25 is assumed.

Formerly, this was accomplished using the /var/hvmail/control/smtproutes file. If that file exists, it is used at a lower precedence than this smtp_route directive.

ip_address * {
  # Deliver all Yahoo mail to localhost port 2500.
  domain yahoo.com {
    smtp_route 127.0.0.1:2500
  }

  # Deliver all Microsoft mail to Google??? That's weird...
  domain msn.com, hotmail.com {
    smtp_route smtp.googlemail.com:25
  }
}

Error Handling

Define what the delivery result should be when the delivery attempt has a connection failure (at either the TCP or SSL layer) or a timeout while the message is transferring. This directive applies until the . has been sent, at which point the message_transfer_response_timeout_action directive applies.

Connected to {IP} but connection died. (#5.4.2)

Connected to {IP} but connection died. (#4.4.2)

Connected to {IP} but connection died. (#2.4.2)

ip_address * {
  # Message transfer timeouts shouldn't be retried.
  domain yahoo.com {
    message_transfer_timeout_action perm_failure
  }
}

Define what the delivery result should be when the delivery attempt has a connection failure (at either the TCP or SSL layer) or a timeout after the . has been sent.

This is the critical period where a timeout or disconnection can cause a duplicate message delivery.

Connected to {IP} but connection died. Possible duplicate! (#5.4.2)

Connected to {IP} but connection died. Possible duplicate! (#4.4.2)

Connected to {IP} but connection died. Possible duplicate! (#2.4.2)

ip_address * {
  domain * {
    message_transfer_response_timeout_action discard
  }
}

DKIM

This directive is to provide compatibility with the PowerMTA configuration directive domain-key to aid in migrating from PowerMTA to GreenArrow.

This specifies DKIM private keys used by the PowerMTA compatibility DKIM signing system. DKIM keys specified with this directive are not available to be used by the X-GreenArrow-DKIM header.

This directive can be specified multiple times to provide multiple DKIM keys that could apply.

What domain name may be used in the signature depends on the pcompat_dkim_identity:

If there IS NOT a pcompat_dkim_identity value defined, then a DKIM key matching one of these domain names is searched for. The first match is used:

• The domain name specified in a Sender header, if present
• The domain name specified in the From header

If there IS a pcompat_dkim_identity value defined, then a DKIM key matching this domain name is searched for:

• The domain name specified in pcompat_dkim_identity
  • If found, then the pcompat_dkim_identity domain name is used as the i= parameter in the signature, and the domain name in the matched pcompat_dkim_key record is used as the d= parameter (except for a * domain, where the pcompat_dkim_identity domain name will be used as the d= parameter)

Searching for a DKIM key that matches a domain name is done as follows:

• Keys defined in an exact-match virtual_mta_injected are searched before keys in a wildcard-matching (*) virtual_mta_injected block.

• Inside of a matching virtual_mta_injected block, the first key (in the order specified in the configuration file) that contains a domain name matching one of these criteria:

  • Equal to the “search domain name”
  • A parent domain name of the “search domain name” – the domain name specified with DKIM key will be used in the DKIM signature
  • A domain of * – the “search domain name” will be used in the DKIM signature

Note: because domain names are matched in the order specified in the configuration file, any pcompat_dkim_key record with a domain name of * should be the last pcompat_dkim_key record in the containing virtual_mta_injected block.

virtual_mta_injected ipaddr-1 {
  pcompat_dkim_key selector example.com /path/to/private_key_filename.pem
}

This is to provide compatibility with the PowerMTA configuration directive dkim-sign to aid in migrating from PowerMTA to GreenArrow.

If enabled, the PowerMTA compatibility DKIM signing system will add a DKIM signature to messages at the time of injection - if a matching DKIM key is found as described in pcompat_dkim_key.

Caveat - The GreenArrow PowerMTA compatibility DKIM signing system signs messages at time of injection. It appears that PowerMTA signs messages at time of delivery.

virtual_mta_injected * {
  pcompat_dkim_sign yes
}

This is to provide compatibility with the PowerMTA configuration directive dkim-identity to aid in migrating from PowerMTA to GreenArrow.

This is used to DKIM sign an email with a domain name that is not the Sender or From header.

For a description of how this is used, see the documentation of the pcompat_dkim_key directive.

virtual_mta_injected * {
  pcompat_dkim_identity example.com
}

Logging

GreenArrow log files are automatically rotated and contain timestamps in external TAI64 format. Refer to the Service Logs documentation for more information on individual log files.

DNS

Log basic information about DNS lookups performed while GreenArrow delivers mail. This is logged to /var/hvmail/log/rspawn-limiter/current. This does not log when GreenArrow’s internal DNS cache is consulted.

greenarrow-remote.6241: dns MX lookup of mail.drh.net resulted in ["mail.drh.net. (10)"]; 0.32ms elapsed
greenarrow-remote.6241: dns A lookup of mail.drh.net. resulted in ["207.99.125.72"]; 1.04ms elapsed

When doing a DNS lookup, GreenArrow Engine will retry a DNS query if it does not receive an answer within a timeout period. Each log line represents the entire process of doing the DNS resolution, so (a) you will not see a separate line for a timeout, and (b) the time spent on timeouts and retries will be included in the elapsed time logged.

ip_address * {
  domain hotmail.com, msn.com {
    log_dns yes
  }
}

SMTP

Log information about SMTP connections being opened and closed. This is logged to /var/hvmail/log/rspawn-limiter/current.

(123) attempting to deliver message: recipient=<[email protected]> sender=<[email protected]> mtaid=<0> sendid=<DUNNO> listid=<DUNNO> local_ip_address=<207.99.125.72>
(123) attempting to connect: domain=<mail.drh.net.> ip=<207.99.125.72>
(123) connected: domain=<mail.drh.net.> ip=<207.99.125.72>
(123) connection closed

The prefix (123) is an identifier for the connection. These identifiers are reused.

ip_address * {
  domain hotmail.com, msn.com {
    log_smtp_connections yes
  }
}

Log the SMTP commands and responses exchanged with the remote SMTP server (along with information about connections being opened/closed). This does not include message data. This is logged to /var/hvmail/log/rspawn-limiter/current.

(123) attempting to deliver message: recipient=<[email protected]> sender=<[email protected]> mtaid=<3> sendid=<a512> listid=<a2> local_ip_address=<default>
(123) attempting to connect: domain=<mail.drh.net> ip=<207.99.125.72>
(123) connected: domain=<mail.drh.net> ip=<207.99.125.72>
(123) <<< 220 mail.drh.net ESMTP
(123) >>> EHLO localhost
(123) <<< 250-mail.drh.net
(123) <<< 250-AUTH LOGIN PLAIN
(123) <<< 250-AUTH=LOGIN PLAIN
(123) <<< 250-PIPELINING
(123) <<< 250 8BITMIME
(123) >>> MAIL FROM:<[email protected]> BODY=8BITMIME
(123) <<< 250 ok
(123) >>> RCPT TO:<[email protected]>
(123) <<< 553 sorry, that domain isn't in my list of allowed rcpthosts (#5.7.1)
(123) >>> QUIT
(123) <<< 221 mail.drh.net
(123) connection closed

The prefix (123) is an identifier for the connection. These identifiers are reused.

ip_address * {
  domain hotmail.com, msn.com {
    log_smtp_commands yes
  }
}

Log the raw bytes exchanged with the remote SMTP server in a hex-dump format (along with information about connections being opened/closed). This includes message data. This is logged to /var/hvmail/log/rspawn-limiter/current.

(123) attempting to deliver message: recipient=<[email protected]> sender=<[email protected]> mtaid=<3> sendid=<a512> listid=<a2> local_ip_address=<default>
(123) attempting to connect: domain=<mail.drh.net> ip=<207.99.125.72>
(123) connected: domain=<mail.drh.net> ip=<207.99.125.72>
(123) <<< received 24 bytes
(123) 32 32 30 20 6d 61 69 6c  2e 64 72 68 2e 6e 65 74  |220 mail.drh.net|
(123) 20 45 53 4d 54 50 0d 0a                           | ESMTP..|
(123) >>> sent 16 bytes
(123) 45 48 4c 4f 20 6c 6f 63  61 6c 68 6f 73 74 0d 0a  |EHLO localhost..|
(123) <<< received 92 bytes
(123) 32 35 30 2d 6d 61 69 6c  2e 64 72 68 2e 6e 65 74  |250-mail.drh.net|
(123) 0d 0a 32 35 30 2d 41 55  54 48 20 4c 4f 47 49 4e  |..250-AUTH LOGIN|
(123) 20 50 4c 41 49 4e 0d 0a  32 35 30 2d 41 55 54 48  | PLAIN..250-AUTH|
(123) 3d 4c 4f 47 49 4e 20 50  4c 41 49 4e 0d 0a 32 35  |=LOGIN PLAIN..25|
(123) 30 2d 50 49 50 45 4c 49  4e 49 4e 47 0d 0a 32 35  |0-PIPELINING..25|
(123) 30 20 38 42 49 54 4d 49  4d 45 0d 0a              |0 8BITMIME..|
(123) >>> sent 46 bytes
(123) 4d 41 49 4c 20 46 52 4f  4d 3a 3c 73 65 6e 64 65  |MAIL FROM:<sende|
(123) 72 40 65 78 61 6d 70 6c  65 2e 63 6f 6d 3e 20 42  |[email protected]> B|
(123) 4f 44 59 3d 38 42 49 54  4d 49 4d 45 0d 0a        |ODY=8BITMIME..|
(123) <<< received 8 bytes
(123) 32 35 30 20 6f 6b 0d 0a                           |250 ok..|
(123) >>> sent 28 bytes
(123) 52 43 50 54 20 54 4f 3a  3c 75 73 65 72 40 65 78  |RCPT TO:<user@ex|
(123) 61 6d 70 6c 65 2e 63 6f  6d 3e 0d 0a              |ample.com>..|
(123) <<< received 71 bytes
(123) 35 35 33 20 73 6f 72 72  79 2c 20 74 68 61 74 20  |553 sorry, that |
(123) 64 6f 6d 61 69 6e 20 69  73 6e 27 74 20 69 6e 20  |domain isn't in |
(123) 6d 79 20 6c 69 73 74 20  6f 66 20 61 6c 6c 6f 77  |my list of allow|
(123) 65 64 20 72 63 70 74 68  6f 73 74 73 20 28 23 35  |ed rcpthosts (#5|
(123) 2e 37 2e 31 29 0d 0a                              |.7.1)..|
(123) >>> sent 6 bytes
(123) 51 55 49 54 0d 0a                                 |QUIT..|
(123) <<< received 18 bytes
(123) 32 32 31 20 6d 61 69 6c  2e 64 72 68 2e 6e 65 74  |221 mail.drh.net|
(123) 0d 0a                                             |..|
(123) connection closed

The prefix (123) is an identifier for the connection. These identifiers are reused.

ip_address * {
  domain hotmail.com, msn.com {
    log_smtp_hexdump yes
  }
}

VirtualMTAs

When this directive is turned on, IP Addresses, Routing Rules, and Relay Servers in GreenArrow Engine’s internal database will be replaced with those that are specified in the configuration file.

If this directive is enabled, default_virtual_mta must be specified.

general {
  define_virtual_mtas_in_config_file yes
}

The VirtualMTA that will be used for delivery when none is specified.

This directive only takes effect if define_virtual_mtas_in_config_file is turned on. If it is, this directive must be specified.

general {
  default_virtual_mta smtp1-1
}

The hostname that will be used for outgoing SMTP connections on this VirtualMTA.

This may not be specified on the * wildcard IP address.

This directive only takes effect if define_virtual_mtas_in_config_file is turned on. If it is, it then this directive must be specified.

ip_address ip-address-1 {
  smtp_source_hostname example.com
}

The IP address that will be used for outgoing SMTP connections on this VirtualMTA.

This may not be specified on the * wildcard IP address.

This directive only takes effect if define_virtual_mtas_in_config_file is turned on. If it is, it then this directive must be specified for each ip_address and relay_server.

ip_address ip-address-1 {
  smtp_source_ip 127.0.0.101
}

These optional credentials will be used as authentication for outgoing SMTP connections on this VirtualMTA.

This directive only takes effect if define_virtual_mtas_in_config_file is turned on.

relay-server my-relay-1 {
  smtp_auth my-username my-password
}

In a domain block inside of an ip_address block this specifies the maximum number of total concurrent connections allowed to these domains from this IP (as a group). If the domain block is for *, then this specifies the maximum number of concurrent connections allowed to any individual domain that does not have a more-specific rule.

In a relay_server block this specifies the maximum number of total concurrent connections allowed by the relay server.

This may be set to zero for no limit.

ip_address ip-address-1 {
  # In the * wildcard case, any individual domain may have a total of 100 concurrent connections.
  domain * {
    max_concurrent_connections 100
  }
  # In this case, the two specified domains can jointly have a total of 500 concurrent connections.
  domain first.example.com, second.example.com {
    max_concurrent_connections 500
  }
}

The maximum number of delivery attempts per unit of time allowed.

The denominator may be sec, s, min, m, hr, or h.

In a domain block inside of an ip_address block this specifies the rate allowed allowed to these domains from this IP (as a group). If the domain block is for * , then this specifies the rate allowed to any individual domain that does not have a more-specific rule.

In a relay_server block this specifies the rate of delivery attempts allowed by the relay server.

ip_address ip-address-1 {
  # In the * wildcard case, any individual domain may send a total of 250 messages per hour.
  domain * {
    max_delivery_rate 250/hr
  }
  # In this case, the two specified domains can jointly send a total of 20 messages per second (72,000 messages per hour).
  domain first.example.com, second.example.com {
    max_delivery_rate 20/sec
  }
}

The Throttle Program that should be used on the given domains for this VirtualMTA. The program will be found by (case-insensitive) name.

This directive only takes effect if define_virtual_mtas_in_config_file is turned on.

ip_address ip-address-1 {
  domain * {
    throttle_program "automatic backoff"
  }
}

Please be aware that the above example of a throttle_program on a * domain does not set a throttle_program on “all domains not otherwise specified”. Instead, this provides a default value of throttle_program to all domain groups that contain throttling directives (max_concurrent_connections and max_delivery_rate). This is because Throttle Programs only operate on explicitly listed domain names.

The destination to which email should be delivered by this VirtualMTA. If the percentage is not specified for one or more destinations, it will be apportioned evenly from whatever is remaining from 100.

Each routing_rule_domain is required to have at least one child routing_rule_destination.

This directive only takes effect if define_virtual_mtas_in_config_file is turned on.

Limit on the number of destinations: We do not recommend adding or editing more than 10,000 destinations. While GreenArrow does not enforce a hard limit on the number of destinations, performance can suffer when exceeding this threshold.

routing_rule proxy-to-other-servers {
  routing_rule_domain * {
    # fastest-server will receive 50% of the messages
    routing_rule_destination fastest-server 50%
    # The remaining three servers will receive 16.66% of the messages each
    routing_rule_destination slower-server-1
    routing_rule_destination slower-server-2
    routing_rule_destination slower-server-3
  }
}

The method of randomization to the destinations. See the Routing Rule documentation for a more detailed explanation. The randomization type can be one of random, message_constant, or email_address_constant.

This directive only takes effect if define_virtual_mtas_in_config_file is turned on.

routing_rule proxy-to-other-servers {
  routing_rule_domain * {
    # fastest-server will receive 50% of the messages
    routing_rule_destination fastest-server 50%
    # The remaining three servers will receive 16.66% of the messages each
    routing_rule_destination slower-server-1
    routing_rule_destination slower-server-2
    routing_rule_destination slower-server-3
    randomization_type message_constant
  }
}

Syntax Checker

To check if your configuration file is valid without applying its settings, run the following.

# greenarrow_config validate
No errors found. Configuration file is acceptable.

This command will let you know that the syntax of your configuration file is correct and warn you of any unknown directives.

Reload Configuration in Active Services

Anytime a service starts up, it will use whatever configuration is currently in /var/hvmail/control/greenarrow.conf. If the configuration cannot be successfully loaded, the service will not start. See the Detecting and Correcting Invalid Configuration section below for more information on troubleshooting an invalid configuration file.

To apply an update to the configuration file, there are reload commands listed below. Combined, these commands will apply all of the configuration in the configuration file.

Update the configuration running on services that do not require restarts

This command will load the new configuration into services that do not require restarts or result in downtime.

This applies most directives in the configuration file. If a directive is not applied by this reload command, it will say so explicitly in the directive’s documentation.

greenarrow_config reload

Update Apache’s configuration

This command will reload Apache with the new configuration. This results in a delay (15-60 seconds depending on server speed) of servicing web requests.

greenarrow_config reload_apache

Running this command without the --force argument will require confirmation by typing y when asked.

Update PostgreSQL’s configuration

This command will reload PostgreSQL with the new configuration. This is considered a HIGH IMPACT operation and will severely disrupt all running services. This operation typically takes 60 seconds.

greenarrow_config reload_postgres

Running this command without the --force argument will require confirmation by typing y when asked.

Detecting and Correcting Invalid Configuration

If a service cannot start due to an invalid greenarrow.conf, this is how you can diagnose the problem.

Run hvmail_init status. When the configuration file is invalid, it will say so at the top of the output.

# hvmail_init status


!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

Errors found in configuration file (/var/hvmail/control/greenarrow.conf):
 - line 26: unknown identifier invalid_grouping_directive

This will prevent multiple services from being able to start or restart.

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!


     QUEUE      SERVICE STARTED       OK FILE EXISTS       RAMDISK ACTIVE
       ram                  yes                  yes                  yes
    bounce                  yes                  yes                  yes
      disk                  yes                  n/a                  n/a

LICENSE: current

SERVICES:
Name                      Primary Service Status  Log Service Status
hvmail-bounce-processor   UP                      UP
hvmail-bounce-qmail-send  UP                      UP
hvmail-config-agent       UP                      UP
hvmail-dd-dispatcher      UP                      UP
hvmail-dd-logreader       UP                      UP
hvmail-disk-qmail-send    UP                      UP
hvmail-dnscache           UP                      UP
hvmail-event-processor    UP                      UP
hvmail-httpd              STARTING
hvmail-logfile-agent      UP                      UP
hvmail-logfile-summary    UP                      UP
hvmail-logfile-writer     UP                      UP
hvmail-postgres           UP                      UP
hvmail-pure-authd-studio  UP                      UP
hvmail-pure-ftpd          UP                      UP
hvmail-qmail-pop3d        UP                      UP
hvmail-qmail-smtpd        UP                      UP
hvmail-qmail-smtpd2       UP                      UP
hvmail-qmail-smtpd3       DOWN
hvmail-ram-qmail-send     UP                      UP
hvmail-redis              UP                      UP
hvmail-redis-np           UP                      UP
hvmail-rpc                UP                      UP
hvmail-rspawn-limiter     UP                      UP
hvmail-simplemh           UP                      UP
hvmail-simplemh2          UP                      UP
hvmail-smtp-sink          UP                      UP
hvmail-studio-worker      UP                      UP

OVERALL STATUS: UP

In the example above, you’ll see the warning banner at the top of the output. Additionally, the hvmail-httpd service is stuck at a state of STARTING. If you look into the log files, you’ll see the error there as well.

# tail /var/hvmail/log/httpd/current | tai64nlocal
2018-04-13 08:10:58.148551500 error (256) while loading greenarrow configuration
2018-04-13 08:10:58.148922500 ERROR: failed to generate configuration
2018-04-13 08:10:59.159582500 /var/hvmail/bin/greenarrow_config: Errors found in configuration file (/var/hvmail/control/greenarrow.conf):
2018-04-13 08:10:59.159584500  - line 26: unknown identifier invalid_grouping_directive
2018-04-13 08:10:59.161435500 error (256) while loading greenarrow configuration
2018-04-13 08:10:59.161436500 ERROR: failed to generate configuration
2018-04-13 08:11:00.182314500 /var/hvmail/bin/greenarrow_config: Errors found in configuration file (/var/hvmail/control/greenarrow.conf):
2018-04-13 08:11:00.182317500  - line 26: unknown identifier invalid_grouping_directive
2018-04-13 08:11:00.183340500 error (256) while loading greenarrow configuration
2018-04-13 08:11:00.184608500 ERROR: failed to generate configuration

The error output should give you enough information to fix the problem in greenarrow.conf. Once the problem has been corrected, the service will automatically start up and resume normal operation.

Dynamic Defaults

Several of the configuration fields within GreenArrow Engine default to being adjusted automatically based upon the available CPU and RAM resources.

To view the values used for these fields, run the following command.

# greenarrow_config show_dynamic_defaults
Directive                                                               Default           Override
--------------------------------------------------------------------------------------------------
general.studio_message_generation_processes_per_campaign                      2                  2
general.studio_workers_shared                                                 0                  -
general.studio_workers_campaigns                                             10                 15
general.studio_workers_general                                                2                  -
general.studio_workers_imports                                                1                  -
general.studio_workers_transactional                                          1                  -
general.passenger_max_requests_in_queue                                     768                  -
general.passenger_min_application_instances                                  16                  -
general.passenger_max_application_instances                                  16                  -
general.apache_max_clients                                                 1024                  -
general.postgres_max_connections                                            300                  -

Estimated memory requirements                                           Default      As Configured
--------------------------------------------------------------------------------------------------
Passenger web processes                                                 3200 MB            3200 MB
Studio worker processes (/service/hvmail-studio-worker)                 2400 MB            3400 MB
--------------------------------------------------------------------------------------------------
The memory estimates listed above do not reflect all components within GreenArrow, but only
these specific components. See the documentation on Memory Utilization for more information:
http://docs.drh.net/greenarrow-engine/Configuration/Performance-Tuning/Memory-Utilization

If you would like to “freeze” those values so that they will no longer dynamically adjust, run the following command.

greenarrow_config freeze_dynamic_defaults >> /var/hvmail/control/greenarrow.conf

If running the above command results in duplicate configuration entries, then the last of the duplicates takes precedence.

Review Per-Domain Configuration

The command greenarrow_config domain_settings is a diagnostic tool used to see, with the current configuration in /var/hvmail/control/greenarrow.conf, what settings apply to a specific IP/Domain combination.

For example, given this configuration:

ip_address * {
  domain * {
    reuse_connections yes
    reuse_connections_timeout 2s
    reuse_connections_max_messages 100
    log_smtp_commands yes
  }
  domain yahoo.com {
    log_smtp_hexdump yes
  }
}
ip_address smtp-1 {
  domain * {
    reuse_connections_max_messages 500
  }
  domain gmail.com {
    reuse_connections_max_messages 2500
    starttls_use yes
  }
}

Here are some example invocations:

# greenarrow_config domain_settings smtp-1 gmail.com
Configuration on IP "smtp-1" to domain "gmail.com":
------------------------------------------------------------------------
reuse_connections                             | true
reuse_connections_timeout                     | 2s
reuse_connections_max_messages                | 2500
starttls_use                                  | true
starttls_require                              | <default>
starttls_require_action                       | <default>
delivery_override                             | <default>
smtp_route                                    | <default>
message_transfer_timeout_action               | <default>
message_transfer_response_timeout_action      | <default>
log_dns                                       | <default>
log_smtp_connections                          | <default>
log_smtp_commands                             | true
log_smtp_hexdump                              | <default>
max_concurrent_connections                    | <default>
max_delivery_rate                             | <default>
throttle_program                              | <default>

# greenarrow_config domain_settings smtp-1 yahoo.com
Configuration on IP "smtp-1" to domain "yahoo.com":
------------------------------------------------------------------------
reuse_connections                             | true
reuse_connections_timeout                     | 2s
reuse_connections_max_messages                | 500
starttls_use                                  | <default>
starttls_require                              | <default>
starttls_require_action                       | <default>
delivery_override                             | <default>
smtp_route                                    | <default>
message_transfer_timeout_action               | <default>
message_transfer_response_timeout_action      | <default>
log_dns                                       | <default>
log_smtp_connections                          | <default>
log_smtp_commands                             | true
log_smtp_hexdump                              | true
max_concurrent_connections                    | <default>
max_delivery_rate                             | <default>
throttle_program                              | <default>

# greenarrow_config domain_settings smtp-2 gmail.com
Configuration on IP "smtp-2" to domain "gmail.com":
------------------------------------------------------------------------
reuse_connections                             | true
reuse_connections_timeout                     | 2s
reuse_connections_max_messages                | 100
starttls_use                                  | <default>
starttls_require                              | <default>
starttls_require_action                       | <default>
delivery_override                             | <default>
smtp_route                                    | <default>
message_transfer_timeout_action               | <default>
message_transfer_response_timeout_action      | <default>
log_dns                                       | <default>
log_smtp_connections                          | <default>
log_smtp_commands                             | true
log_smtp_hexdump                              | <default>
max_concurrent_connections                    | <default>
max_delivery_rate                             | <default>
throttle_program                              | <default>

# greenarrow_config domain_settings smtp-2 yahoo.com
Configuration on IP "smtp-2" to domain "yahoo.com":
------------------------------------------------------------------------
reuse_connections                             | true
reuse_connections_timeout                     | 2s
reuse_connections_max_messages                | 100
starttls_use                                  | <default>
starttls_require                              | <default>
starttls_require_action                       | <default>
delivery_override                             | <default>
smtp_route                                    | <default>
message_transfer_timeout_action               | <default>
message_transfer_response_timeout_action      | <default>
log_dns                                       | <default>
log_smtp_connections                          | <default>
log_smtp_commands                             | true
log_smtp_hexdump                              | true
max_concurrent_connections                    | <default>
max_delivery_rate                             | <default>
throttle_program                              | <default>

Example File

Here is an example file:

general {
  studio_message_generation_processes_per_campaign 1
  studio_workers_shared 5
  studio_workers_campaigns 0
  studio_workers_general 3
  studio_workers_imports 0
  studio_workers_transactional 1

  passenger_max_requests_in_queue 256
  passenger_min_application_instances 2
  passenger_max_application_instances 4

  apache_max_clients 500

  postgres_max_connections 200
}