GreenArrow Documentation

GreenArrow Updates

We release updates to GreenArrow Engine and Studio periodically. You can see the currently available releases at any time by viewing the GreenArrow Engine Changelog and GreenArrow Studio Changelog.

Updates for Cloud Customers

We update the majority of our Cloud Customers about once a month. The updates typically occur in the first few business days of the month.

Updates For On-Premise Customers

On-Premise customers can either ask GreenArrow technical support to perform updates for them or perform updates on their end by following the instructions in this document.

How to Find Your Currently Installed Versions

Recent GreenArrow installations make their version numbers available in the web interface.

GreenArrow Engine’s version number is shown in the footer of each Engine web page, next to the copyright statement. For example, this system is running Engine 4.1.223:

Copyright © 2012–2018 DRH Internet Inc. | Version 4.1.223

To view GreenArrow Studio’s version number, navigate to “Admin” => “System” and view the “Version Number” field:

GreenArrow Studio Version

If you’re running an older GreenArrow installation which does not show versions in the web interface or just prefer to use the command line, then you can also SSH into your server and run the following command to view GreenArrow’s versions:

rpm -q greenarrow-engine greenarrow-studio 2> /dev/null || dpkg -l greenarrow-engine greenarrow-studio

The exact format of the above command’s output will vary, depending on what Linux distribution you’re using. As an example, here’s what the output will look like in CentOS:

greenarrow-engine-4.1.210-0.x86_64
greenarrow-studio-4.103.5-0.x86_64

The above indicates that GreenArrow Engine 4.1.210-0 and GreenArrow Studio 4.103.5-0 are installed.

Operating System Updates

We recommend keeping non-GreenArrow packages up-to-date. When updating both GreenArrow and non-GreenArrow packages, the safest approach is to perform non-GreenArrow package updates first.

Update Prerequisites

The GreenArrow update procedure has the following prerequisites:

  1. An experienced Linux systems administrator who can execute the update procedure.

  2. The /var/hvmail/bin/greenarrow_blockers file must exist, and when executed, it must output text telling you that it’s safe to proceed with the update. This is described in more detail in step 2 of the update procedure below.

Update Procedure

Prerequisites

Verify that you’ve met all of the prerequisites listed above before proceeding.

This section describes how to update both GreenArrow Engine and Studio.

GreenArrow Engine and Studio have some shared code, so don’t be alarmed if you’re a GreenArrow Engine only customer, and see references to GreenArrow Studio in the output of the commands you run during the update.

GreenArrow has a lot of moving parts, so we recommend reading through the entire update procedure before proceeding, and considering whether the update should be scheduled. Depending on how old your current GreenArrow version is, what hardware is in use, and how much data is present, you can expect anywhere from a few seconds to a few minutes of downtime during most updates.

  1. SSH into the GreenArrow server as the root user, or a user with equivalent permissions.

  2. Check if it’s safe to proceed with the update, and only proceed if the greenarrow_blockers command outputs text telling you that it’s safe to proceed:

    greenarrow_blockers
    

    Stop here if greenarrow_blockers returns anything other than this:

    It is safe to upgrade this system by following the instructions at:
    http://docs.drh.net/greenarrow-engine/Server-Management-and-Backups/GreenArrow-Updates
    

  3. Check what version of GreenArrow Engine is installed:

    rpm -q greenarrow-engine 2> /dev/null || dpkg -l greenarrow-engine
    

  4. Read through the Version-Specific Steps section for any extra manual steps which are needed as part of the update process. Complete them at the time indicated.

  5. Update GreenArrow’s packages.

    On Red Hat-based Linux distributions, such as CentOS, run:

    yum --enablerepo=greenarrow update 'greenarrow-*'
    

    On Debian-based Linux distributions, such as Ubuntu:

    1. Enable GreenArrow’s package repository:

      sed -e '/greenarrow main/ s/^#*//' -i /etc/apt/sources.list.d/greenarrow.list
      

    2. Install the updated packages:

      apt-get update && apt-get --only-upgrade install 'greenarrow-*'
      

    3. Disable GreenArrow’s package repository to prevent accidental updates in the future:

      sed -e '/greenarrow main/ s/^#*/#/' -i /etc/apt/sources.list.d/greenarrow.list
      

  6. Run migrations. This step normally takes anywhere from a few seconds to a few minutes to complete:

    hvmail_migrate migrate
    

  7. Complete any post-update steps that you noted after reading through the the Version-Specific Steps section.

  8. Verify that some common misconfigurations are not present. Each test gets a simple pass/fail result:

    hvmail_check_config
    

  9. Verify that all services are running normally. Any services with an abnormal state are shown in red:

    hvmail_init status
    

  10. Send yourself a test email (replacing [email protected] with your actual email address):

    date | /var/hvmail/bin/mailsubj "GreenArrow Test Message" [email protected]
    

That’s it! You’re now running the latest GreenArrow releases.

Version-Specific Steps

The following table is in ascending chronological order. Please begin reading through it at the first release listed which comes after your currently installed version and work your down. Here are two examples:

  1. If you have GreenArrow Engine 4.1.216-0 or later installed, then there are no version-specific steps to complete.

  2. If you have GreenArrow Engine 4.1.180-0 installed, then you would begin with the upgrade instructions for 4.1.182-0.

All steps in this table should be completed before updating GreenArrow’s packages unless otherwise indicated:

4.1.182

July 7, 2017

In this update, GreenArrow switches from version 2.2 of the Apache HTTP Server to version 2.4. GreenArrow’s default configuration is automatically transitioned to Apache 2.4 style syntax as part of the package update, but any customizations made to Apache’s configuration should be reviewed ahead of time for incompatibilities. If any configuration file changes are needed, apply them post-update.

See this document for a list of potential issues.

Incompatible Apache customizations could be located in three places:

  1. The /var/hvmail/control/httpd.custom.conf configuration file.
  2. The /var/hvmail/control/httpd.ssl.custom.conf configuration file.
  3. Any .htaccess files that exist within the /var/hvmail/apache/htdocs directory. You can display a list of these files by running:

    find /var/hvmail/apache/htdocs -type f -name .htaccess
    

4.1.183

August 8, 2017

This update removes the duplicate clicktrackingid column from the Event Notification System. The click_tracking_id column is now consistently used to store the Click Tracking ID, whereas previously, the clicktrackingid and/or click_tracking_id columns were used, depending on the event type:

  1. If you’re processing Event Notification System data, then make any necessary updates on your end to handle the change in Click Tracking ID columns.

  2. Run the following command to gauge the time that this update will take. If the command takes more than 5 minutes to run or returns a value greater than 10,000, this update may take longer than normal:

    echo "SELECT COUNT(*) FROM events WHERE clicktrackingid != ''" \
    | /var/hvmail/postgres/default/bin/psql -U greenarrow
    

  3. Check whether the Event Processor is explicitly configured to transmit the clicktrackingid column:

    grep clicktrackingid /var/hvmail/control/event_processor.json
    

    If it is, remove the explicit clicktrackingid reference post-update, then verify that your edit did not introduce any syntax errors:

    hvmail_event_processor --check-syntax
    

4.1.216

April 18, 2018

This update introduces the greenarrow.conf configuration file. If GreenArrow’s Apache or PostgreSQL installation has a customized configuration, then one or more configuration directives may need to be moved into greenarrow.conf.

For this update, please:

  1. Complete all other upgrade steps.

  2. Run the following command, and examine the output:

    greenarrow_config validate
    
    If any conflicts are found, then a warning describing the conflict and how to resolve it are displayed as part of the output.

4.1.226

August 16, 2018

Starting with this update, the contents of the /var/hvmail/control/httpd.ssl.custom.conf file get included in alll HTTPS VirtualHost definitions. Previously, it was only included in the default HTTPS VirtualHost’s definition.

Before performing this update, please review the contents of your /var/hvmail/control/httpd.ssl.custom.conf file (which is blank by default) and verify that you’re okay with it being included in all HTTPS VirtualHost definitions.

If the new behavior presents a problem, then instead of defining VirtualHosts in /var/hvmail/control/httpd.ssl.listen, define them in /var/hvmail/control/httpd.ssl.custom.conf. See our HTTP Server documentation for more details on those two files.

4.1.227

August 31, 2018

This release lowers the dynamic defaults for apache_max_clients and passenger_max_requests_in_queue, reducing overall memory utilization to be closer to what it was before the April 2018 release of Engine 4.1.216.

If you’re running Engine 4.1.215 or earlier, then your release predates dynamic defaults, so no extra steps are required.

If you’re running Engine 4.1.216 or later, then we recommend the following:

  • Before upgrading, compare your existing configured or default limits (which you can get from the show_dynamic_defaults command) to the new defaults for these limits, which are described below. Most installations can use the new defaults for optimum performance.

  • After upgrading, monitor your utilization of these limits. You may want to adjust customized values in favor of the new defaults. You may find that customized values are no longer needed.

The new dynamic defaults are calculated as follows:

All systems start with the following:

apache_max_clients = 150
passenger_max_requests_in_queue = 100

If the system has more than 4GB of RAM and at least 4 CPU cores, those numbers are increased:

modifier = ( memory_in_gb - 4.0 ) * 32
apache_max_clients += modifier
passenger_max_requests_in_queue += 3 * modifier / 4