Obscured Email Addresses
- Table of Contents
- Why use obscured email addresses?
- What does this protect?
- Obscured Email Address
- Limitations
- Internal Error
The email address written to GreenArrow’s log files and database can be replaced with an obscured address.
- For SimpleMH, this is done with the X-GreenArrow-Obscured-Email header.
- For Marketing Studio Remote Lists, this is done with the obscured_email column.
When providing an obscured address, the real recipient address will be delivered to as normal. However, all logging to disk and the GreenArrow database will be done using the obscured email address.
Why use obscured email addresses?
For some installations, you may want to limit the amount of recipient data written to disk by GreenArrow.
Some examples of why you may want to use this feature:
- To ease GDPR compliance.
- You want to deliver email without increasing the surface area of software that stores your recipient addresses.
What does this protect?
When an obscured email is provided, that address will be used (instead of the recipient’s real email address) in the following areas:
- Email addresses stored by GreenArrow Engine:
- Delivery attempt logs
- Web server logs
- Bounce processor logs
- Event Processor logs
- Other service logs – excluding SMTP server logs which only log email addresses if configured
- SimpleMH and Studio Remote List recipient addresses
- SimpleMH click and open tracking
- The bounce_bad_addresses table
- Bounce processor repeat bounce tracking
- Events
- Send Summary files
- Email addresses stored by GreenArrow Studio:
- Campaign statistics for Remote Lists
- Special Sending Rule errors
The real subscriber email address can appear on disk briefly over the course of delivery in the following areas:
- The disk-queue
- Adjust the Queue Lifetime setting to reduce the amount of time the address can appear on disk.
- The Redis AOF and RDB files in
/var/hvmail/data/redis- Redis is used during Studio and SimpleMH delivery. As soon as the first delivery attempt occurs, the address is removed from Redis. However, it can take some time before the removal is reflected on disk. See the “Limitations” section below for information on reducing how long it is before these files are rewritten.
- Marketing Studio campaign log files in
/var/hvmail/var/tmp- Most campaigns delete these files after they finish.
- Campaigns that fail due to software errors can retain their log files on disk.
Real subscriber email addresses that exist in the ram-queue and bounce-queue are moved from RAM to the following disk locations when GreenArrow is stopped, and back to RAM when GreenArrow is started:
/var/hvmail/qmail-ram/savedqueue/var/hvmail/qmail-bounce/savedqueue
The above temporary disk locations may end up in backups, if a backup is taken while they are present.
Obscured Email Address
The obscured email can be provided in a few different formats, depending on what data you wish to retain in GreenArrow’s logging and database. The table below describes what happens in each possible situation.
The resulting email that is used for logging and in the database always ends in either @_anon_ or ._anon_.
| Input type | Example input | GreenArrow anonymizes to | Example anonymized address |
|---|---|---|---|
| A blank value or is not provided. | NULL |
No modifications are made to the email address. | (The original email address) |
A value that does not include an @ (i.e. is considered to be just the localpart). |
justlocalpart |
The value is appended with @_anon_. |
justlocalpart@_anon_ |
A value that ends in @. |
alias@ |
The value is appended with _anon_. |
alias@_anon_ |
A value that includes @ and ends in neither @, @_anon_ nor ._anon_. |
[email protected] |
The value is appended with ._anon_. |
[email protected]._anon_ |
A value that already ends in @_anon_ or ._anon_. |
part@_anon_full@domain._anon_
|
The value is used unaltered. |
part@_anon_full@domain._anon_
|
Limitations
Marketing Studio Remote List Event Tracking
In order for events (clicks, opens, unsubs, etc) to be tracked properly in Marketing Studio while using this feature, you should include a distinct_id in your remote query.
The Event Notification System will deliver its events including both the obscured email address and this distinct_id.
Bad Address Suppression
SimpleMH Bad Address Suppression does not work on messages injected using this feature. This is because SimpleMH Bad Address Suppression works using the recipient’s email address, but data is written to bounce_bad_addresses using the obscured email.
Marketing Studio Suppression Lists work for Studio Remote Lists, but we recommend writing your suppression lists using MD5 or SHA256 suppression to avoid writing those addresses to disk.
Could Not Process Bounce Messages
The GreenArrow bounce processor will ordinarily write messages which it cannot process to disk.
See the write_could_not_process_bounce_messages configuration directive to disable this behavior.
Event Processor
The Event Notification System can include recipient email addresses in the bounce_text column.
This applies to events of type bounce_all and bounce_bad_address.
To mitigate this:
- Set bouncemaxbytes to 1 byte (since 0 means unlimited).
- Set store_bounce_details_length to 0.
Redis
GreenArrow uses an internal Redis server for its delivery system. Redis’s data can temporarily contain recipient email addresses.
To mitigate this, you can instruct Redis to rewrite its AOF file more frequently, causing this data to be retained on disk for less time.
cat >> /var/hvmail/control/redis.conf <<EOF
auto-aof-rewrite-percentage 50
auto-aof-rewrite-min-size 10mb
EOF
Internal Error
If an internal error is encountered while processing obscured email addresses,
a bounce will be written to the address
[email protected].
This error comes as a result of file corruption or a communication error within GreenArrow. This is not a normal situation, but is documented here to aid in troubleshooting this situation.