1.2 Advanced Delivery

To keep organizations secure by default , Exchange Online Protection does not allow safe lists or filtering bypass for messages that are identified as malware or high confidence phishing (HPHISH). But there are specific scenarios that require the delivery of unfiltered messages:

Third-party phishing simulations: Simulated attacks can help you identify vulnerable users before a real attack impacts your organization.

Security operations (SecOps) mailboxes: Dedicated mailboxes that are used by security teams to collect and analyze unfiltered messages (both good and bad).

Advanced Delivery - Phish Simulation Troubleshooting

The Basics - Double-check the sending domain, sending IP and simulation URLs to allow

As described in the product documentation here: Use the Microsoft 365 Defender portal to configure third-party phishing simulations in the advanced delivery policy - Office 365 | Microsoft Docs , for Advanced Delivery to be applied to the message, both the sending domain (P1Sender domain or DKIM domain) and sending IP need to be configured correctly in Advanced Delivery.

Note that, if the phishing simulation is an email-based one, it is now not necessary anymore to configure the "phishing" URL(s) under simulation URLs to allow.

Identifying if the sending domain and sending IP are configured correctly

This guide assumes that you have collected the output of Get-PhishSimOverrideRule in the user's environment, as well as the message headers of the phishing simulation message. Next, look at the Authentication-Results header of the message. Here is an example:

Authentication-Results:
 spf=pass (sender IP is 207.211.31.120) smtp.mailfrom=corporate-updates.com;
 dkim=none (message not signed) header.d=none;
 dmarc=bestguesspass action=none header.from=corporate-updates.com;
 compauth=pass reason=109
        

From the above, we know the following:

• The sending IP is 207.211.31.120.
• The P1Sender/MAILFROM domain is corporate-updates.com (from the smtp.mailfrom header field).
• There is no DKIM signature. If there had been one, the DKIM domain would be found under the header.d field.

Now we need to look at the Get-PhishSimOverrideRule output to see if the sending IP and P1Sender domain are configured or not. Here is an example:

PS > $PhishSimOverrideRule = Import-Clixml "C:\Temp\Get-PhishSimOverrideRuleOutput.xml"
PS > $PhishSimOverrideRule.SenderIpRanges
205.139.111.0/24
207.211.31.0/25              # <-- "207.211.31.120" is included here
207.211.32.0/25
PS > $PhishSimOverrideRule.Domains | sort
corp-accounts.com
corp-news.com                # <-- "corporate-updates.com" is not listed here
instant-promos.com        

From the above, we can see that the sending IP is configured correctly but the P1Sender domain is not configured in the list of domains. The user/customer needs to either add the P1Sender domain (corporate-updates.com) into their Advanced Delivery configuration, or DKIM-sign the message and add the DKIM domain into their Advanced Delivery configuration.

(only for non-email phishing simulations (e.g. Teams, Office docs etc.)) Identifying if the simulation URL(s) to allow is/are configured correctly

This guide assumes that you have collected the output of the following command in the user's environment:

Get-TenantAllowBlockListItems -ListType Url -Allow -ListSubType AdvancedDelivery
        

Assuming that the phishing URL here is 
=============================

hxxps://grubhub[.]at[-]us[.]therelayservice[.]com/service/VSv8afw1KbaZ1_E4WtMXjbKPWTHRnmvpGECYxYAoj_zy6BjVRouXzeEECWqjFIwzWmO7_wosmbYHjd-5g-q1rxTjOfMnNuDkcwmMM7nQ5EmNryP7s_lWjxQUMJMyLmd2yoheoEud6yroula0kiTw-XJB35BVAxRJdx0pFJEpIxa3-t4Rd5a9-wzX_os27oggG5lrNVTQTlnjub_PnfgyDLryQZd3dAxatJXLJqu6Zdq--GlIGO6bKggu3EQm1LE1msCErXfrlPZk8W3zeCfHSYg6XHoGZaEsha-11j62YgBrriDKlCfpBMOQkauu15iiTI_s90mA8dtpbGg3VJpnjGtjldP6F1dx0mQw0hnII48LRMiQpQF5MUFqT8fawyon4Vwod5zE9EaeopWnhHK9zlHe50C29pZkCfjddRRv2tY (obfuscated for security reasons),         

we compare this with the list of phishing simulation URLs configured by the user to see if any match:

PS > $TABLItems_Url_Allow_AdvancedDelivery = Import-Clixml "C:\Temp\Get-TenantAllowBlockListItemsOutput.xml"
PS > $TABLItems_Url_Allow_AdvancedDelivery | FT Value,LastModifiedDateTime,ModifiedBy

Value                   LastModifiedDateTime ModifiedBy
-----                   -------------------- ----------
therelayservice.com/* 12-Jul-22 8:26:21 PM [email protected]
*.therelayservice.com   12-Jul-22 8:25:23 PM [email protected]
        

In this case, none of the entries match completely (the first one does not match any subdomain and the second one does not match any path). See URL syntax for the Tenant Allow/Block List - Office 365 | Microsoft Docs for detailed examples and explanations on the URL syntax for TABL URL entries.

Problem 1: I’ve setup a Phish Simulation override but my mail is still quarantined/junked

Complex Routing Scenario with MX pointing to EOP

The user points MX to EOP and implemented a complex routing scenario via Centralized Mail Transport or Conditional Mail Routing and users targeted by the Phish Simulation are hosted in EXO, emails flowing as follows:

Phish Simulation does not work with Complex Routing because in such scenarios EOP cannot identify the IP of the true source of the email, making it impossible to trigger the PhishSimOverrideRule scoped to sending IP and P1/DKIM domain. (Enhanced Filtering does not support Complex Routing scenarios)

Note: Do not attempt to work around this Complex Routing limitation by adding OnPrem/3rd Party users infrastructure IPs to PhishSimOverrideRule sending IP list. This would effectively bypass spam filtering for any Internet Sender impersonating the P1 domain used for the Phish Simulation.

Message is detected as malware

MS latest update empowers security admins and SecOps teams to evaluate messages with malware verdicts independently. With this enhancement, messages delivered to pre-configured SecOps mailboxes in the advanced delivery policy can bypass the malware filter. By providing greater control over the analysis of such messages, organizations can increase their overall security posture, ultimately leading to improved protection against cyber attacks. No actions are required to enable this update.

Both sending IP and P1/DKIM domain need to match, not just one

The IP address in the Authentication-Results header needs to match the IP listed in the Phish Simulation settings. In addition, either the P1 domain or a valid DKIM signing domain needs to match a domain listed in the Phish Simulation settings. See "The Basics - Double-check the sending domain, sending IP and simulation URLs to allow" section above for a refresher on how to check these.

If the user MX does not point to M365, you may need to enable Enhanced Filtering/Skip Listing to allow the correct IP to be detected.

Note: If the user MX does not point to M365, then ETR SCL -1 will still allow any email, be it phishing simulation email or anything else. However, Advanced Delivery is still needed for non-email scenarios such as Teams-based simulations. So, configuring the policy is still the best method.

Problem 2: My links are still wrapped

Phish Simulation override will prevent detonation of URLs and attachments. It will not prevent Safe Links from wrapping URLs. If you want URLs to not be wrapped, you need to utilize the DoNotRewriteUrls (a.k.a DNR) list in the Safe Links policy.

NOTE: The DNR list is known to be incompatible with Advanced Delivery. When they are used together, it may cause alerts (e.g. "A potentially malicious URL click was detected") to not be suppressed, so don't recommend DNR to be used with Advanced Delivery.

Problem 3: IntraOrg email is quarantined.

NOTE : Today Advanced Delivery/SecOpsMailbox and Phishing Simulation override only works for Inbound email for all verdicts. With Intra org protection being released, if users switched on intra-org protection for phish/spam verdict, they will notice a regression in SecOps Mailbox Advanced Delivery behavior - emails marked as phish/spam intra-org will be quarantined, because the SecOps Mailbox override today is not configured for Intra-org.

Currently, the advanced delivery policy for third-party phishing simulations doesn't support simulations within the same organization (DIR:INT), especially when email is routed through an Exchange Server gateway before Microsoft 365 in Hybrid mail flow.

Work Around :

1. Create or use an separate anti-spam policy to cover the SecOps mailboxes that does not quarantine intra-organizational messages since MS doesn't recommend disabling intra-org protection for all mailboxes.
2. For Phishing Simulation,

• Create a dedicated send connector that doesn't authenticate the phishing simulation messages as internal.
• Configure the phishing simulation to bypass the Exchange Server infrastructure and route mail directly to your Microsoft 365 MX record (for example, contoso-com.mail.protection.outlook.com).
• Although you can set intra-organization message scanning to None in anti-spam policies we don't recommend this option because it affects other email messages

Problem 4: The message is delivered, but a user click fires an alert

Check the note under problem 2 about the DNR list. When the URL is added to the Do Not Rewrite list, alerts can still fire when the user clicks on the message. Microsoft recommends not adding these URLs to the DNR list. Note this is different from disabling URL rewrite entirely on the policy, which can be done as long as the user has a version of Outlook that supports the SafeLinks API (Outlook for Windows v16.0.15317.10000 or later, and Outlook for Mac v16.74.23061100 or later). This can also happen if the message is allowed via other means besides Phish Sim.

For example, if Phish Sim is misconfigured and they also allow the message via ETR, it may get delivered, but since it wasn’t tagged as a Phish Sim message, alerts can still fire. See "The Basics - Double-check the sending domain, sending IP and simulation URLs to allow" section above for details on how to ensure the Phish Sim policy applies to the message.

Public Content Links

Microsoft Docs | Configure the delivery of third-party phishing simulations to users and unfiltered messages to SecOps mailboxes

Permissions

Microsoft Docs | Permissions