Sender Policy Framework (SPF) is a simple idea - it allows domain owners to specify the IP addresses from which recipients can expect mail. However, mail service providers will often have many IPs and IP ranges which they will use to send mail - and these IPs and IP ranges are subject to change. Rather than having to constantly ask their customers to update these IPs and IP ranges, mail service providers will ask their customers to simply include the IP ranges hosted at an address they specify. Microsoft, for example, will ask their customers to publish into their SPF record, the highlighted section of the following SPF record:
"v=spf1 include:spf.protection.outlook.com ~all"
If you run an SPF query against that address, you'll see the following:
"v=spf1 ip4:220.127.116.11/15 ip4:18.104.22.168/16 ip4:22.214.171.124/14 ip4:126.96.36.199/17 ip6:2a01:111:f400::/48 ip6:2a01:111:f403::/49 ip6:2a01:111:f403:8000::/50 ip6:2a01:111:f403:c000::/51 ip6:2a01:111:f403:f000::/52 include:spfd.protection.outlook.com -all"
Because Microsoft owns the domain spf.protection.outlook.com, they can simply have their customers publish that one entry, while maintaining their own list of sending IPs globally. Notice how this SPF record includes "spfd.protection.outlook.com" too - this is known as a nested lookup.
Problems with SPF
One of the chief problems with SPF is centered around the DNS Lookup limit as defined in the SPF RFC. Essentially, the specification only allows a domain owner to trigger a maximum of 10 (ten) DNS lookups - and each time a domain owner adds a new include, they trigger at least another DNS lookup. Thus, by simply adding the Microsoft include to your SPF, you are triggering 2 additional DNS lookups - one for spf.protection.outlook.com and one for spfd.protection.outlook.com.
Adding additional service providers - like your mail marketing system or your CRM - only compounds this issue and we find that domain owners very quickly consume the 10 allotted lookups.
When the limit is reached, most mail filters will ignore the SPF record wholesale - meaning that both delivery and security issues can arise.
To meet this problem, the Sendmarc platform utilizes a feature called SPF Optimization. SPF Optimization effectively reduces all your SPF includes down to the IP level, as IP addresses and ranges are not counted against the DNS lookup limit. Note that this feature is only available to customers making use of the Sendmarc SPF Managment feature.
SPF Optimization works by storing the mechanisms a given SPF record utilizes, resolving those down to an IP address level (through all nested lookups) and publishing to the public internet only the IPs and ranges. Thus, while the you might see spf.protection.oulook.com in the Sendmarc interface, the internet will see all the Microsoft IPs. Resolution happens on a consistent basis, so no changes made by a provider are missed.
An excerpt of a domain with SPF Optimization enabled
Another important note is that this is done on an opportunistic basis, meaning that only those domains that are approaching the lookup limit will see any changes.
This feature is enabled by going to Domains clicking on the domain you wish to enable the feature for, clicking Edit, SPF Settings, Advanced Settings and toggling the Enable toggle under SPF Optimization.