Is Microsoft 365 Really Public Records Compliant?
If you work for a government agency or government-owned entity you’re probably aware of the rules that require you to retain public records. Electronic records, including emails, are covered by these requirements and you might think that Microsoft 365 “has your back” because you’ve bought their email archive feature set.
What you might not realise, however, is that not all email archives are created equal.
Records Retention Requirements in Australia & New Zealand
In Australasia there are multiple acts that pertain to record retention and, depending on the specific industry, range from 7-10 years for the likes of financial records all the way up to 20 years after a patient dies for health records. For industries such as health we have effectively entered an era where electronic records cannot be deleted and this presents challenges for organisations and their service providers.
For government agencies and publicly owned entities the requirement to store and retrieve public records, including emails, is controlled by legislation that defines a process and time-frame to access public records.
Doesn’t Microsoft 365 have me covered?
Microsoft 365, nee Office 365, provides mailbox backup and email archiving with e-discovery & legal hold features.
At a high level it looks like this should meet every organisation’s records management requirements however there are some big gaps in the Microsoft 365 offering that should set off alarm bells for most CIOs. I set out the four main issues we have found with Microsoft 365 for records management and in particular with respect to public records compliance.
Issue #1 - Microsoft 365 Exchange Online Archive
Microsoft’s version of email archive is available as an extra add-on for their Exchange Online and Microsoft 365 offerings. This Microsoft 365 service creates a shadow mailbox which allows users to retrieve emails they might have accidentally deleted or lost (stolen laptop or PST file corruption for example). However this shadow mailbox is fully managed by the user which means the user can delete emails from the shadow mailbox as well.
In this scenario deleted emails are gone forever and can never be produced if needed in the future e.g. for regulatory compliance purposes. This “end-user management” strategy also allows users to purge the organisation’s mail flow of any emails they don’t want the boss to see - another potential problem in case emails are needed for investigative purposes.
So the Exchange Online Archive isn’t an appropriate tool for records compliance, public or otherwise.
Issue #2 - Microsoft 365 E3 & E5
The Microsoft 365 E3 & E5 bundles for Enterprise customers come with a feature called Information Protection which allows organisations to protect certain information in their 365 tenancy.
This sounds great in principle until you start looking at how it works. Unless you turn on information protection for every single mailbox within the organisation Information Protection won’t help. This is because Information Protection relies on rules to be created by the customer to identify which assets to protect and therefore prevent users from deleting. We have heard reports of some E5 customers enabling Information Protection on all electronic assets in their organisation however we haven’t heard how usable this tool is under these conditions.
For Information Protection to work properly you have to know which information you will need to protect in the future. Of course this is impossible and unless you create blanket rules to protect everything you’re guaranteed to miss some assets.
So while the Information Protection feature within the E3 and E5 bundles may have the ability to provide public records compliance it requires knowledge of the future to work properly. Given the gaps identified above with the Exchange online archiving service, the uncertainty around which data is being captured by Information Protection makes it unsuitable for public records compliance as well.
Issue #3 – When staff members leave
Due to the fact that Microsoft 365 is licensed on a per-mailbox basis, all features are tied to an individual mailbox. This provides a certain level of granularity for licensing purposes but it also means that, in the case of archive mailboxes, customers risk deleting company emails when deleting an ex-staff member’s mailbox.
When a user leaves the organisation ordinarily the user’s mailbox is deleted and a new mailbox created for the new staff member. However in this scenario, because Microsoft 365’s archiving solution is tied to the user’s actual mailbox, the archive is also deleted. This would be catastrophic for an agency that needs to ensure public records compliance as this deletion has created gaps in the organisation’s email archive.
Under this model, organisations need to continue paying for ex-staff mailboxes just so they can ensure public records compliance. These unnecessary license costs are only going to increase as staff turn over within agencies.
Issue #4 – The (unnecessarily) high price of compliance
We can’t really fault Microsoft’s pricing for their mailbox archiving solution which is fairly consistent with the market (though as pointed out above it won’t help an organisation achieve compliance due to its user-centric nature).
However the additional cost for the extra features that e-discovery comes with in the E3 and E5 bundle puts it well out of reach of most organisations in Australasia. For example a 1,000 user agency in Australasia will need to add $600,000 for E3 or $1m for E5 to their IT budget each year to not-quite-achieve public records compliance.
Given the complexity and extremely high price of these package it’s not unreasonable to assume that most organisations that need to comply with long term email archiving requirements are struggling to manage their obligations. They either have the Microsoft 365 E3 or E5 bundles and don’t have them fully configured and are therefore not actually compliant or have been put off by the high price and are struggling to find a solution to take them to the cloud.
SMX’s Immutable Object Store
In support of Microsoft 365 customers SMX took a different approach to storing customer emails in our cloud archive service. Firstly, like all our services, the SMX email archiving service applies to all emails sent to or from the organisation. This means customers don’t need to turn on archiving for certain users and no possibility of having an incomplete record of the organisation’s mail flow.
However the main difference with SMX’s approach is that archived emails can’t be deleted by individual users. This essentially provides legal hold functionality across the entire organisation’s email flow from day one.
SMX’s approach still provides the ability to restore individual emails and complete mailbox contents just like traditional mailbox backups did. It’s kind of like the Microsoft 365 Archive + Information Protection all-in-one.
Legacy archive ingestion
SMX’s cloud email archiving solution was designed from the ground up to be capable of ingesting your organisation’s email legacy archive. Most organisations have a legacy archiving solution that is in urgent need of retirement and it’s reassuring to know that after migrating to a cloud provider your legacy emails are available in the same interface as your current emails.
In the Microsoft 365 world there is no standard place to put your legacy archive. Yes you could create new mailboxes to hold this information but it would be an aggregation of many previous and current staff member’s emails and would therefore require some further sorting to ensure it could be found easily again. It’s just another example of how the Microsoft 365 archiving isn’t appropriate for reliable long term records compliance and nor is it an efficient use of taxpayer funds.
Migration to the cloud
As part of migrating to the cloud, SMX provides a completely managed service that includes ingesting your legacy data. This is a 4-step process where we sample and prepare your data, transfer it to our facility, perform a complete analysis of your legacy emails to determine the percentage of corrupt emails and then perform the ingestion. This process provides certainty around migration time-frames & costs and has allowed SMX to migrate customers large and small including telcos such as Spark.
SMX is a Microsoft IP Co-sell partner and selected the Azure platform to host our email archiving service. We make use of some specialised tools within the Azure platform to ensure your data is stored at multiple geographically distributed data centres and always available when you need it.
SMX is Australasia’s Trusted Email Archiving Service Provider
SMX is one of the only local providers that can guarantee your data is stored and supported in Australasia. This immediately obviates any data sovereignty concerns that might come up when evaluating large global providers.
SMX is the only specialist email security provider appointed to the New Zealand government’s TaaS panel and has been providing email security services to government agencies since 2005.
We maintain at least four copies of all emails stored in the SMX archive at any one time spread across two geographically distributed regions on Microsoft’s Azure cloud storage platform in Australia and each copy is encrypted at rest using AES256-bit encryption.
We also make extensive use of the Cassandra distributed database to ensure your data is always available. One important feature for distributed and resilient data storage that Cassandra includes is the constant data rebalancing performed by each node. This ensures that we’re making not only the most efficient use of space on each node and that we have at least four copies of each email, but we also negate the ability for bit rot to set in.
Bit rot affects data that has been stored for a long period of time and can affect encrypted data in particular. Bit rot has various causes, from cosmic radiation to dust on hard drive spindles but can be obviated by either good practice or good software (or both). In SMX’s case our storage nodes are constantly rebalancing the data they’re responsible for. This involves reading and writing email blobs across the entire cluster so we can ensure your data is safely encrypted and available when you need it.
The answer is...
In the beginning I asked the question “Is Microsoft 365 really public records compliant?” Given the issues outlined above with the Microsoft 365 services:
- allowing staff members to influence the organisation’s mail flow,
- uncertainty around which emails are actually protected,
- the yawning chasm created when users leave the organisation and
- the unnecessarily high price of compliance
I think the answer is an uncategorical no, you should not be relying on Microsoft 365 as the source of information to meet your public records compliance requirements.
In my opinion a user-centric model might be good for licensing desktops but is completely inappropriate for ensuring retention and retrieval of public records. For that you need a provider that addresses your requirements from an entire organisation point of view.