How to Cut Cloud Storage Costs by Securely Archiving Data with Dataverse Retention Policies

Highlights

• New Dataverse feature to help reduce data storage costs by archiving old records to compressed, immutable long-term retention.
• Meet legal and compliance requirements by retaining historical data accessibly and securely.
• Speed up performance by reducing the active database size.
• Use native Dataverse retention policies to avoid building a custom archive solution.
• Manage retention policies across environments by making them solution aware.

Data volumes are expanding faster than a rising loaf of bread these days. And all that comes with headaches. Effectively managing cloud storage usage and meeting ever-growing compliance rules to keep historical records accessible.

But thanks to Microsoft's freshly baked solution, you can automate archiving stale info to keep costs down and auditors happy.

Read on to learn how Dataverse retention policies can help you refine your data management, retaining unlimited records without burning through your budget.

Using custom retention policies, admins can set policies to identify inactive records. These will automatically move records into a read-only state and keep them securely stored in Dataverse. Retained records remain easily accessible through Dynamics 365 and Power Apps.

Because data is compressed in Dataverse long term retention, your organisation can benefit from reduced database sizes, leading to lower storage costs without compromising security.

Microsoft says every GB moved to Dataverse long term retention will consume, on average, 50% less database capacity. Some exceptions do apply, including file attachments which aren’t compressed, so keep reading for details.

With inactive data moved out, the active database size is reduced, which can improve performance. Less data to query means faster response times.

While it’s not a magic recipe for all storage needs, with careful planning and testing, Dataverse retention can help you tackle ever-increasing data and compliance challenges.

Serversys is your go-to partner to help you implement custom retention policies. Our experts are ready to provide the advice and guidance you need to ensure these are tailored to your unique needs. Contact us to learn more.

How Dataverse Retention Works

At a high level, Dataverse retention allows administrators to set policies that identify legacy records based on their specified criteria. These are then permanently moved into the Dataverse long-term retention data store.

Retained data is securely stored in a Dataverse managed data lake and remains backed by Azure Active Directory security. Data is kept in Azure for secure storage and compressed for efficiency and remains manageable by Dataverse.

Crucially, the retained data is read-only and immutable, which means these records cannot subsequently be moved back into the active database.

However, retained records continue to work within your environment infrastructure. For instance, any existing behaviours, such as logic rules, will continue to apply to records moved to long term retention.

Read-only access to retained data is available through advanced find within an app, Power Automate cloud flows and Dataverse OData APIs.

All Dataverse standard and custom tables can be moved to Dataverse long term storage, along with attachments. Audit and system tables are notable exceptions.

Dataverse long term retention doesn’t require an additional purchase.

Setting Retention Policies

Administrators must a set custom policy for each table with rules for long term data retention. Each rule will automatically cascade down to child tables once enabled on a parent.

Dataverse views will then be created to identify which legacy records should be retained from each table.

Once table rules are enabled, and views are defined, retention policies can be scheduled to run each week, month, or another frequency.

So, an immutable retention store ensures records cannot be altered after retention, helping demonstrate compliance. Criteria views then provide flexibility to target specific data types and date ranges.

Retention policies are solution aware. This allows policy definitions to be managed through solutions and portable across environments, saving time and simplifying application lifecycle management.

Reporting Storage Capacity

With Dataverse data retention enabled, the reported capacity in Power Platform admin reports will detail both live (active) and retained (inactive) data.

Reports will display two entries for each table enabled for long term retention. For example, “Contact-Retained” as well as “Contact” for active data.

Example Use Cases

• Dataverse retention policies can be useful for many scenarios, including:
• Financial service organisations can meet document and email retention requirements for audits.
• Healthcare providers can archive patient records while keeping them available to comply with regulations.
• Retailers can preserve transaction data for operational analysis.
• Older support cases can be archived while retaining access as needed.
• Marketing campaign performance records can be compressed and retained for future analysis.
• Archive contact data for one-time vendors, suppliers, and contractors when projects are completed.
• Retain proposal and contract documents after their expiration date.
• Keep interaction histories associated with lost prospects and churned customers.

Retaining File Attachments

Using compression, Dataverse long term retention will reduce database storage, but this does not apply to attached files.

Attachments associated with retained records are archived in the retention store without compression. This applies to all file types, including Word docs, PDFs and JPEGs.

As a result, attachments remain at full storage size even after retention policies move these files to long-term storage.

Further consideration may be needed to determine if all attachments to archived records must be retained. For instance, storage savings may be possible by excluding some file types through retention policy criteria.

Other Considerations and Limitations

Additional Dataverse long-term retention points to consider include:

• This may not be a viable solution for organisations with large datasets that involve complex relationships.
• Retention focuses on structured Dataverse table data. Unstructured data would require an alternative solution.
• Advanced custom plugins and extensions may not function properly once data is moved to the Dataverse-managed data lake.
• Running data policies consume API requests in the Power Platform, which are deducted from the requests available with a plan.
• Dataverse retention is meant for scheduled policy-based retention. This isn’t a solution to recover data deleted in error.

Finally, Dataverse retention policies focus on retaining and archiving historical data for compliance purposes, not deleting it.

Admins can incorporate retention policies as part of a broader data lifecycle strategy to cover expiry and deletion as needed. Contact our experts for advice in defining your data retention policy and lifecycle strategy.

Summary

• Cost Savings - Moving inactive data into long-term retention will help reduce database usage. Most data will be compressed in retention by up to as much as 50%.
• Reduces Database Size - With inactive data moved out, the active database size is reduced, which can improve performance.
• Compliance - Retaining data long-term will help to meet legal, regulatory and audit requirements by keeping old records accessible.
• Data Immutability - Retained data is read-only, so there's no risk of subsequent edits. Retained data remains secured and searchable.
• Native Platform Support - Dataverse provides built-in retention support, so there's no additional license purchase or IT investment required to build and maintain it.
• App Development - Making policies solution-aware aids application lifecycle management across environments.

Microsoft Resources

• Dataverse long term data retention overview
• Set a data retention policy for a table

Next Steps

Dataverse retention policies benefit organisations by securely and cost-effectively retaining historical data long-term while reducing active database size.

Contact Serversys to learn more and explore how Dataverse retention policies can be configured for your Dynamics 365 or Power Apps database.