When to Archive Data and Not Migrate It

By: Marie van Roekel, Principal Customer Success Architect at Salesforce.org

When it comes to thinking about data migration or data archival, a conversation between your Salesforce Admin and a Customer Success Architect at Salesforce.org might sound like this:

Admin: I need to migrate 1.4 million contact records from our legacy systems to our new Salesforce org.

Architect: Really? Do you need all of those contacts day to day?

Admin: Well…I think we might need them.

Architect: This sounds like we might need to have a discussion about hot, warm, and cold data as part of your data migration strategy!

Let’s turn to an analogy to paint the picture.

Cold Data Storage

You’ve just bought a new home—exciting! It’s time to pack up all of the stuff in your current home for the big move. As you start filing boxes, you notice something. That bedroom set you inherited from your grandmother doesn’t really fit in with your plans for the new guest bedroom. But you can’t part with it because… well, grandma means a lot to you! Maybe a storage unit is in order. Or maybe you can convince your sibling to hold onto it for you. As long as it’s still “yours,” but stored somewhere else, you’re happy.

That would be similar to data you put into cold storage. You still want to touch it once in a while, but it’s OK if you have to take a trip to another location to get it. This is data that you may need to pull up to reference for special requests or compliance purposes. Generally, this isn’t data you would need to edit. This data could very well be stored off-platform. Users can request it and wait a day or two for it to come available. This kind of data could be a list of constituents or alumni that never engaged with you after multiple attempts.

Warm Data Storage

Then you move into the kitchen and start packing up the plates and silverware. You find on a top shelf some fancy china that you use once or twice a year for special occasions. You don’t want to have to go somewhere else to get it, but you also don’t want to have to take up kitchen cabinet space in your new home with it (after all, that new kitchen is a bit tight on storage space, but it’s still perfect for what you want!). You carefully pack it away into a box and label it to be stored in a basement closet in your new home.

That would be similar to data that you put into warm storage. This is data you need for occasional use, but you’re OK if you have to go to a special place in your home. This may be data for annual reporting requirements. This data may be stored on platform or off, but should be accessible by the users with some additional effort on their part. It may take a few extra clicks or maybe an hour for a job to run a report. In Salesforce, this could be a list of constituents or alumni that haven’t engaged with you in several years.

Hot Data Storage

The plates and silverware that you are packing are like the data you need every day to meet the needs of your users and constituents. This is your hot data and would be on the platform so that access is immediately granted upon request. In Salesforce, this could be a list of currently engaged constituents or alumni.

Why is a Data Archival Strategy Important?

You’re probably thinking this all sounds OK, but why does it matter?

• First, you want to focus on the future. When you start to bring over cold data into your new system, it can bring with it its own set of issues. The data may not be very clean or very accurate. This can decrease user confidence that they’re using a system with good data.
• Second, more records equal increased data storage (and possibly related files being stored, too). If data storage is a concern, having an archiving strategy for those cold records can reduce your needs.
• Third, privacy regulations are becoming more important every day. Having ways to limit access to colder data, and having ways to log who is requesting it, can only help with any privacy regulation compliance.
• Fourth, more records and high data volume can lead to slower system performance. Users may get frustrated by the clutter and inability to quickly find what they need. Keeping only the operationally required data helps people find the necessary records quickly.

Finally, many organizations have to comply with other regulatory bodies. However, many of those regulations don’t require that the data be immediately accessible. A retention policy with retrieval mechanisms is typically sufficient.

But my users are telling me they need all of this data! What do I do?

A few thoughts here.

• Try shadowing some of the users through various work days to see if what they’re telling you is accurate.
• Look for reports in your old system to see if what the users are telling you matches what reports are in use.
• You may get lucky and there are analytics features available that you can use to check usage, too.

At the end of the day, you want to bring over the data needed to be fully operational on Day 1 and archive the data that could “get in the way.” And remember, if it’s found later that some data is needed in the hot category, you can always migrate it!

This blog is part one of a three-part series on Data Archival Strategies, and part of our larger “Ask an Architect” content series. For more technical inspiration from Salesforce.org Architects, read the rest of our Ask an Architect blog series.

To learn more about engaging a Salesforce.org Customer Success Architect in your organization, please contact your Account Executive.

About the Author

Marie van Roekel is a Principal Customer Success Architect at Salesforce.org, based in the Netherlands. As part of the Advisory team, she works primarily with our Nonprofit customers to ensure they are successful in their implementations of Salesforce technologies and best practices, enabling them to better achieve their mission.