Last year approximately 4,000 people attempted to thru-hike the Appalachian Trail, a 2,190 mile trail reaching across the east coast from Georgia to Maine in the United States. Northbound hikers (the most common form of thru-hikers on the A.T.) start at Springer Mountain and hike for an average of five to seven months before finishing at Mt. Katahdin in Maine. During their time on the trail, hikers face both mental and physical challenges, which can impact their success. This is why only 1,128 out of 4,000 people completed the journey last year. While these are shocking statistics, it’s important to note that the hikers who did succeed came prepared. They trained by going on day hikes, laid the groundwork through research and experimentation, and packed effectively and efficiently.
Preparation is essential on any journey — in the woods or in the office. Large companies increase their chances of a successful cloud migration when they identify their needs through research and analysis ahead of time. Employees must be prepared for any and all changes, crises, and successes that may arise on their data journey. So, while I recognize that most CDOs are probably not gearing up today (or in the near future) to hike 2,000 miles from Georgia to Maine, many CDOs are preparing for an equally transformative journey within their organizations — the journey to the cloud.
Why make the move to the cloud?
Today, many companies are looking to move to the cloud due to its exponential benefits. These benefits include:
- Scalability. The cloud can support larger workloads and more users than on-premise
- Cost. The cloud reduces operational costs such as updates and maintenance by automatically making these changes without IT
- Performance. The cloud enables companies to improve performance and therefore, overall customer experience
- Flexibility. Unlike on-prem, which lives at a single site, the cloud can be accessed anywhere so users always have the data they need when they need it
How to start your cloud migration journey?
The ultimate goal of any data-driven company on a cloud migration journey is to move trusted and governed data to the cloud. While this sounds easy, it’s actually rather challenging. Large enterprises have a substantial amount of data stored in their on-premise data warehouses. Moving all this data to the cloud can be time consuming and eat up resources. It can also create data swamps if all sorts of data is sitting in the cloud warehouse, data that might be stale or redundant. So how should companies embark on this journey?
The answer? Companies must pay more attention to understanding where data is at every point in time, who has access to it, and how it is used. This is important because they must know which data can be trusted, which data is actually used, and which data is compliant before transferring their data into the cloud. To achieve this, companies must turn to data lineage for help.
Collibra Lineage provides valuable context surrounding your data. It helps you take inventory of your data and determine which data should be moved to the cloud. Our technical lineage view gives you visibility into the source of your data so you can determine its accuracy. More specifically, our detailed technical lineage view allows you to see transformations, drill down into table, column, and query-level lineage, and navigate through your data pipelines, thus giving you the full context of your data. Knowing the full context of your data (the source, the transformations etc.) is crucial for cloud migration because you only want to migrate over accurate and complete data to the cloud. Luckily, with technical lineage, you can see the history of your data and even look into which data was used to build your on-premise data warehouse. You can then dig into this data to see how often the data has been used and its accuracy to ensure you are moving only valuable data to the cloud. With our governance capabilities, you can easily get more information about your data by going right to the data owner if you have additional questions. In addition, after you make the move to the cloud, you can use data lineage to check your data’s integrity. By viewing the technical lineage diagram post transfer, you can make sure your data is intact and secure and was not leaked during the move.
Getting to the top of the mountain with data lineage
On your cloud migration journey you will face challenges. Like a thru-hiker, you will face peaks that seem insurmountable, obstacles that slow you down, and overall moments of pain and frustration. These moments can be discouraging and can lead to failure. In fact, based on research from analyst reports, the success rate is rather slim. However, in order to be successful, you must bring only the essentials on your journey. As a CDO you do not need to worry about fitting food, shelter and warmth in a small pack on your back, but you do need to worry about creating a data-driven company grounded in accurate and trustworthy data. Investing in data lineage guarantees you are bringing the right data from on-premise to the cloud.
Collibra Lineage ensures you are prepared to take on the long and daunting journey of migrating to the cloud by providing visibility into and context about your data. With this greater understanding of your data, you can be confident that your cloud is populated with accurate, trustworthy and relevant data to drive optimal business decisions.
