As organizations transition to Microsoft Fabric, they gain access to a powerful suite of integrated analytics tools designed to streamline data interaction. However, one major challenge arises: How do you connect your existing databases to Fabric?
Fabric provides several methods for accessing external data seamlessly, with shortcuts and mirroring being two key approaches. These approaches allow users to work with external data sources directly within Fabric, maintaining performance, security, and accessibility.
In this post, we’ll explore how shortcuts and mirroring work, their differences, and how to determine which method best suits your business needs.
Shortcuts
Rather than moving or duplicating data, shortcuts provide a way to reference external storage locations directly from OneLake. They act as virtual links, making it possible to interact with data stored in other services as if it were natively available in Fabric.
How Shortcuts Work
A shortcut consists of two key components:
- Target Path: The actual location where the data resides.
- Shortcut Path: The location within OneLake where the shortcut is created.
Shortcuts function like symbolic links in a file system—they point to data without moving or copying it. Any Fabric-compatible service, such as Power BI, Apache Spark, or SQL, can interact with shortcut-linked data seamlessly.
Supported Data Sources for Shortcuts
OneLake shortcuts support multiple filesystem data sources, including:
- Internal OneLake locations
- Azure Data Lake Storage (ADLS) Gen2
- Amazon S3
- S3 Compatible Storage
- Google Cloud Storage (GCS)
- Dataverse
This broad compatibility ensures that data from different platforms can be accessed without requiring migration or duplication, keeping storage costs low while maintaining real-time connectivity.
Why Use Shortcuts?
Shortcuts are a great fit for scenarios where:
- You need quick access to external data without importing it into Fabric.
- Your data is frequently updated at the source, and you want to avoid maintaining multiple copies.
- You want to optimize storage costs by keeping data in its original system.
- You are working across multiple cloud platforms and need seamless access.
Since shortcuts integrate natively with OneLake, they ensure that users and analytical tools can work with external data effortlessly, maintaining a unified namespace across platforms.
Mirroring
Unlike shortcuts, mirroring brings a continuously synchronized copy of your external data into Fabric’s OneLake, ensuring Fabric has its own managed version of the data.
Mirroring allows Fabric to store and process external data directly within OneLake, reducing dependency on external systems and improving performance for querying and analytics workloads.
How Mirroring Works
Mirroring provides a near real-time sync between OneLake and the original database. This allows businesses to:
- Access a synchronized copy of their external data within Fabric.
- Improve query performance by keeping data local to Fabric’s computing environment.
- Reduce dependence on external systems, making analytics more reliable and efficient.
Types of Mirroring in Fabric
Fabric provides three distinct types of mirroring, each suited for different data integration needs:
- Database Mirroring: With database mirroring, entire databases and tables are replicated in OneLake, ensuring Fabric has a fully synchronized version of the original data.
- Metadata Mirroring: Metadata mirroring does not copy actual data into Fabric but instead synchronizes metadata elements such as schemas, table structures, and catalog names. This method keeps the data in its original storage location while allowing Fabric to access and query it through shortcuts.
- Open Mirroring: Open mirroring leverages the Delta Lake table format to allow developers to write change data directly into a mirrored database item in Fabric. This approach provides flexibility for custom applications that need direct integration with Fabric’s analytics platform.
Why Use Mirroring?
Mirroring is a great fit for scenarios where:
- You need Fabric to have a synchronized, query-ready copy of your data.
- Your external system may experience downtime, and you want Fabric to remain independent of its availability.
- Your analytics workloads require reliable and high-speed data access.
Since mirrored data is physically stored in OneLake, Fabric can perform high-speed analytics and reporting without querying the external system directly.
Shortcuts vs. Mirroring: Which One Should You Use?
Both shortcuts and mirroring offer ways to seamlessly integrate external data into Fabric, but they serve different purposes.
Use shortcuts when:
- You want a lightweight, storage-efficient way to access external data.
- The data is frequently updated and should always be retrieved live from the source.
- You need to work with large datasets stored across multiple cloud platforms.
Use mirroring when:
- You need Fabric to have a synchronized, queryable copy of your external data.
- Your queries require consistent performance without reliance on an external system.
- You need to ensure data availability for analytics even when the source system is offline or undergoing maintenance.
By leveraging both approaches strategically, businesses can create a highly efficient, scalable, and cost-effective data ecosystem in Fabric.
Final Thoughts
Shortcuts and mirroring are two powerful ways to bridge external data with Microsoft Fabric. Whether you need on-demand access without data movement or a fully replicated copy for in-depth analysis, Fabric offers a flexible solution.
By understanding the differences, benefits, and trade-offs of each approach, organizations can build a seamless data integration strategy that aligns with their analytics needs. The key is to evaluate your data usage patterns, performance requirements, and storage constraints.
For businesses that require real-time access without duplication, shortcuts offer a lightweight, efficient solution. Meanwhile, mirroring provides the advantage of high-performance querying and independence from external system availability. A hybrid approach, leveraging both shortcuts and mirroring, can maximize efficiency, providing the best of both worlds.
As Microsoft Fabric continues to evolve, its data integration capabilities will only grow stronger. Organizations that strategically adopt these features can stay ahead in the ever-changing data landscape, ensuring that their analytics environment is optimized for both agility and performance.
Sources:
- Microsoft Learn: OneLake shortcuts
- Microsoft Learn: What is Mirroring in Fabric?
Leave a Reply