Microsoft recently announced several new features for Microsoft Fabric to coincide with the first annual Microsoft Fabric Community Conference held in Las Vegas over the last week of March. There are far too many to cover them all, but this will highlight some of the most impactful features. You can check the Microsoft Fabric Blog for more details on the items covered and to see what else has been added.
Folders
Fabric has added the ability to create folders within a workspace in public preview. This will make it easier to organize your workspaces and find the items you are looking for. Folders are supported by deployment pipelines, but they are not yet supported by Git integration. See Create folders in workspaces (preview) - Microsoft Fabric | Microsoft Learn for more information.
Dataflow Gen 2 Fast Copy
Dataflow Gen 2 provides a low code approach to ingest and transform data. The dataflow Gen2 fast copy capability allows you to ingest large files or database tables much faster. Files must be at least 100 MB and for Azure SQL DB the data source must be at least 1 million rows. Fast Copy will let you ingest terabytes of data with a low code experience.
Gateway Support for Fabric Data Pipelines
Gateway support for Fabric Data Pipelines is in public preview. This allows you to easily connect to on-premises data sources and copy data into Fabric. Prior to this, you could use the gateways with Dataflow Gen2 but not with Fabric Data Factory data pipelines. However, Dataflow Gen2 has some limitations such as lack of support for incremental data loading. The on-premises gateway support allows you to use Fabric pipelines with connectors for SQL Server as well as Azure, Amazon, and Google sources that are not accessible from the public Internet. See How to access on-premises data sources in Data Factory - Microsoft Fabric | Microsoft Learn for instructions and Data pipeline connectors in Microsoft Fabric - Microsoft Fabric | Microsoft Learn for a full list of supported data sources.
Mirroring
Mirroring in Fabric is now in public preview. Without mirroring, you would need to develop a process to copy source data and keep your Fabric data in sync as changes are made. Mirroring eliminates the development effort and simply lets you select the tables you want to synchronize to Fabric. You can then create near real-time reports directly using the mirrored data, further transform and combine it with other data in Fabric, or use Spark notebooks and their data science capabilities to analyze the data.
Mirroring from the following databases is available in preview, with support for additional databases on the roadmap:
- Azure SQL Database
- Snowflake
- Azure Cosmos DB
The storage used by the mirrored database is free, up to the number of terabytes of the capacity. For example, an F64 capacity includes 64 TB of storage for mirroring.
Git Integration
As Power BI has evolved from a self-service and departmental reporting tool to Fabric, an enterprise-wide analytics, data engineering and data science platform, managing the development lifecycle has become critical. Fabric has supported integration with Git since the May 2023 Preview release of Fabric, but only for a limited number of types of Fabric items. Now additional object types including data pipelines, lakehouses and notebooks are supported by Git integration. Using Git integration along with Fabric deployment pipelines provides a complete solution for version control, collaborative development, and isolation of non-production environments from production.
Conclusion
Microsoft is continuing to enhance Fabric to support its vision of an intelligent data platform. New capabilities like on-premise gateway support, mirroring and fast copy reduce the friction and effort needed to analyze data wherever it may be in your data estate. Folders make it easier to keep organized and find the assets you need. Git integration allows you to effectively management your development and release processes.
AI Copilot experiences are infused throughout the process; the best way to understand it is to see it in action, so check out the video “Unleashing Innovation: Exploring Microsoft's Fabric Copilot Webinar Recording” on our site here.
If your company wants to learn more about Microsoft Fabric, do not hesitate to reach out to Spyglass to help develop an approach to successfully implement Fabric.