Using the Flexible File System Task with Local Storage

Using the Flexible File System Task with Local Storage

John Miner John Miner

John Miner

Data Architect at Insight

发布日期: 2020年4月3日
+ 关注

There are many existing on premises SSIS packages that use the File System Task to copy and delete files. For instance, an ingestion process might save files to an inbound directory. After processing the files, a cleanup process might timestamp the files and move the files to an archive directory. Reprocessing can be done by moving files from the archive to inbound directory.

If your company is thinking of lifting and shifting these packages to Azure, how can you replicate these file management tasks using SSIS?

Microsoft has been supplying the SSIS developer with a set of tools for Azure since 2012. The most recent Azure Feature Pack for Visual Studio 2019 was released to general availability on November 5, 2019. 

Most systems designed for Azure use two types of storage: Blob Storage and Data Lake Storage. The Flexible File Task is the next evolution in managing files regardless where they are stored. This control will work with local, blob and data lake storage systems.

In this weeks MS SQL Tips article, I go over how to use the new control with local storage.


要查看或添加评论，请登录

John Miner的更多文章

  • Why use Tally Tables in the Fabric Warehouse?
    2025年2月26日

    Why use Tally Tables in the Fabric Warehouse?

    Technical Problem Did you know that Edgar F. Codd is considered the father of the relational model that is used by most…

  • Streaming Data with Azure Databricks
    2025年2月25日

    Streaming Data with Azure Databricks

    Technical Problem The core functionality of Apache Spark has support for structured streaming using either a batch or a…

    1 条评论
  • Upcoming Fabric Webinars from Insight
    2025年2月19日

    Upcoming Fabric Webinars from Insight

    Don't miss the opportunity to boost your data skills with Insight and Microsoft. This webinar series will help you…

  • How to develop solutions with Fabric Data Warehouse?
    2025年2月18日

    How to develop solutions with Fabric Data Warehouse?

    Technology Details The SQL endpoint of the Fabric Data Warehouse allows programs to read from and write to tables. The…

  • Understanding file formats within the Fabric Lakehouse
    2025年2月10日

    Understanding file formats within the Fabric Lakehouse

    I am looking forward to talking to the Cloud Data Driven user group on March 13th. You can find all the presentation…

    3 条评论
  • Engineering a Lakehouse with Azure Databricks with Spark Dataframes
    2025年2月3日

    Engineering a Lakehouse with Azure Databricks with Spark Dataframes

    Problem Time does surely fly. I remember when Databricks was released to general availability in Azure in March 2018.

  • Create an Azure Databricks SQL Warehouse
    2025年1月21日

    Create an Azure Databricks SQL Warehouse

    Problem Many companies are leveraging data lakes to manage both structured and unstructured data. However, not all…

    2 条评论
  • How to Load a Fabric Warehouse?
    2025年1月9日

    How to Load a Fabric Warehouse?

    Technology The data warehouse in Microsoft Fabric was re-written to use One Lake storage. This means each and every…

  • My Year End Wrap Up for 2024
    2024年12月26日

    My Year End Wrap Up for 2024

    Hi Folks, It has been a very busy year. At the start of this year I wanted to learn Fabric in depth.

    1 条评论
  • Virtualizing GCP data with Fabric Shortcuts
    2024年12月16日

    Virtualizing GCP data with Fabric Shortcuts

    New Technology Before the invention of shortcuts in Microsoft Fabric, big data engineers had to create pipelines to…

社区洞察

其他会员也浏览了