Convert any data into a CSV and upload directly to an Azure Blob Container
hanss @ Adobe Stock

Convert any data into a CSV and upload directly to an Azure Blob Container

Especially with medium-sized and large companies, you often have to do it:

Transfer data to a CSV format for other systems to import. It's easy with the help of a useful library. You can find out how it works here.

01. Convert "any" object to CSV first

To convert any .NET type (which should contain only primitive types such as int, string, double, DateTime) into a CSV, first load the NuGet package "CsvHelper" into your .NET library:

Via Package Manager Console

PM> Install-Package CsvHelper

Via .NET CLI Console

> dotnet add package CsvHelper


02. Upload CSV to an Azure Blob Container

When that's done, you can create a csv using the following code and save it directly to an Azure Blob container:

using CsvHelper;
using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.Auth;
using Microsoft.WindowsAzure.Storage.Blob;
using System;
using System.Collections.Generic;
using System.IO;
using System.Threading.Tasks;

namespace ConvertObjectToCsv
{
    class Program
    {
        const string STORAGE_ACCOUNT_NAME = "yourstorageaccountname";
        const string STORAGE_PRIMARY_KEY = "yourstorageaccountname";
        const string STORAGE_CONTAINER_NAME = "yourstorageaccountname";

        static void Main(string[] args)
        {
            DoCsvOperations();

            Console.WriteLine("Hello World!");

            Console.ReadLine();
        }


        private static async void DoCsvOperations()
        {
            List<Stats> stats = new List<Stats>();

            Stats stat1 = new Stats() { DateTime = DateTime.Now, Status = "1", Timestamp = 22154654 };
            Stats stat2 = new Stats() { DateTime = DateTime.Now, Status = "2", Timestamp = 22154654 };
            Stats stat3 = new Stats() { DateTime = DateTime.Now, Status = "3", Timestamp = 22154654 };
            Stats stat4 = new Stats() { DateTime = DateTime.Now, Status = "4", Timestamp = 22154654 };
            stats.Add(stat1);
            stats.Add(stat2);
            stats.Add(stat3);
            stats.Add(stat4);

            using (var writer = new StringWriter())
            using (var csv = new CsvWriter(writer))
            {
                csv.WriteRecords(stats);
                string csvContent = writer.ToString();

                Stream streamToUploadToBlob = GenerateStreamFromString(csvContent);
                await UploadFile("file.csv", "", DateTime.Now, streamToUploadToBlob);
            }
        }

        static CloudStorageAccount StorageAccount { get; set; }
        static CloudBlobClient BlobClient { get; set; }
        static CloudBlobContainer CloudBlobContainer { get; set; }

        public static async Task<string> UploadFile(string fileName, string originFileName, DateTime date, Stream fileStream)
        {
            try
            {
                StorageAccount = new CloudStorageAccount(
                    new StorageCredentials(
                        STORAGE_ACCOUNT_NAME,
                        STORAGE_PRIMARY_KEY), true);

                BlobClient = StorageAccount.CreateCloudBlobClient();
                CloudBlobContainer = BlobClient.GetContainerReference(STORAGE_CONTAINER_NAME);

                fileStream.Position = 0;
                CloudBlockBlob cloudBlockBlob = CloudBlobContainer.GetBlockBlobReference(fileName);
                cloudBlockBlob.Metadata.Add("DateTime", date.ToString());
                if (!string.IsNullOrWhiteSpace(originFileName)) cloudBlockBlob.Metadata.Add("OriginFileName", originFileName);
                using (StreamReader str = new StreamReader(fileStream))
                {
                    var result = await str.ReadToEndAsync();
                    await cloudBlockBlob.UploadTextAsync(result);
                }

                return cloudBlockBlob.Uri.ToString();
            }
            catch
            {
                return null;
            }
        }

        public static Stream GenerateStreamFromString(string s)
        {
            var stream = new MemoryStream();
            var writer = new StreamWriter(stream);
            writer.Write(s);
            writer.Flush();
            stream.Position = 0;
            return stream;
        }
    }

    public class Stats
    {
        public string Status { get; set; }
        public int Timestamp { get; set; }
        public DateTime DateTime { get; set; }
    }
}

Checkout the Code Snippet also here:


PAUL DENNIS GEORGE

Cloud Integration Developer at Silver Chain Group

2 年

Hi Stephan, I have a CSV file which has 100's of line of data. My requirement is for each customer there are many lines and there are many customers. So basically, one header for each customer and then lines under that header. Then the next customer + lines of data under that. Would you have any solution using csv helper for that ?

回复

Thank you for sharing this. Is it possible to use Stream stream = await blockBlobClient.OpenWriteAsync(true)

回复

要查看或添加评论,请登录

Stephan Bail的更多文章

社区洞察

其他会员也浏览了