登录查看更多内容

Sync SQLServer table to Apache Doris An Open Source, Real-Time Data Warehouse - Part 1

Michael Olayemi Olawepo

Software Test automation | Software engineering Management | Scrum | Data engineering| Rapid Application Development | Low Code Tools | Open Source

发布日期: 2024年4月2日

In 2023, I have a few draft posts I didnt have much time to finish, this is one of them. This is part of my learning and personal POCs, I will try to post them in parts through this year.

In the last 2–3 years I have in been interested in opensource tools for data engineering mostly; Databases, ETL tools and analytics. One of my goals is to investigate tools that can augument legacy sqlserver DB to handle various analytics needs and workload. Few of the topics of interest for me are data synchronization ( batch and streaming), change data capture (CDC), analytics dashboard and securely sharing data with 3rd parties.

While researching I found Apache Doris and StarRocks, starocks is a fork from Doris three years ago. However in this post I will be exploring Apache Doris.

要查看或添加评论，请登录

Michael Olayemi Olawepo的更多文章

Introducing Obot: An Open-Source AI Agent Platform (Part 2)

2025年3月23日

Introducing Obot: An Open-Source AI Agent Platform (Part 2)

Chat with Multiple Databases: Oracle, Postgres, MySQL, Microsoft SQL Server In Part 1 of this series, we introduced…
Introducing Obot: An Open-Source AI Agent Platform (Part 1)

2025年3月20日

Introducing Obot: An Open-Source AI Agent Platform (Part 1)

Recently I had the opportunity to explore Obot, an open-source AI agent platform that’s changing the game for building…
Run Large Language Model in docker — AI on Your Local Environment.

2025年1月9日

Run Large Language Model in docker — AI on Your Local Environment.

Introduction Last year, at the Collision Conference, I had a thought-provoking discussion with a friend from the Czech…
Securing a Data Service API with Keycloak — Identity and Access Management (Part 2)

2024年12月25日

Securing a Data Service API with Keycloak — Identity and Access Management (Part 2)

Summary of Part 1 In the previous article, we explored Keycloak’s identity and access management capabilities, focusing…
Securing a Data Service API with Keycloak: Identity and Access Management (Part 1)

2024年12月11日

Securing a Data Service API with Keycloak: Identity and Access Management (Part 1)

KeyCloakDemo In this article, we will explore how to set up Keycloak for user management and JWT token issuance for…
Mobile Test Automation Setup using Robotframework & Appium

2024年7月30日

Mobile Test Automation Setup using Robotframework & Appium

In my last post here I talked about possibility of getting to Prague on time for a cold beer, I made it onetime :-)…
Resty bash — fetching business data with bash. How I made my first $40 freelancing.

2024年7月9日

Resty bash — fetching business data with bash. How I made my first $40 freelancing.

It is summer and I have a few more draft posts I never posted, so I will continue to try and post them, here is one of…
Function as a service ( faas) example with kubernetes, fission and rabbitmq

2024年3月31日

Function as a service ( faas) example with kubernetes, fission and rabbitmq

In the last few decades, we have seen evolution of how services are deployed from Monoliths to microservices and…
Data Processing with Apache Pulsar and Apache Nifi (A gradual journey to exploring the FLIP* stacks)

2022年10月12日

Data Processing with Apache Pulsar and Apache Nifi (A gradual journey to exploring the FLIP* stacks)

I recently read about the release of Apache Nifi and Apache pulsar connector on this page here , As part of my R&D…
Open Application Model (OAM) — Bringing some sanity to application deployment on kubernetes (k8s)

2022年4月18日

Open Application Model (OAM) — Bringing some sanity to application deployment on kubernetes (k8s)

Virtual machines, docker, vagrant, severless, Faas platforms, I thought they are cool and powerful and in some ways…

See all articles

Sync SQLServer table to Apache Doris An Open Source, Real-Time Data Warehouse - Part 1

Michael Olayemi Olawepo

Software Test automation | Software engineering Management | Scrum | Data engineering| Rapid Application Development | Low Code Tools | Open Source

Michael Olayemi Olawepo的更多文章

社区洞察

其他会员也浏览了

Simple tools can also be powerful aka Power of Unix utilities

Understanding Database Auditing

TYPES OF GEODATABASES

Hive File Formats and Compression DataStage Jobs With Information Server Hive Connector (Part 1)

?? SQL vs MySQL: What Every Developer Should Know in 2025!

Distinct Vs Group BY in SQL

Memory & Disk Partition in Apache Spark

All Answers of SQL 2016 all in One. - 10 Great Reasons to look at SQL 2016 now.

CONNECTING FASTAPI TO POSTGRES DATABASE

Michael Olayemi Olawepo的更多文章

Introducing Obot: An Open-Source AI Agent Platform (Part 2)

Introducing Obot: An Open-Source AI Agent Platform (Part 1)

Run Large Language Model in docker — AI on Your Local Environment.

Securing a Data Service API with Keycloak — Identity and Access Management (Part 2)

Securing a Data Service API with Keycloak: Identity and Access Management (Part 1)

Mobile Test Automation Setup using Robotframework & Appium

Resty bash — fetching business data with bash. How I made my first $40 freelancing.

Function as a service ( faas) example with kubernetes, fission and rabbitmq

Data Processing with Apache Pulsar and Apache Nifi (A gradual journey to exploring the FLIP* stacks)

Open Application Model (OAM) — Bringing some sanity to application deployment on kubernetes (k8s)

社区洞察

其他会员也浏览了

Simple tools can also be powerful aka Power of Unix utilities

Understanding Database Auditing

TYPES OF GEODATABASES

Hive File Formats and Compression DataStage Jobs With Information Server Hive Connector (Part 1)

?? SQL vs MySQL: What Every Developer Should Know in 2025!

Distinct Vs Group BY in SQL

Memory & Disk Partition in Apache Spark

All Answers of SQL 2016 all in One. - 10 Great Reasons to look at SQL 2016 now.

CONNECTING FASTAPI TO POSTGRES DATABASE