S3 speed 40 Gb/sec read.
X14 Minio s3 & Spark applience.

S3 speed 40 Gb/sec read.

Nearly 40 Gigabyte/Sec sustained read.An typical enteprise S3 gives 0.5 Gigabyte/Sec. Giving a factor of 80 times "faster" backend for Delta/Iceberg storage. Earlier X14 S3/spark applienced based on PCI 4.0 gave 22.8 GB/Sec which goes in line with PCI 5.0 doubled bandwith.


An follow up on this would be an TPC-DS/H test for the following process engines Iceberg/Delta aswell as Arrow/datafusion engine. it is good its Christmas giving me time to do this :)


Original article with more details is on my X14 page


Julatouch Pipatanangura

Architect - Analytics (ML/AI, BI), Data Governance/Management, Contractor

1 年

Oh nice. Bra maskin att k?ra duckdb p?

Andreas Forssblad

Data Engineer/DevOps p? Handelsbanken

1 年

Snyggt Rickard!!

要查看或添加评论,请登录

社区洞察

其他会员也浏览了