S3 speed 40 Gb/sec read.
Nearly 40 Gigabyte/Sec sustained read.An typical enteprise S3 gives 0.5 Gigabyte/Sec. Giving a factor of 80 times "faster" backend for Delta/Iceberg storage. Earlier X14 S3/spark applienced based on PCI 4.0 gave 22.8 GB/Sec which goes in line with PCI 5.0 doubled bandwith.
An follow up on this would be an TPC-DS/H test for the following process engines Iceberg/Delta aswell as Arrow/datafusion engine. it is good its Christmas giving me time to do this :)
Original article with more details is on my X14 page
Architect - Analytics (ML/AI, BI), Data Governance/Management, Contractor
1 年Oh nice. Bra maskin att k?ra duckdb p?
Snabbt
Data Engineer/DevOps p? Handelsbanken
1 年Snyggt Rickard!!