Architecture Weekly #171 - 18th March 2024
Oskar Dudycz
Software Architect & Consultant / Helping fellow humans build Event-Driven systems
Welcome to the new week with the latest #ArchitectureWeekly!
I'm calling May the Event Sourcing month, as you may join in May my workshops that will speed up your journey! Interested?
Do you like onsite conferences? You can mash those two together, as I’ll be giving pre-conference workshops on my favourite conferences:
On Techorama, you'll get solid foundations. The workshop will be hands-on and teach you how to use Event Sourcing. I won’t lie: you won’t be an expert after it, but you’ll be able to start practically applying it to your projects, learn what you don’t know, and have a plan for advancing it.
On DDD Europe, we'll focus on ensuring that our design will work in production. Even when we get there, it’s worth revisiting lessons learned and discovering what we can improve in preparing for the upcoming challenges.
All the exercises will be backed by the versatile experience I got throughout my career. I’ll share my experience building systems like Marten and EventStoreDB and working as an architect and consultant in many Event Sourcing projects.
Feel free to comment or drop me a message if you'd like to learn more!
Can’t be there or want a private experience? We can also solve it when you reach me. No worries!
I also run private trainings and other forms of help like:
I’m happy to help!
If you don’t know what to choose, don’t hesitate to email me. I’m here to help. Check the details on my trainings page.
By the way. I’ll speak tomorrow at Kafka Summit, explaining the Event modelling anti-patterns. If you’re here, feel free to join the session or just chat!
It’s extremely motivating when you start a fresh open-source project and soon get such quality blog articles as Mário Bittencourt did about Emmett. I love how he phrased the intention behind it: "Just enough ceremony". I'll need to reuse it!
Besides explaining Emmett's principles, I believe that’s also a great intro to Event Sourcing. Read more:
Ok, enough about stuff I’m directly involved. Let’s do a quick recap of the links we have.
March 1989, 25 years ago, Tim Berners-Lee published his proposal on World-Wide Web. Yup, just 25 years ago. Interestingly, it was called “Information Management”. That’s what we do on Instagram, right? You can read the whole proposal here:
Don’t worry; it’s not that long and a nice read. It’s good to go back to foundations.
Figma is a popular web-based design tool enabling real-time collaborative interface design and prototyping. They faced the challenge of a PostgreSQL database growing 100x since 2020. They were pretty happy about PostgreSQL and didn’t want to reintroduce a new approach like NoSQL or even closer CockroachDB. They had to invest in horizontal scaling and sharding. They wanted to do it evolutionarily.
To do it, they used an interesting strategy. They utilized Postgres views to create logical separations of data that mimic how data would be divided across different shards. This means they could define views that only include rows from a table that match certain criteria related to sharding, such as all records belonging to a specific user group or dataset. By querying these views instead of the actual tables, the application behaves like the data is already partitioned, allowing the Figma team to test sharding logic and query routing mechanisms before implementing physical sharding.
Figma also developed DBProxy, a service to dynamically route queries to the correct database shard, addressing the challenge of query execution in a sharded environment.
You can clearly see in the last Architecture Weekly that it’s becoming a trend. Investing in proper sharding and load balancing that’s smart enough to understand the product needs is becoming critical.
Read more:
Also, Cloudflare open-sourced their Rust-based tool for networking. It also allows it to be used as a load balancer but can do more. That’s also in the spirit of the last edition links on Secure-by-design. I think that we’ll see more and more Rust tools in this space, as this security should also start with the choices of the tool authors.
领英推荐
We’re also living in interesting times for databases. Horizontal scaling may also mean putting data on the edge and treating, e.g., web or mobile applications as part of our sharding strategy. That, of course, brings other challenges like conflict resolution, but it also relieves our backend services.
See, e.g. what people can do with Databases on WASM or SQLite:
See also a nice case study on GitLab Metrics and SQLite:
Last week, I linked and discussed the cast.ai report on Kubernetes ’ overprovisioning. I commented:
Of course, remember that it’s the report prepared by the tool that is built for detecting such utilisation, so it’s in their interest to prove that. Plus, they analysed those clusters to which they had access (still, a few thousand clusters). So, as always, think for yourself.
We didn’t have to wait long for the next set of news, now from CNCF by themselves - they also published the report with similar conclusions. See more:
Are you looking for good modelling resources for this week? No worries, I've got you covered. Check:
Rafa? and Kenny discuss the details and provide heuristics for diving deeper into our model to discover important parts. Neal and the DDD Crew show how to design a proper communication workflow.
Btw. I found out that you can now use Draw.io inside Miro. For me, it’s great, as I always had to copy-paste them, dropping the quality and taking a lot of time. See:
Now, grab the popcorn:
Last month, Sainsbury’s announced its decision to use more automated tills and warehouse robots as well as AI-forecasting tools to ensure it has the right stock in stores as part of a ￡1bn cost-cutting effort over the next three years.
We didn’t have to wait long:
Sainsbury’s has resolved technical issues that led to the cancellation of online grocery deliveries and card payments in stores on Saturday.
The supermarket chain was unable to fulfil the “vast majority” of its online orders after an overnight software update led to problems that affected some stores, grocery online services and its ability to contact customers.
Who knew?
Read full coverage here:
Watch also John Oliver doing the same for Boeing:
All of that may sound funny, but it’s not. We’re creating the software that humans will use. We must ensure we’re professional and know where to cut the corners. Believing that AI will magically solve it or that we can increase the pace by dropping the quality or security is amateurship. Read more:
Check also all links at: https://www.architecture-weekly.com/p/architecture-weekly-171-18th-march
Cheers
Oskar
p.s. I invite you to join the paid version of Architecture Weekly. It already contains the exclusive Discord channel for subscribers (and my GitHub sponsors), monthly webinars, etc. It is a vibrant space for knowledge sharing. Don’t wait to be a part of it!
p.s.2. Ukraine is still under brutal Russian invasion. A lot of Ukrainian people are hurt, without shelter and need help. You can help in various ways, for instance, directly helping refugees, spreading awareness, and putting pressure on your local government or companies. You can also support Ukraine by donating, e.g. to the Ukraine humanitarian organisation, Ambulances for Ukraine or Red Cross.