Architecture Weekly #179 - 13th May 2024

Welcome to the new week!

Last week, I covered the latest improvements to Java 22 around pattern matching and records . They enable explicit business logic modelling, making it concise and guarded by the compiler. As usual, I put it into the context of Event Sourcing.

This time, I followed up on how to write a left-fold streams collector in Java to roll your events into state! This is also applicable for regular map/reduce cases when you need to reduce a collection into a single item.

• Oskar Dudycz - How to write a left-fold streams collector in Java

Speaking about Event Sourcing, a humble reminder that in 2 weeks, the DDD Europe conference starts. If you’d like to prepare for going into production with your event-sourced system or validate your assumptions, check my pre-conference Production Grade Event Sourcing workshop. See more here:

• DDD Europe - 27th-29th of May - Production-Grade Event Sourcing: Modelling, DevOps, Process.

I aim to provide you with collaborative workshops where you can gain certainty if what you designed for prod will work and you won't have nasty surprises.

I want to pass you a condensed dose of experience I gathered throughout my career. It’s also a rare chance, as I’m not giving public workshops often.

Also, I decided to give each attendee a half-year subscription to Architecture Weekly. If you join, you'll get access to over 20 hours of recordings, which should be a decent follow-up. See the full list of them .

Let’s continue the Event-Driven Architecture theme. Handling errors in event-driven systems is non-trivial (but show me where that’s easy!). There are already established patterns, but they’re not widely known among the audience, and tools do not always provide them out of the box. Adam Bellemare provided a thorough article going through the possible cases and how to deal with them:

• Adam Bellemare - Preventing and Fixing Bad Data in Event Streams — Part 1

I also found an interesting tool, PGMQ, a plugin for Postgres that allows us to use it as a simple messaging system. What’s more, the API and guarantees are compatible with Amazon SQS.

• Postgres Message Queue (PGMQ) - A lightweight message queue. Like AWS SQS and RSMQ but on Postgres

I haven’t tried it yet, so please don’t take this as an encouragement to replace Amazon SQS and go with it to production, but I like the idea. For many, simple solutions can be more than enough without introducing additional complexity (my first thought is the local IoT solutions).

AWS Lambda is the most well-known serverless tool. Still, some areas are confusing and not covered enough, such as cold starts. Many people want to eat cookies and have cookies. So, have pay-as-you-go flexibility and scalability but have the performance and fine-tuning in the regular systems. As always, the magic is that there’s no magic.

Because we’d like to pay for the consumed resources, then they shouldn’t be consumed if we don’t. Our functions are “put to sleep” if they’re not used for some time and need to be woken up. This waking up is called “cold start” as it takes more time than regular call.

Mohit Palriwal provided good coverage explaining common myths and misconceptions.

• Mohit Palriwal - Unraveling the Enigma: Debunking Myths Surrounding Lambda Cold Starts

Before starting to use serverless architecture, we should understand the consequences. If we constantly need to have our services warmed up, we should do a sanity check and verify if a containerised solution running constantly isn’t the best option. Why? Because that probably means that we have a constant load.

AWS also released a new guidance on the usage of their DocumentDB. It explains both basic strategies around schema management and modelling. It’s a nice primer for people using it and the original database they cloned: MongoDB (and other document databases). It’s not extremely deep and thorough, but it can be a decent starting point for understanding that:

• document databases are not freehand drawings,
• they have structured data,
• they have a schema,
• you can model a relationship with them,
• …but differently in the relational approach.

Read more:

• AWS - Data modeling with Amazon DocumentDB

If you’re considering using it, check my article: General strategy for migrating relational data to document-based .

It’s been a while since I mentioned WebAsssembly, but my interest is ongoing. I still believe that it’s the future of backend processing. It’s getting easier and easier to run WebAssembly as the computation engine. Read in the Docker-own coverage:

• Docker - Wasm vs. Docker: Performant, Secure, and Versatile Containers

Matt Butcher explained how we can integrate multiple languages in a single environment. Introducing interfaces and modules makes that easier than before. Also, improvements in the ahead-of-time compilation in JVM and .NET areas make that part easier. Step by step, we’re getting to the place where WASM can be a secure way to integrate various dev environments:

• Matt Butcher - Polyglot Programming with WebAssembly: A Practical Approach

And hey, you can even write VSCode plugins with WASM:

• VSCode - Using WebAssembly for Extension Development

The loudest event in the developer community was a Marvel+DC universum-like collaboration. OpenAI started to work with StackOverflow hand-in-hand, or more api-to-api:

• OpenAI - API Partnership with Stack Overflow

For StackOverflow, it’s a swan song; they’re just trying to stay afloat by monetising the leftovers they had. For OpenAI, that’s a chance to get easier recent answers that don’t exist in the pre-trained models. Is it also a swan song for OpenAI? Maybe. I think they realise that what they trained so far is good enough (?) for now. Yet, in the future, they need user, “expert” data input, or they will be cannibalizing themselves by training themselves on the generated by themselves data. Well, time to get more popcorn. Be quick and check how people reacted to it, and then moronic StackOverflow answer to their reactions:

• The Register - Stack Overflow simply bans folks who don't want their advice used to train AI

You see, I’m sceptical about the Generative AI business model. I think that they don’t have any. Still, I appreciate the mature approaches and the help we can get from machine learning. A great example of how to explain how they work in this article:

• Jay Alammar - The Illustrated Word2vec

If you like the good coverage, then TimescaleDB did a good example explaining Open Telemetry metrics in detail:

• Timescale - A Deep Dive Into OpenTelemetry Metrics

If you’re looking for something entertaining but still contentful, check out the great talk by Dean Schuster. I saw it live at last Build Stuff, and it was the best talk I’ve seen there!

• Dean Schuster - The Crisps and Pickle Story: What's Really Behind Infamous, Historic UI Failures?

Check also other links!

Cheers

Oskar

p.s. I invite you to join the paid version of Architecture Weekly. It already contains the exclusive Discord channel for subscribers (and my GitHub sponsors ), monthly webinars , etc. It is a vibrant space for knowledge sharing. Don’t wait to be a part of it!

p.s.2. Ukraine is still under brutal Russian invasion. A lot of Ukrainian people are hurt, without shelter and need help. You can help in various ways, for instance, directly helping refugees, spreading awareness, and putting pressure on your local government or companies. You can also support Ukraine by donating, e.g. to the Ukraine humanitarian organisation, Ambulances for Ukraine or Red Cross .