November 26, 2020

How to master microservices data architecture design

Optimizing microservices applications for data management takes the right combination of application design and database technology. It isn't a matter of simply choosing one database model over another, or placing a database in some external container. Instead, it comes down to staying focused on a set of database attributes known as ACID: atomicity, consistency, isolation and durability. Atomicity dictates that database operations should never be left partially complete: It either happens, or it doesn't. These operations shouldn't be parsed or broken out into smaller sets of independent tasks. Consistency means that the database never violates the rules that govern how it handles failures. For example, if a multistep change fails halfway through execution, the database must always roll back the operation completely to avoid retaining inaccurate data. Isolation is the principle that every single database transaction should operate without relying on or affecting the others. This allows the database to consistently accommodate multiple operations at once while still keeping its own failures contained. Durability is another word for a database's resilience. Architects should always plan for failure and disruptions by implementing the appropriate rollback mechanisms, remaining mindful of couplings, and regularly testing the database's response to certain failures.

Building a Self-Service Cloud Services Brokerage at Scale

The concepts and architecture behind a cloud brokerage are continually evolving. In a recent cloud brokerage survey on pulse.qa, an online community of IT executives, 29 % of the respondents answered that they outsourced the development of their cloud brokerage to a regional systems integrator (SI) or professional services firms. More interesting, is that 56% of the respondents built and launched their brokerage using a hybrid team of their own staff and expert outside contractors. When choosing a third-party SI or professional services firm, look for a provider with experience building brokerages for other customers like your organization. You should also investigate their strategic alliances with the CSPs and tools providers your organization requires in your brokerage. When it comes to expert outside contractors, the same rules apply. You might get lucky with finding such highly skilled contractors through contingent staffing firms – the so-called body shops – if you’re willing to go through enough resumes. However, when finding contractors for your cloud brokerage you’ll probably need to exercise your own team member’s professional networks to find the right caliber of cloud contractor.

The Future of Developer Careers

As a developer in a world with frequent deploys, the first few things I want to know about a production issue are: When did it start happening? Which build is, or was, live? Which code changes were new at that time? And is there anything special about the conditions under which my code is running? The ability to correlate some signal to a specific build or code release is table stakes for developers looking to grok production. Not coincidentally, “build ID” is precisely the sort of “unbounded source” of metadata that traditional monitoring tools warn against including. In metrics-based monitoring systems, doing so commits to an infinitely increasing set of metrics captured, negatively impacting the performance of that monitoring system AND with the added “benefit” of paying your monitoring vendor substantially more for it. Feature flags — and the combinatorial explosion of possible parameters when multiple live feature flags intersect — throw additional wrenches into answering Question 1. And yet, feature flags are here to stay; so our tooling and techniques simply have to level up to support this more flexibly defined world. ... A developer approach to debugging prod means being able to isolate the impact of the code by endpoint, by function, by payload type, by response status, or by any other arbitrary metadata used to define a test case. 

Brain researchers get NVMe-over-RoCE for super-fast HPC storage

NVMe-over-fabrics is a storage protocol that allows NVMe solid-state drives (SSDs) to be treated as extensions of non-volatile memory connected via the server PCIe bus. It does away with the SCSI protocol as an intermediate layer, which tends to form a bottleneck, and so allows for flow rates several times faster compared to a traditionally connected array. NVMe using RoCE is an implementation of NVMe-over-Fabrics that uses pretty much standard Ethernet cables and switches. The benefit here is that this is an already-deployed infrastructure in a lot of office buildings. NVMe-over-RoCE doesn’t make use of TCP/IP layers. That’s distinct from NVMe-over-TCP, which is a little less performant and doesn’t allow for storage and network traffic to pass across the same connections. “At first, we could connect OpenFlex via network equipment that we had in place, which was 10Gbps. But it was getting old, so in a fairly short time we moved to 100Gbps, which allowed OpenFlex to flex its muscles,” says Vidal. ICM verified the feasibility of the deployment with its integration partner 2CRSi, which came up with the idea of implementing OpenFlex like a SAN in which the capacity would appear local to each workstation.

Understanding Zapier, the workflow automation platform for business

In addition to using Zapier to connect workflows, companies have turned to it for help during the COVID-19 pandemic. Foster said his company has helped smaller firms move their business online quickly, connecting and updating various applications such as CRM records. “Many small business owners don’t have the technical expertise or someone on staff that can build these sites for them,” he said. “So they turn to no-code tools to create professional websites, and built automations with Zapier to reach new customers, manage inventory, and ensure leads didn’t slip through the cracks.” Saving employees time spent on repetitive tasks is a common benefit, said Andrew Davison, founder of Luhhu, a UK-based workflow automation consultancy and Zapier expert. He pointed to the amount of time wasted when workers have to key in the same data in different systems; that situation is only getting worse as businesses rely on more and more apps. “Zapier can eliminate this, meaning staffing costs can be reduced outright, or staff can be redeployed to more meaningful, growth-orientated work,” he said. “And human error with data entry is avoided — which can definitely be an important thing for some businesses in sensitive areas — like legal, for example.”

Microsoft's low-code tools: Now everyone can be a developer

Microsoft's new wave of low- and no-code tools in the Power Platform builds on this, providing tooling for UI construction, for business process automation, and for working with data. This fits in well with the current demographic shifts, with new entrant workers coming from the generation that grew up with open-world building games like Minecraft. Low-code tools might not look like Minecraft worlds, but they give users the same freedom to construct a work environment. There's a lot of demand, as Charles Lamanna, Microsoft CVP, Low Code Application Platform, notes: "Over 500 million new apps will be built during the next five years, which is more than all the apps built in the last 40 years." Most of those apps need to be low-code, as there's more than an app gap -- there's also a developer gap, as there's more demand for applications than there are developers to build that code. Much of that demand is being driven by a rapid, unexpected, digital transformation. People who suddenly find themselves working from home and outside the normal office environment need new tools to help manage what were often manual business processes. The asynchronous nature of modern business makes no-code tooling an easy way of delivering these new applications, as Lamanna notes: "It's kind of come into its own over the last year with the fastest period of adoption we've ever seen across the board from like a usage point of view, and that's just because of all these trends are coming to a head right now."

Read more here ...