Edition 5d: AWS Well-Architected Framework: Performance Efficiency Pillar

Performance Efficiency Pillar

??????????? The Performance Efficiency pillar encompasses the capacity to effectively use computer resources to fulfill system needs and sustain this efficiency amidst fluctuations in demand and advancements in technology.

Design Principles

??????????? The cloud has five design concepts that promote performance efficiency:

·?????? Democratize advanced technologies: Enhance the efficiency of incorporating sophisticated technologies for your team by assigning intricate jobs to your cloud vendor. Instead of requesting your IT personnel to acquire knowledge about hosting and managing a new technology, contemplate using the technology as a service. Technologies such as NoSQL databases, video transcoding, and machine learning need particular knowledge. Within the cloud, these technologies are transformed into services that can be used by your team, allowing them to concentrate on product creation rather than the allocation and administration of resources.

·?????? Go global in minutes: By distributing your workload over various AWS Regions globally, you can provide your consumers a superior experience with reduced latency and minimum expenses.

·?????? Use serverless architectures: Serverless architectures eliminate the need for individuals to operate and maintain physical servers for conventional computing tasks. For instance, serverless storage services have the capability to function as static webpages, eliminating the need for web servers. Additionally, event services have the ability to host code. By using managed services, the task of overseeing physical servers is eliminated, resulting in reduced operational responsibilities. Additionally, the use of managed services may lead to decreased transactional expenses due to their ability to function on a larger scale in the cloud.

·?????? Experiment more often: By using virtual and automatable resources, you may efficiently conduct comparative testing across various instances, storage options, and settings.

·?????? Consider mechanical sympathy: Gain a comprehensive understanding of the manner in which cloud services are used and consistently employ the technological strategy that is in line with your objectives for your job tasks. Take into account data access patterns while choosing database or storage methods.

Best Practices

??????????? The Performance Efficiency pillar encompasses five domains in which we must establish and identify best practices.

·?????? Architecture selection

·?????? Compute and hardware

·?????? Data management

·?????? Networking and content delivery

·?????? Process and culture

Architecture Selection

??????????? The most efficient strategy for a certain job might vary, and sometimes involves combining numerous techniques. Well-Architected workloads use a variety of systems and include diverse features to enhance performance.

Cloud Resource Selection

??????????? Frequently, it is necessary to use various strategies in order to achieve optimal performance across a range of tasks. Well-Architected systems use a variety of methods and features to enhance performance.

Compute and Hardware

??????????? Architectures may use diverse computational options for distinct components and enable numerous features to enhance performance. Choosing an inappropriate computing option for a given architecture might result in reduced performance efficiency.

AWS Compute

??????????? Within the AWS platform, there are three distinct options for accessing computational resources.

·?????? Instances: Instances refer to virtualized servers that can have their capabilities modified using a button or an API request. Due to the dynamic nature of resource allocation in the cloud, you have the flexibility to conduct experiments with various server types. At AWS, the virtual server instances are available in various types and sizes, providing a diverse range of features such as solid-state disks (SSDs) and graphics processing units (GPUs).

·?????? Containers: Containers are a kind of operating system virtualization that enable you to execute a program and its dependencies in separate processes that are segregated from other resources. AWS Fargate provides a serverless computing solution for containers, whereas Amazon EC2 is suitable for those who want more control over the setup, customization, and administration of their computing environment. Additionally, you have the option to choose from other container orchestration systems, such as Amazon Elastic Container Service (ECS) or Amazon Elastic Kubernetes Service (EKS).

·?????? Functions: Functions separate the execution environment from the code you want to implement. For instance, AWS Lambda enables you to execute code without the need to launch an instance.

Compute Resource Selection

??????????? The optimal computing solution for a task depends on factors such as application design, use patterns, and configuration parameters. Architectures have the ability to use diverse computing solutions for separate components and activate certain features in order to enhance performance. Choosing an inappropriate computing solution for a given architecture might result in reduced performance efficiency.

Data Management

??????????? The most suitable data management solution for a specific system depends on factors such as the type of data (block, file, or object), the way it is accessed (random or sequential), the required speed of data transfer, how often it is accessed (online, offline, or for archival purposes), how often it is updated (write once, read many times or dynamic), and the constraints on its availability and durability.

AWS Storage

??????????? Within the AWS platform, storage options are offered in three distinct formats.

·?????? Object Storage: Object storage offers a flexible and resilient framework for accessing data from any internet location. It is often used for user-generated content, active archiving, serverless computing, storing large amounts of data, and backup and recovery purposes. Amazon Simple Storage Service (Amazon S3) is a storage service that provides exceptional scalability, data availability, security, and performance for storing objects. Amazon S3 is specifically engineered to achieve a durability rate of 99.999999999% (eleven nines) and serves as a storage solution for many applications used by enterprises worldwide.

·?????? Block Storage: Block storage offers reliable, uniform, and fast block storage for every virtual host, similar to direct-attached storage (DAS) or a Storage Area Network (SAN). Amazon Elastic Block Store (Amazon EBS) is specifically intended to provide permanent storage that can be accessed by EC2 instances. It allows you to optimize your applications by providing the appropriate storage capacity, performance, and cost.

·?????? File Storage: File storage enables simultaneous access to a communal file system across several systems. Amazon Elastic File System (Amazon EFS) is well-suited for many purposes, such as storing massive content repositories, development environments, media stores, or user home directories. Amazon FSx enables the efficient and cost-effective deployment and operation of popular file systems, allowing you to take advantage of the extensive capabilities and high-speed performance of widely used open source and commercially licensed file systems.

Data Storage, Manage and Access Patterns

??????????? The optimal storage solution for a system depends on the type of access operation (block, file, or object), access patterns (random or sequential), required throughput, frequency of access (online, offline, archival), frequency of updates (WORM, dynamic), and availability and durability requirements. Efficiently designed systems use a variety of storage options and activate various features to enhance performance and optimize resource utilization.

Networking and Content Delivery

??????????? The most efficient networking solution for a task depends on factors such as latency, throughput needs, jitter, and capacity. The available location possibilities are determined by physical restrictions, such as user or on-premises resources. These limitations may be counterbalanced by using edge locations or strategically allocating resources.

??????????? Networking on AWS is virtualized and offers several kinds and settings. This facilitates the alignment of your networking requirements. AWS provides many product features, such as Enhanced Networking, Amazon EC2 networking optimized instances, Amazon S3 transfer acceleration, and dynamic Amazon CloudFront, to enhance network traffic optimization. AWS provides several networking capabilities, such as Amazon Route 53 latency routing, Amazon VPC endpoints, AWS Direct Connect, and AWS Global Accelerator, which may be used to minimize network distance or jitter.

Process and Culture

??????????? To cultivate a culture that promotes the optimal performance of cloud workloads, it is advisable to take into account the following fundamental concepts and practices.

Key Principles

·?????? Infrastructure as code: Specify your infrastructure configuration using methodologies like AWS CloudFormation templates. Utilizing templates enables you to include your infrastructure into source control in conjunction with your application code and customizations.

·?????? Deployment pipeline: Employ a continuous integration/continuous deployment (CI/CD) pipeline, which encompasses several components such as a source code repository, build systems, deployment mechanisms, and automated testing, to facilitate the deployment of your infrastructure.

·?????? Well-defined metrics: Establish and oversee metrics to measure and track key performance indicators (KPIs). We advise use a combination of technical and business indicators.

·?????? Performance test automatically: As a component of your deployment procedure, start performance tests automatically after the faster tests have been executed successfully. The automation should establish a fresh environment, configure basic parameters such as test data, and thereafter execute a sequence of benchmarks and load tests.

·?????? Load generation: It is advisable to develop a set of test scripts that imitate artificial or pre-recorded user experiences. These scripts should be independent and not interconnected, and you may need to use pre-warming scripts to provide accurate results.

·?????? Performance visibility: It is important for your team to have clear visibility of key data, particularly metrics related to each build version.

·?????? Visualization: Utilize visualization approaches that effectively highlight the locations of performance difficulties, hot spots, wait situations, or poor usage. Superimpose performance data onto architectural diagrams, such as call graphs or code, to swiftly pinpoint faults.

·?????? Regular review process: By adopting a performance review process, you may drive incremental change if your architecture is underperforming.

·?????? Continual optimization: Embrace a culture that focuses on consistently improving the performance efficiency of your cloud workload.

Bibliography

Acceldata. (2022, September 7). How to Architect a Data Platform. Retrieved from acceldata.io: https://www.acceldata.io/article/what-is-a-data-platform-architecture

Amazon Web Services. (n.d.). AWS Well Architected Framework. Retrieved from aws.amazon.com: https://aws.amazon.com/architecture/well-architected/?wa-lens-whitepapers.sort-by=item.additionalFields.sortDate&wa-lens-whitepapers.sort-order=desc&wa-guidance-whitepapers.sort-by=item.additionalFields.sortDate&wa-guidance-whitepapers.sort-order=desc

Amazon Web Services. (n.d.). What is AWS? Retrieved from aws.amazon.com: https://aws.amazon.com/what-is-aws/?nc1=f_cc

DAMA International. (2024). DAMA-DMBOK: Data Management Body of Knowledge: 2nd Edition, Revised. Los Angles: Technics Publications.

en.wikipedia.org. (n.d.). Data Management Association. Retrieved from en.wikipedia.org: https://en.wikipedia.org/wiki/Data_Management_Association

Groover, M. (2021). Speed of Advance. Lion Crest Publications.

Hiltbrand, T. (2024, May 9). From Data-Driven to Data-Centric: The Next Evolution in Business Strategy. Retrieved from tdwi.org: https://tdwi.org/Articles/2024/05/09/PPM-ALL-From-Data-Driven-to-Data-Centric-Next-Evolution-in-Business-Strategy.aspx

Intrepid Tech Ventures. (n.d.). Understand your data asset. Retrieved from theintrepidventures.com: https://theintrepidventures.com/value-proposition/understand-your-data-asset/

Khan, S. M. (2024, May 9). The data product lifecycle: Getting the most out of your data investments. Retrieved from starburst.io: https://www.starburst.io/blog/data-product-lifecycle/

Roberts, S. (2023, April 18). Understand the four Vs of Big Data. Retrieved from theknowledgeacademy.com: https://www.theknowledgeacademy.com/blog/4-vs-of-big-data/

Rowshankish, R. L. (2023, July 31). The evolution of the data-driven enterprise. Retrieved from mckinsey.com: https://www.mckinsey.com/capabilities/mckinsey-digital/our-insights/tech-forward/the-evolution-of-the-data-driven-enterprise

Simon, B. (2021, July 21). Complete Guide to PPT Framework | Smartsheet. Retrieved from smartsheet.com: https://www.smartsheet.com/content/people-process-technology#:~:text=for%20IT%20%26%20Ops-,What%20Is%20the%20People%2C%20Process%2C%20Technology%20Framework%3F,maintain%20good%20relationships%20among%20them.

Tharran, A. S. (2023, October 22). The Evolution of Data Science: Past, Present, and Future. Retrieved from linkedin.com: https://www.dhirubhai.net/pulse/evolution-data-science-past-present-future-aditya-singh-tharran-bmmre/

?#AWS #WellArchitectedFramework #Security ##SecurityPillar #OperationalExcellence #OperationalExcellencePillar #Reliability ##ReliabilityPillar #CostOptimization #CostOptimizationPillar #Sustainability #SustainabilityPillar #DataDrivenCompany #TechnologyPlatform

#DataManagement #DataStrategy #DataLifecycle #DAMA-DMBOK

#DAMA #DMBOK #DataDrivenCompany #DataDriven #BusinessStrategy #PPT #People #Process #Technology #PeopleProcessTechnology #Organization #Company #Firm #Business #Data #DataLake #DataWarehouse #Databases #OLTP #OLAP #BigData #Hadoop #DAMA #DMBOK #DataGovernance #DataIngestion #DataVisualization #DataProcessing #ETL #ELT #MasterData #Metadata #DataSecurity #Security #OperationalExcellence #Relaibility #Sustainability #CostOptimization #PerformanceEfficiency #Kenesis #DynamoDB #Redshift #RedshiftSpectrum #QuickSight?#Trino #Iceberg #Parquet #S3 #Lambda #EC2 #ECS #EKS #VPC #SecurityGroups #Python #PySpark #Spark #SparkSQL #SparkStreaming #DataFrames #RDDs #CoudFormation #AWSConfig #MachineLearning #AI #AI/ML #DataEngineer #MLEngineer #LLMs #DataManagement #DAMA #Newsletter #KnowledgeSharing

#AWS #DataDrivenCompany #TechnologyPlatform #DataManagement #DataStrategy #DAMA-DMBOK #WellArchitectedFramework #DataGovernance #DataIngestion #DataVisualization #DataProcessing #ETL #ELT #DataSecurity #Security #OperationalExcellence #Reliability #Sustainability #CostOptimization #PerformanceEfficiency #MachineLearning #AI #DataEngineer #MLEngineer #KnowledgeSharing #AWS #CloudComputing #WellArchitectedFramework

?