登录查看更多内容

What are the best ways to scale an informatics system that involves data analytics?

由人工智能和领英社区提供技术支持

Informatics is the science of processing, analyzing, and communicating data in various domains, such as health, education, business, and engineering. Informatics systems often involve complex data pipelines that collect, store, transform, and visualize data from multiple sources and formats. To scale an informatics system that involves data analytics, you need to consider several factors that affect its performance, reliability, and usability. In this article, you will learn about some of the best ways to scale an informatics system that involves data analytics, such as:

此文章中的业界达人

由社区从 4 条内容中精选。了解更多

1 Choosing the right architecture

The architecture of an informatics system defines how its components interact and communicate with each other, such as data sources, data warehouses, data lakes, data marts, ETL processes, analytic engines, and dashboards. Choosing the right architecture depends on the type, volume, velocity, and variety of data as well as the business goals, analytical needs, and user expectations. Common architectural patterns for scaling informatics systems include the Lambda architecture; this combines batch and stream processing to handle both historical and real-time data. It consists of three layers: a batch layer that stores and processes raw data in batches; a speed layer that processes data streams in near real-time; and a serving layer that provides a unified view of the data for queries and analytics. The Kappa architecture simplifies the Lambda architecture by eliminating the batch layer and relying only on stream processing. Lastly, the Microservices architecture breaks down the informatics system into smaller, independent services that communicate through APIs. Each service can be scaled, deployed, and updated independently for greater flexibility, modularity, and fault tolerance.

添加您的观点

Jim H.

Sr. Director of Informatics | PhD in Human Genetics | AI, Data: Management, Architecture, Science, Products
举报内容
Databricks Delta Lakehouse system. Unity Catalog works w Azure/AWS/GCP. Catalogs for hierarchy and governance, schema and tables for data.

已翻译

赞
Heather Michael

Strategic Health Informatics Leader | Expert in Population Health and Advanced Analytics Solutions | Driving Organizational Growth and Health Outcomes Improvement
举报内容
One thing that I don’t see mentioned often is that to really scale an idea or concept effectively, end users have to really understand the purpose of the concept. I have found that operationally focused teams have a difficult time tapping into informatics, data science, or analytics teams because they do not fully understand how to leverage them. While we (the technical folks) are focused on foundation and architecture, we effectively leave our stakeholders behind. To scale, business needs to understand where their questions/needs fall (descriptive, diagnostic, predictive analytics) before they can understand why scaling informatics is critical.

已翻译

赞

2 Optimizing the data pipeline

The data pipeline is the sequence of steps that move and transform data from its source to its destination, such as data ingestion, data cleansing, data integration, data enrichment, data analysis, and data visualization. Optimizing the data pipeline involves improving its efficiency, quality, and security. To do this, one should consider parallelizing and distributing the data processing to reduce processing time and handle larger volumes of data. This can be done with the help of tools such as Hadoop, Spark, Flink, Kafka, and Storm. Automating and orchestrating the data workflow with software or scripts for scheduling, triggering, monitoring, logging, and error handling can also minimize manual effort and human errors. Tools like Airflow, Luigi, Oozie, and Azkaban can help in this regard. Lastly, securing and encrypting the data transmission through protocols like SSL (secure sockets layer), TLS (transport layer security), VPN (virtual private network), or AES (advanced encryption standard) can ensure its confidentiality, integrity, and availability.

添加您的观点

Glen Goh (BEng, PMP?, DipE, SDipBA)

Data | Digitalisation | Transformation
举报内容
At times, in the data journey, the cleaning of the data takes up most of the time and hence, deters people from bothering with retrospective data for ingestion. That aside, if given the ability to 'start over', it may be wise to have most efforts (manpower typically) to be placed at the design of data collection. This, when done well, allows for the optimisation of the data pipeline for all data-hungry processes to pull data from this pipeline (think actual pipes as an imagery). Of course, falling hand-in-hand, would be access control. Data is only useful in the correct hands but be mindful on proper access rights at onset. Defining the access rights would be up to your organisation but effectively, rights should be fit for purpose.

已翻译

赞

3 Designing the user interface

The user interface is the part of the informatics system that allows users to interact with data and analytics, such as web pages, mobile apps, dashboards, charts, and reports. Designing the user interface involves optimizing its usability, functionality, and aesthetics. To do this, you should apply the principles of user-centered design by understanding user needs, preferences, and behaviors. Additionally, you should implement the best practices of data visualization by using visual elements like colors, shapes, sizes, and layouts to display data. Furthermore, leveraging the power of web technologies like HTML, CSS, JavaScript, and frameworks like React, Angular or Vue can create dynamic user interfaces that run on different browsers and devices. Scaling an informatics system that involves data analytics needs careful planning, testing and evaluation of various aspects of the system. By following these best ways discussed in this article, you can scale your informatics system to meet growing and changing demands of data and analytics.

添加您的观点

4 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?

添加您的观点

Eric Ung

Consultant
举报内容
Sturdy foundations in understanding your goal is fundamental in what you are trying to model. If you are trying to model medicinal information, consider what is the basis of your axioms and definitions and also consider a concrete example of what you are trying to do. Probability and statistics are two different ideas.

已翻译

赞

Computer Science

+ 关注

给文章评分

我们借助人工智能创建了此文章。您认为这篇文章怎么样？

很棒不太好

举报此文章

查看全部

What are the best ways to scale an informatics system that involves data analytics?

1

2

3

4

1 Choosing the right architecture

2 Optimizing the data pipeline

3 Designing the user interface

4 Here’s what else to consider

Computer Science

给文章评分

感谢您的反馈

更多Computer Science相关文章

更多相关阅读内容

What are the best ways to scale an informatics system that involves data analytics?

1

2

3

4

1 Choosing the right architecture

2 Optimizing the data pipeline

3 Designing the user interface

4 Here’s what else to consider

Computer Science

给文章评分

感谢您的反馈

查看其他技能