Data System components

Any system will have multiple pieces to the puzzle. A quick overview of all the components, processes and systems for Data Applications.

1.??? Data Ingestion and Collection:

·???????? Various Data sources, diverse formats, ingestion tools and processes, staging area

2.??? Data Storage:

·???????? Relational and NoSQL DB, file systems, warehouses, marts

3.??? Data Processing, Preparation and Transformation

·???????? Data cleaning, wrangling, standardizing systems

·???????? ETL processing systems

·???????? Batch, stream and in-memory, map-reduce systems and frameworks.

4.??? Data Consumption

·???????? Reporting, Visualization, Machine Learning, BI and AI tools

5.??? Data Governance and Security:

·???????? Data catalog, quality monitoring, access, encryption, masking tools and processes

·???????? Business and Technical Metadata management, data lineage, data dictionaries

6.??? Monitoring and Alerting:

·???????? Logging, Monitoring and Alerting Tools

7.??? Business Continuity Planning

·???????? Backup and recovery strategies, systems and plans.

8.??? Data Lifecycle Management:

·???????? Data Archival, retention, purging policies and systems

9.??? Data Compliance and Regulation:

·???????? Auditing, compliance and control policies and systems

10. Collaboration and Workflow Tools:

·???????? Project management and collaboration systems and processes.

The list would cover most or all of the systems.

Any additions and updates are welcome.

#data #system #process #datamanagement #dataengineering #compliance #regulation #datatechnology