Collibra – Trino/Starburst – Apache Ranger Integration

This data governance and security framework involves a systematic process starting with the harvesting of data from a Trino/Starburst data source, a distributed SQL query engine designed for large-scale datasets. The harvested data is then seamlessly integrated into Collibra, a data governance and cataloging platform, where organizations can define and manage comprehensive data governance policies. These policies, covering aspects such as data access, sharing, privacy, and security, can be updated within Collibra to adapt to changing business needs.

??????????????????????? The integration extends to Apache Ranger, a robust data security and access control framework, which enforces and manages data access policies for Trino/Starburst data sources. The synchronization between Collibra and Apache Ranger ensures that any updates made to governance policies in Collibra are promptly reflected in the access control policies enforced by Apache Ranger. This synergy between data harvesting, Collibra integration, and policy enforcement through Apache Ranger creates a cohesive approach to data governance and security. It not only enhances data protection by controlling access to the Trino/Starburst data source but also facilitates audit and monitoring through Ranger, enabling organizations to track and review data access for compliance and governance purposes. In summary, this integrated system establishes a secure and compliant environment, ensuring that data usage aligns with defined policies and regulations

????? Trino/ Starburst - Collibra Integration framework, which is built around Lorang Technology’s proprietary Metadata Integration Framework (MIF) addresses the following business and functional challenges that are encountered in provisioning access to big-data sources and data access governance

??? Lack of a centralized governance platform that controls access to multiple big-data sources (e.g., Hive, Kafka, Elastic Search, etc.).

??? Non-homogenous data-access sharing methods with different rules, sharing agreements and ownership.

??? Lack of dedicated Query Access rule Governance frameworks.

??? Data discovery challenges due to multiple metadata platforms and non-uniform discovery features.

??? Lack of a uniform and automated dataset checkout mechanism.

??? Automated security checks on data (e.g., PII, SDE elements) are not present during checkout and provisioning.

??? Complete provisioning of dataset without control over the rights among the group of users.

Add Value

1) Data Harvesting: Data is harvested from a Trino/Starburst data source. Trino/Starburst, formerly known as Presto, is a distributed SQL query engine designed for querying large datasets across multiple data sources.

2) Collibra Integration: The harvested data is then integrated with Collibra. Collibra is a data governance and cataloging platform that helps organizations manage and organize their data assets.

3) Data Governance Policies: Collibra is used to define and manage data governance policies. These policies can include rules and guidelines for data access, sharing, privacy, and security.

4) Policy Updates: Data governance policies within Collibra are updated to reflect changes or new requirements, such as access restrictions or data handling guidelines.

5) Ranger Integration: Apache Ranger, a data security and access control framework, is used to enforce and manage data access policies for Trino/Starburst data sources.

6) Synchronization: The policy updates made in Collibra are synchronized with Apache Ranger. This ensures that the access control policies in Ranger align with the policies defined in Collibra.

7) Access Control: Apache Ranger enforces these policies by controlling access to the Trino data source. It can restrict or grant access based on the defined governance policies.

8) Data Security: The integration between Collibra, Ranger, and Trino/Starburst enhances data security and compliance by ensuring that only authorized users and applications can access the data source.

10) Audit and Monitoring: Ranger provides audit and monitoring capabilities, allowing organizations to track and review data access and policy enforcement.

11) Data Governance Compliance: This use case facilitates data governance compliance by ensuring that data usage aligns with the defined policies in Collibra and is enforced through Ranger.

In summary, this use case illustrates how data harvesting from Trino/Starburst, policy updates in Collibra, and policy enforcement in Apache Ranger work together to ensure data governance and security for a Trino/Starburst data source, meeting compliance and data protection requirements.