登录查看更多内容

How do you mask and tokenize data?

由人工智能和领英社区提供技术支持

Data is one of the most valuable assets for any organization, but it also comes with risks and responsibilities. Data breaches, privacy violations, and regulatory compliance are some of the challenges that data engineers face when working with sensitive or personal data. One of the ways to protect data and reduce exposure is to mask and tokenize it. In this article, you will learn what masking and tokenization are, why they are important, and how to apply them in your data engineering projects.

此文章中的业界达人

由社区从 7 条内容中精选。了解更多

Carlos Fernando Chicata

Algunas insignias de community Top Voice | Ingeniero de datos | AWS User Group Perú - Arequipa | AWS x3
Samyukta Hariharan

AI Data Engineer @ SAP
Sai Srikanth B.

Senior Data Engineer and Designer (AI & Strategy) at Commbank | CEO(Aus) at MIT Aus Technologies| AWS 3x, Google…

1 What is masking and tokenization?

Masking and tokenization are two techniques for transforming data in a way that preserves its original structure and format, but obscures or replaces its actual content. Masking involves replacing or hiding some or all of the data elements with dummy values, such as asterisks, random characters, or generic terms. Tokenization involves substituting the data elements with non-sensitive tokens, such as numbers, symbols, or encrypted values, that have no meaning or relation to the original data. Both masking and tokenization aim to prevent unauthorized access, disclosure, or misuse of the data, while allowing authorized users to perform their tasks.

添加您的观点

Sai Srikanth B.

Senior Data Engineer and Designer (AI & Strategy) at Commbank | CEO(Aus) at MIT Aus Technologies| AWS 3x, Google Certified | GenAI Engineer | MBA Candidate- Melbourne Business School | Podcast Enthusiast
举报内容
Data masking is a key tool in real life data storage and usage techniques. This plays a pivotal role especially masking of PII information and several masking techniques are combined with tokenisation or data encryption. These days real life tokenisation and assigning tokens based on several techniques have become crucial in data storage.

已翻译

赞

2 Why are masking and tokenization important?

Masking and tokenization are important for several reasons. First, they help data engineers comply with data protection laws and regulations, such as the General Data Protection Regulation (GDPR) or the Health Insurance Portability and Accountability Act (HIPAA), that require data minimization, anonymization, or pseudonymization of personal or sensitive data. Second, they help data engineers reduce the risk of data breaches, leaks, or thefts, by limiting the exposure and impact of the data in case of a security incident. Third, they help data engineers optimize the performance and efficiency of their data pipelines, by reducing the size and complexity of the data and avoiding unnecessary encryption or decryption processes.

添加您的观点

Sai Srikanth B.

Senior Data Engineer and Designer (AI & Strategy) at Commbank | CEO(Aus) at MIT Aus Technologies| AWS 3x, Google Certified | GenAI Engineer | MBA Candidate- Melbourne Business School | Podcast Enthusiast
举报内容
Data Masking helps in not only securing the information but also adhering the data protection laws. There are different masking techniques in which SHA 256, 512. Master data masking, tokenisation and encryption helps in data protection. They also seamlessly help in pipeline automation and this helps a lot in seamless data integration with minimal data complexity

已翻译

赞

3 How to mask data?

Masking techniques vary depending on the level of protection and functionality needed. Static masking applies the masking rules to data before it is moved or copied to a different environment, while dynamic masking applies the rules as the data is requested or accessed. Partial masking replaces or hides a portion of the data element, while full masking replaces or hides the entire element. To mask data, you can use built-in functions, custom scripts, or third-party software and services. For instance, Oracle Data Masking and Subsetting, Microsoft SQL Server Dynamic Data Masking, AWS DataBrew, Faker library in Python, Delphix Data Masking, Informatica Data Masking, or IBM InfoSphere Optim Data Privacy.

添加您的观点

Sai Srikanth B.

Senior Data Engineer and Designer (AI & Strategy) at Commbank | CEO(Aus) at MIT Aus Technologies| AWS 3x, Google Certified | GenAI Engineer | MBA Candidate- Melbourne Business School | Podcast Enthusiast
举报内容
There are several techniques for data masking. Usually, every organization follows different techniques to achieve this. Sensitive information masking is crucial and helps in concealing the real value of data. These days the tools in the market have several tools that come with inbuilt masking features, however, custom masking is still remains popular. Several methods like Pseudonymization, Anonymization, Encryption, etc.. are used in industries

已翻译

赞

4 How to tokenize data?

Tokenization is a method of protecting data that involves generating and assigning tokens to original data elements. Depending on the level of protection and functionality needed, there are various tokenization techniques, such as reversible tokenization, irreversible tokenization, format-preserving tokenization, and format-changing tokenization. To tokenize data, you can use built-in tokenization functions from DBMS or cloud platforms, custom tokenization scripts written with programming languages or frameworks that support data encryption or hashing, or third-party tokenization software that provide specialized or advanced features. For example, Google Cloud Data Loss Prevention (DLP) API, Azure SQL Database Tokenization, MongoDB Field Level Encryption, Protegrity Data Security Platform, Micro Focus Voltage SecureData, or TokenEx Data Protection Platform.

添加您的观点

Sai Srikanth B.

Senior Data Engineer and Designer (AI & Strategy) at Commbank | CEO(Aus) at MIT Aus Technologies| AWS 3x, Google Certified | GenAI Engineer | MBA Candidate- Melbourne Business School | Podcast Enthusiast
举报内容
This technique is mostly used in financial world, in relation to payment processing, demands the substitution of a credit card or account number with a token. Unlike encryption, tokenization of data cannot be reversed. Rather than using a breakable algorithm, a tokenization system substitutes sensitive data by mapping random data, thus the token cannot be decrypted. The token is a placeholder, with no essential value. The true data is kept in a separate location, such as a secured offsite platform. The original data does not enter your IT environment. If an attacker penetrates your environment and accesses your tokens, they have gained nothing. Thus, tokens cannot be used for criminal undertakings

已翻译

赞

5 How to choose between masking and tokenization?

Masking and tokenization are not mutually exclusive and can be used in combination or separately, depending on data engineering goals and requirements. Factors that may influence the decision include the sensitivity and value of the data, the functionality and usability of the data, and the cost and complexity of the solution. For example, if the data is highly sensitive or valuable, such as financial or health information, tokenization may be preferable as it provides a higher level of security. If the data needs to be processed, analyzed, or manipulated, such as for data mining, machine learning, or reporting, masking may be more suitable as it preserves the original structure of the data. Additionally, if a solution needs to be simple, fast, or cheap, such as for testing, development, or prototyping, masking may be more cost-effective than tokenization. Ultimately, the choice between masking and tokenization should take into account your specific requirements and objectives.

添加您的观点

Carlos Fernando Chicata

Algunas insignias de community Top Voice | Ingeniero de datos | AWS User Group Perú - Arequipa | AWS x3
举报内容
The masking algorithm always is more related with functional purpose on the format aspect of the data in all aspect; while tokenization can define more sophisticated privacy strategy plan to enrich the use and handle of the value show. Evaluate how national laws and standard regulations indicate about it: maybe indicate in what cases you one of them.

已翻译

赞
Sai Srikanth B.

Senior Data Engineer and Designer (AI & Strategy) at Commbank | CEO(Aus) at MIT Aus Technologies| AWS 3x, Google Certified | GenAI Engineer | MBA Candidate- Melbourne Business School | Podcast Enthusiast
举报内容
When comparing data masking vs data tokenization, it’s important to understand that one approach isn’t inherently better than the other. Both protect sensitive, PII data, leading to privacy compliance. To decide which approach is best for each of your use cases, start by answering these questions: Where is sensitive data used most? Where is it as risk? Which industry-related privacy regulations and security standards is the organization subject to? Where do the greatest vulnerabilities currently lie?

已翻译

赞

6 Here’s what else to consider

This is a space to share examples, stories, or insights that don’t fit into any of the previous sections. What else would you like to add?

添加您的观点

Samyukta Hariharan

AI Data Engineer @ SAP
举报内容
One thing to note is that protection of data is necessary not just at the storage level but also during movement of data from a sender to a receiver. For example, in cybersecurity and cryptography, sometimes public key and private key (and at times secret key) encryption and decryption are used to verify the veracity of the data that is being transported over the network. Some other times, login information in stored in such a way that when the user enters their username and password, during verification only username is transferred as-is over the network while the password is hashed completely, after which the username is used as a common key to map the pre-existing password stored in the DB, that was hashed using the same algorithm.

已翻译

赞

Data Engineering

+ 关注

给文章评分

我们借助人工智能创建了此文章。您认为这篇文章怎么样？

很棒不太好

举报此文章

查看全部

How do you mask and tokenize data?

1

2

3

4

5

6

1 What is masking and tokenization?

2 Why are masking and tokenization important?

3 How to mask data?

4 How to tokenize data?

5 How to choose between masking and tokenization?

6 Here’s what else to consider

Data Engineering

给文章评分

感谢您的反馈

更多Data Engineering相关文章

更多相关阅读内容

How do you mask and tokenize data?

1

2

3

4

5

6

1 What is masking and tokenization?

2 Why are masking and tokenization important?

3 How to mask data?

4 How to tokenize data?

5 How to choose between masking and tokenization?

6 Here’s what else to consider

Data Engineering

给文章评分

感谢您的反馈

查看其他技能