Electronic Data Interchange (EDI) is a process of exchanging business information electronically between trading partners. It can improve efficiency, accuracy, and speed of transactions, but it also poses some data quality and governance risks. How can you handle these risks and ensure your EDI data is reliable, secure, and compliant? Here are some tips to help you.
One of the first steps to handle EDI data quality and governance risks is to understand where your EDI data comes from, what standards and formats it follows, and how it is mapped and translated. Different EDI data sources may have different levels of quality, accuracy, and completeness, and may require different validation and verification rules. You should also be aware of the industry or regulatory standards that apply to your EDI data, such as ANSI X12, EDIFACT, or HIPAA, and ensure your EDI data conforms to them. By understanding your EDI data sources and standards, you can identify and address any potential issues or inconsistencies in your EDI data.
1. Encrypt Data with hash and decode with authorized party 2. SFTP https transferring channel 3. Seldom use XML ,replaced with Json
EDI data can be we exchanged over internet via secured medium of transport like https as2 connection where data can be transmitted security with enhanced security where encryption signature validation are involved.
Another tip how to ensure efficient EDI operation, is to remember, that products, BOMs, specifications and processes change sometimes. It means that partners may continue to interchange with data, which do not make a sense for real business at the moment, while people start texting each other again. That's why teams should not just focus on data integrity and accuracy, but also carefully consider exact impact of each key and character on material and process flows.
One cannot stress enough the importance of reading and understanding the specs, testing and lastly reviewing the output both before go-live and after. The key to quality EDI is going back after go-live and verifying your live output for each partner, documenting the issues and ensuring each and every issue gets resolved.
EDI data should only be exchanged through secure connections like SFTP or AS2. AS2 is more secure as it operates using a transaction based transfer. SFTP is secure but is essentially a secure connection to a file server. Choosing the type of transmission method really depends on the level of security needed for the data being sent over.
Another step to handle EDI data quality and governance risks is to implement EDI data quality controls and checks throughout the EDI process. This includes defining and enforcing data quality rules, criteria, and policies, such as data formats, values, ranges, codes, and identifiers. You should also perform regular data quality audits, assessments, and reviews to monitor and measure your EDI data quality performance and identify any errors, gaps, or anomalies. Additionally, you should establish data quality feedback mechanisms, such as error reports, alerts, or notifications, to inform and correct any data quality issues as soon as they occur.
The ANSI x12 standard provides a template for what type data should be located in each element field. Making sure the your system implements data type checking on both incoming and outgoing transactions should be the first step to ensuring data quality.
One of the most crucial checks is to monitor the workflow itself. Not being aware that an ORDER has not been received translates to a missed opportunity for the business. (Same for all the other types of messages). Exceptions should be communicated to the teams to rectify anomalies. It's imperative to avoid using hard-coded values in the implementation. While each customer may require different implementations, the inbound and outbound data remain standardized. This serves as a reminder of the fundamental rule in software development: 'keep it simple'. Your core solution should hinge on the accuracy of master data, including item masters, customer references, customer information, etc.
Sometimes the system is not very flexible and some data validation is simply missing and not easy to add. I had a situation where invoice EDI messages were rejected by the customer because they contained illegal characters. Usually they appeared because people copied & pasted data from Excel or other non-plain text sources, or manually input an accented character instead of a standard one for example. I analyzed which fields tend to contain offending data, established it was only in descriptive fields, and constructed a lookup table to replace unwanted characters with either nothing or with an equivalent which is human readable. This was added to the EDI mapping application, which offered much more flexibility than the originating system.
A third step to handle EDI data quality and governance risks is to secure and protect your EDI data from unauthorized access, modification, or loss. This includes encrypting your EDI data in transit and at rest, using secure protocols and channels, such as AS2, SFTP, or VPN, to exchange your EDI data, and applying authentication and authorization mechanisms, such as digital signatures, certificates, or passwords, to verify your EDI trading partners. You should also backup your EDI data regularly and have a disaster recovery plan in case of any data breach or outage. Furthermore, you should comply with any data privacy and security regulations, such as GDPR, CCPA, or PCI DSS, that apply to your EDI data.
Just to add to above mentioned. One thing I found helpful was to archive the inbound transactions storing them indexed by business transaction ID. That helpes to quickly locate and re-process the data upon request received from business/operations side.
Depending on the type of transactions and importance of retaining the EDI documents changes but it is recommended to have some standardized data retention plan set for your organizations EDI. We use a 6 months policy that keeps all documents sent and received in our production environment while anything older is moved to an Archive to reduce storage costs for the data.
A fourth step to handle EDI data quality and governance risks is to document and communicate your EDI data governance framework. This includes defining and documenting your EDI data governance roles, responsibilities, and accountabilities, such as data owners, stewards, custodians, or users, and their respective tasks, functions, and authorities. You should also document and communicate your EDI data governance processes, procedures, and standards, such as data lifecycle management, data change management, data quality management, or data security management, and how they are executed, monitored, and controlled. By documenting and communicating your EDI data governance framework, you can ensure your EDI data is managed consistently, transparently, and effectively.
A fifth step to handle EDI data quality and governance risks is to train and educate your EDI data stakeholders. This includes providing your EDI data stakeholders with the necessary knowledge, skills, and tools to understand, use, and maintain your EDI data. You should also provide your EDI data stakeholders with the relevant information, guidance, and support to follow your EDI data governance framework and policies. Additionally, you should update and refresh your EDI data stakeholders' training and education as your EDI data needs, standards, or technologies change. By training and educating your EDI data stakeholders, you can enhance their awareness, engagement, and compliance with your EDI data quality and governance goals.
Yes, this is super important, especially in corporations with SOPs, instrucions and procedures for everything. People perform better when they know WHY certain things are important, as well as how minor discrepancies might grind everything to a halt. And that is especially important in companies where just-in-time deliveries are crucial – every hiccup or glitch might ruin the customer's schedule. Made a short Power Point presentation on how we cooperate with our customers, what information goes where and when, and presented it to people involved. It really made a difference, people were commenting "OK, now I understand why this procedure is so detailed. Now that I know why – it's obvious and instructions seem like common sense".
A sixth step to handle EDI data quality and governance risks is to collaborate and coordinate with your EDI trading partners. This includes establishing and maintaining clear and effective communication channels, agreements, and expectations with your EDI trading partners regarding your EDI data exchange, quality, and governance requirements, preferences, and issues. You should also align and harmonize your EDI data standards, formats, and protocols with your EDI trading partners to ensure interoperability and compatibility. Moreover, you should share and solicit feedback, best practices, and lessons learned with your EDI trading partners to improve your EDI data quality and governance performance and outcomes.
One of the most challenging issues I saw was a manual entry and all the mistype errors related to that. Specifically for smaller trading partners. It may sound obvious but in my experience it did helped when I worked with trading partner directly, identified how do they enter the data and advised that whenever possible they have to implement a selection of the data from preset data lists instead of manually typing the data in.
Cooperation is key. We have a common goal – to have the process run smoothly and efficiently. In case of smaller customers, like Vladimir wrote below, we sometimes offered to clean up their data periodically and provided them with simple tools to facilitate just that – minimize manual entry. If the customer works on data that's been massaged and cleaned by you – it's like a dream situation :) Well worth the added effort.
Although EDI interchanges are potentially supported by a large and extensive set of languages and alphabets I see non-traditional western alphabets (e.g. CJK) getting major issues being adopted. Non-traditional Western alphabets (e.g., CJK) have great problems being adopted, for several reasons beyond the compatibility with UNICODE of internal systems. This prevents classical EDI patterns being adopted in some major eastern countries. Have you never seen an Edifact message showing e.g. Chinese or Arabic characters?
Use services from an EDI integrator in order to build technical flows between trading partners. They have secured platforms that’ll do the job better than you could develop it on your own.