exFAT File System – to Save or Not to Save?

Background:

exFAT (Extended File Allocation Table), developed by Microsoft, is the preferred file system for use in many flash memory (USB devices, SD cards, etc.), embedded devices, and external drives with a unique combination of features.

The Most Prominent of These Features Are:

• A fast, lightweight file system with low memory and power requirements optimised for flash memory and embedded devices.
• Read and write cross-platform compatibility with Windows, macOS, Linux, and Android.
• Improved free space allocation and deleted file performance.
• Support for files larger than 4GB.

The TL; DR:

The filesystem’s fast, lightweight, and low resource usage design is likely its biggest weakness, leading to potential data loss scenarios, especially when used as shared storage in mixed Apple Mac device and macOS environments.

Our Recommendations:

If you buy a hard disk drive with exFAT

• Reformat with APFS or HFS+ in a macOS environment and NTFS in a Windows environment.

If you must use exFAT:

• Only use exFAT devices for transferring data between systems, e.g. between a camera and your PC.
• Leave the drive connected to your PC for a period to complete pending read and write operations.
• Always use the “Safely Remove Hardware and Eject Media” option on Windows and “Eject” on MAC before disconnecting the storage device.
• Avoid using exFAT-formatted drives in shared data storage environments, e.g. network-attached storage (NAS), servers, etc.

Our Experience:

Over the years, we have helped many customers, mostly multi-media content creators, recover data from exFAT formatted storage devices facing potential data loss situations.

The Latest Issue:

In a recent incident, we encountered a curveball after transferring data recovered from two electro-mechanically failed 20TB hard drives to an exFAT-formatted 36TB G-RAID device using a Windows workstation. It's worth mentioning that the two failed 20TB drives were formatted with exFAT and had been used without any issues in a multi-generation Mac device and Windows workstation environment.

Within a week after delivering the data, the client contacted us about random folders that disappeared from the G-RAID device.

We re-formatted the G-RAID device with exFAT, this time using a macOS device, and transferred the recovered data. Two weeks later, the customer contacted us again, reporting that newly added videos and photos had vanished from the drive.

Upon scanning the G-RAID device, we discovered that the missing data had been classified as "deleted data". The primary folder structures, organised by event date and client name, were nowhere to be found. Only the last-level folders containing the footage remained, necessitating the time-consuming process of meticulously matching event dates with file timestamps to confirm a successful recovery. At this point, we were uncertain what was causing this “unique” problem.

The third time around, we re-formatted the G-RAID device with Apple's Native HFS file system and transferred the recovered data using the latest version of macOS. No further problems have been reported.

More Research:

This prompted us to do some research and additional testing; the results were concerning.

Since the release of macOS Big Sur and subsequent updates, numerous reports surfaced on the internet, highlighting issues ranging from read-only access to complete data loss. Some of these reported issues:

Read-Only Issues on Big Sur:

• Users upgrading to macOS Big Sur reported exFAT USB external drives becoming "read-only," leaving them unable to modify or save files. This issue restricts the drives’ full functionality and disrupts the seamless cross-platform compatibility of the exFAT file system.

Incompatibility with Catalina:

• Catalina users faced challenges sharing files between macOS and Windows on exFAT-formatted drives. Files written by Catalina on the exFAT drive seem to disappear when accessed from Windows 10, leading to data inconsistency and loss of trust in the file system.

Recognition Problems with New MacBooks:

• Users with newer MacBook models, such as the MacBook Pro 2020 and MacBook Air M2 running macOS Ventura, reported issues with exFAT drives not being recognised. This raises concerns about compatibility and potential hardware-software conflicts.

Mounting Delays and First Aid Failures:

• Some users using exFAT for its cross-platform support encountered drive mounting delays or complete failures to mount. Additionally, the First Aid utility, designed to diagnose and repair disk issues, often cannot complete its tasks, leaving users without a reliable tool for troubleshooting.

Data Inaccessibility and Deletion Concerns:

• Perhaps the most alarming issue is the inability to access data from drives formatted with exFAT on an older MacBook when using a newer MacBook. In some cases, the hard drive is not recognised at all, leading to potential data loss and the inability to retrieve the data.

?Other Mac User Reports:

"Since updating to software version Big Sur, my exFAT USB is "read-only" with no way of accessing the files."

"I formatted to exFAT my SSD using my PC but my Macbook Pro doesn't detect it."

"I have a drive formatted in exFAT which I have happily been sharing files between Windows 10 and Mojave, with no issues whatsoever, until Catalina. Under Catalina, I can read files and write files to/from the exFAT drive, but any file written to the drive by Catalina cannot be found under Windows 10; it is just not there."

"Hi, I have a new external HDD, which I backed files up using my old MacBook. The external HD is formatted with exFAT. When I plug it into my new MacBook Pro (2020), it isn’t recognised at all. The drive still works perfectly well with my old MacBook."

"... I have so many issues with external drives formatted in exFAT. Since I use both Windows and Mac, I can’t use APFS (I think). Drives take forever to mount or never mount at all. First aid cannot be completed."

"I’m facing an issue to read files that are in ExFAT format created in the past with a MacBook Pro 2018 version in an external hard drive. Recently, with a new MacBook Air M2 with macOS Ventura I’m unable to access those files, the external hard drive is not even recognized so not possible to read them."

Finding a Possible Explanation:

This recent issue with our customer was unexpected, given that our customer successfully used exFAT file systems to store hundreds of terabytes in the past without data disappearing. We learned later from the client that the emergence of problems coincided with the recent OS updates of some of their Notebooks and Desktops. In addition to the data disappearance, they also had video playback issues, indicating a potential correlation with recent macOS updates and codec-related challenges.

Adding another layer of complexity, the G-RAID device in question is not a conventional single hard drive system; it employs a built-in hardware RAID configured with 2 x 18TB hard drives, configured as a RAID-0 volume for improved performance. This hardware RAID configuration could also influence the exFAT file system management and, specifically, how it handles file caching.

Clear As Mud:

It is still unclear why these issues occur, but it may also be attributed to the following:

• In addition to the absence of a catalog file, another layer of complexity arises when considering the differences in GUID (Globally Unique Identifier) implementation between exFAT and Apple's file systems. GUIDs are unique identifiers assigned to various elements within a file system, such as volumes, partitions, and file entries, to ensure their distinct identification.
• Apple's file systems, including HFS+ and APFS, utilise GUID Partition Tables (GPT) to manage disk partitions. GPT is a modern partitioning scheme that employs GUIDs to uniquely identify each partition on a storage device. This method aligns seamlessly with Apple's file systems, providing a robust and reliable structure for managing partitions and their associated data.
• On the other hand, exFAT, being a file system developed by Microsoft, may implement its own GUID scheme for partition identification. When exFAT-formatted drives are connected to Apple machines, the differences in GUID implementation can introduce challenges. Apple's macOS, optimised for its native file systems and GPT, may encounter difficulties interpreting exFAT GUID structures, potentially leading to recognition issues, mounting delays, or unexpected behaviour.

In Summary:

While exFAT was designed to facilitate seamless file exchange between Windows and macOS, users’ real-world experiences tell a different story. The issues range from read-only problems to complete data loss, affecting users across various macOS versions and hardware configurations.

The unique combination of exFAT’s distinct file allocation table (FAT) structure, the absence of a catalog file, and potential disparities in GUID implementation can contribute to users’ reported issues when using exFAT.

As users continue to encounter challenges, it is imperative for Apple to address these issues to provide a more reliable and consistent experience for those relying on ExFAT for cross-platform file compatibility.?Until then, users are advised to exercise caution and consider alternative file systems for their long-term data storage needs.

These complexities also highlight the importance of considering file system compatibility and the intricacies of partitioning schemes when seeking seamless interoperability across different platforms.

Contact Us Today for a No Obligation Data Recovery or Data Retention Consultation.