Some words on model repository security on the Hub

There some discussions about Hugging Face and model repository security (e.g., malicious LLMs) over the past weeks, so I'm sharing/formating here a great summary written by Omar Sanseviero our Chief Llama Officier summarizing the work we've done with the community for the last years in the area: safe serialization methods, malware scanning, and more. You can find more details on our Hub Security page. Let's go!

1. Pickle scanning

By default, libraries such as PyTorch, TensorFlow, and sklearn use pickle to serialize files.

Pickle, unfortunately, allows people to run arbitrary code (!) that means that loading a model could lead to running whatever someone wants on your computer. (By the way, this is something anyone using python should know nothing specific to ML or HF here, see the very clear official python doc warning)

As a first step to alleviate it, the Hub added a security scanner that scans for malware (using ClamAV, an open-source antivirus). Read more details here.

As a second step, we added Pickle import scanning. We scan all the imports referenced in the pickle file and raise warnings when the pickle uses imports that are suspicious (e.g. could lead to arbitrary code). Read more on pickle scanning on Hugging Face hub here.

2. Safetensors

Scanning is not perfect. A better mitigation strategy is to not use pickle at all!

Pushing beyond PyTorch default save format, Hugging Face kicked off the development of safetensors, an efficient and, most importantly, safe format:

To assess the safety of this new format, we collaborated with EleutherAI and Stability AI to conduct an external security audit through the recognized Trail of Bits organization. You can read more about this audit here.

Safetensors is now the default format across many libraries in our ecosystem and abroad, including the famous transformers library!

Other formats, such as GGUF, have arisen in the ecosystem as safer alternatives than pickle as well!

3. Social validation features

Hugging Face has built-in social features like likes, community tabs, files inspectors, reporting mechanism, and spam detection tools.

In GitHub/npm/pip you would surely not randomly download and run random code from any source like for instance a github repo with 0 likes and a suspicious author. The same is true on the Hugging Face Hub, pay attention to the social features associated to a model and take a moment to look at the likes/community/leaderboard/model files before downloading and running a model from a suspicious account with 0 likes and very small download numbers.

If a model comes from an official trusted org such as Google, Salesforce, MistralAI, etc, the risk is much lower.

Conclusion

In the last weeks, there have been a few press releases from companies that sell security features.

Apart from the obvious conflict of interest, these reports are not showing in our opinion anything that the community did not know already (using piclkle is not good in python) and things the ecosystem has been working on for the last years as you saw above.

Combining safe file formats with trusted sources is a great way to keep yourself safe so I hope you'll generally follow on the hub the same safe practices you're surely following in your online life :)

Stay safe, stay Huggy!

Credit: Omar Sanseviero with small edits from yours truly. Original thread at https://twitter.com/osanseviero/status/1763331704146583806