🧵 Here's how we're tackling text de-identification to remove personal information from code datasets at BigCode: - An annotated benchmark 📑 - A pipeline for PII detection and anonymization 🚀 - A demo to visualize anonymized samples 🔍 (1/n)

Between now and Christmas🎄 we are running a series on experiments to figure out what the best pre-processing is for code datasets such as The Stack. We'll share the W&B dashboards of these 🎅-models so if you are interested you can follow along!

Today we are releasing The Stack v1.1! 🚀 We added more data, included more programming languages, and extended the list of permissive licenses used. huggingface.co/datasets/bigco… Also the first batch of opt-out requests was removed from the dataset.

Sufficiently advanced Neural Networks are fundamentally a new species of computation. They are not source code. Thus, they cannot be licensed as open source (which was designed for source code)…

Who's going to upload the 100,000th public model on the Hugging Face hub? hf.co/models

We're releasing our second Hugging Face Ethics&Society newsletter, it's a Big One 🤗 The Winter edition talks bias in ML, tools from the 🤗 team to address it, and the role of collaboration across the ML development chain in mitigating risks 🧑‍🤝‍🧑 hf.co/blog/ethics-so… 1/5

We crossed 100,000 public AI models on the Hugging Face hub available for free to all. Thank you to the whole community of contributors. Proud to make ML more open & collaborative!

Announcing a holiday gift: 🎅SantaCoder - a 1.1B multilingual LM for code that outperforms much larger open-source models on both left-to-right generation and infilling! Demo: hf.co/spaces/bigcode… Paper: hf.co/datasets/bigco… Attribution: hf.co/spaces/bigcode… A🧵:

Finally, we summarized our findings in a technical report with a wonderful group of collaborators: Paper: hf.co/datasets/bigco… So what's next? Scaling to larger models and training more languages early next year! 🚀

It's been an exciting year for 🤗Transformers. We tripled the number of weekly active users over 2022, with over 1M users most weeks now and 300k daily pip installs on average🤯

We are now inviting participation from the community to help shape the future of Responsible AI Licenses (RAIL)! RAIL has recently been adopted by many new AI materials (e.g. BigCode)...

#Startup vibes with Carlos Muñoz Ferrandis 💪🔥