Sharing a summary of two amazing LLM benchmarking papers published just last week: - GAIA: the General AI Assistant benchmark (disclaimer: I'm a co-author with amazing co-authors) - GPQA: the Graduate Level Google Proof QA benchmark (disclaimer: the authors are also awesome) When two teams (covering a diverse range of actors like Anthropic, Cohere, New-York University, Hugging Face, Meta AI) independently come up with benchmarks that share so many aspects (while being really different in goals and approaches), you know the future of LLM benchmarking is basically changing under your eyes. Both are super difficult with ~30% GPT4 success rate, both are small (450 questions) and carefully hand-crafted question by question, with a single gold answer and a strong interest for the reasoning it-self rather than memorization capabilities. Both very challenging test bed for coming models capabilities And above all: I'm super excited that these are open-source benchmark giving us common ground for comparison of the coming frontier models. On to the future (of open evaluation)! Papers and more information: - GAIA: https://huggingface.co/papers/2311.12983 - GPQA: https://huggingface.co/papers/2311.12022

[Friday evening short video] Where I'm summarizing a paper that caught my attention this week: "Role play with large language models" by Murray Shanahan, Kyle McDonell and Laria Reynolds. Paper can be found here: https://www.nature.com/articles/s41586-023-06647-8 Not a technical paper! This one is on anthropomorphism and AI – An interesting approach to explaining AI. It's doesn't solve everything but I'll likely experiment with using this approach when talking about AI to new comers to the field of LLM.

Gradio is the easiest way to build delightful machine learning web apps. Version 4.0 is Gradio's biggest update so far, and comes with many new features that make Gradio more extensible and customizable than ever before! One of those features is the ability to build your own custom Gradio component 🔥🤯 In this live coding session, Freddy from the Gradio team at HuggingFace, will show you how you can make your own Multimodal Chatbot component that interleaves text with audio, video, and images! You will learn: * How to bootstrap your custom component project with the gradio cli * How to implement a Gradio Chatbot component's backend in python * How to implement a Gradio Chatbot component's frontend in Svelte * How to launch the gradio component development server to instantly test your changes * How to publish your component to PyPi and HuggingFace spaces from the command line! In part 2 of this series, we will build a custom multimodal textbox component to send text and files to our chatbot so stay tuned!

Join us as we launch Gradio 4.0, discuss new features (such as building custom components in your Gradio apps), as well as answer questions

https://huggingface.co/

In this session of Computer Vision Study Group, Johannes walks us through the paper Segment Anything Model (SAM), a foundational segmentation model for zero-shot segmentation.

HuggingCast is a new live show where we talk about the latest news in the beautiful world of open source AI, and run through practical demos you can apply in your work! This is the recording of our fourth episode which aired on 7/27/23 - where Philipp and Jeff told you all about 🦙🦙 LLaMa 2, how to use it, and answer all your questions about it! See you for the next season - maybe! #llama #meta #llm #generativeai #huggingface #hugging

Thank you for participating in our first Open Source AI Game Jam! Play the games: https://itch.io/jam/open-source-ai-game-jam Join our discord: https://hf.co/join/discord 00:00 Galactic Bridge 00:03 Expanding Universe 00:05 Fish-Dang Bot 00:07 Everchanging Quest 00:10 Word Conquest 00:14 Singularity 00:17 Galactic Domination 00:20 Galactic Bridge 00:25 Apocalypse Expansion 00:28 Hexagon Tactics 00:31 Yabbit Attack 00:34 Singularity 00:39 Word Conquest 00:44 Everchanging Quest 01:07 Hexagon Tactics 01:12 Expanding Universe 01:15 Apocalypse Expansion 01:17 Snip It 01:20 Yabbit Attack 01:27 Announcing the Winner Open Source AI Game Jam, game development, artificial intelligence, open source, game design, game jam, AI algorithms, machine learning, game mechanics, game programming, game engines, game assets, game prototypes, game submission, community collaboration, indie game development, game industry, creative coding, open data, game challenges, game innovation, game enthusiasts, game AI, game mechanics, game graphics, game audio, game storytelling, game immersion, game experimentation, game optimization, game testing, game monetization, game marketing, game distribution, game analysis, game feedback, game updates, game community, game awards, game tutorials, game resources, game inspiration, game trends, game showcase

Welcome to the first ever Open Source AI Game Jam! Sign up at https://itch.io/jam/open-source-ai-game-jam Then join our discord: https://hf.co/join/discord

Real Image Latent Editing with Edit Friendly DDPM and Semantic Guidance https://huggingface.co/spaces/editing-images/ledits