Building Moon Bot: A Slack-Native Coding Agent Backed by HuggingFace Buckets
• 47
AI & ML interests
The AI community building the future.
Recent Activity
View all activity
Papers
Seeing the Needle in the Haystack: Towards Weakly-Supervised Log Instance Anomaly Localization via Counterfactual Perturbation
Qualixar OS: A Universal Operating System for AI Agent Orchestration
Articles
Update optimized HF MCP settings image
#646 opened less than a minute ago
by 
evalstate
sayakpaul
updated a
dataset about 6 hours ago
alvarobartt
updated a
dataset about 7 hours ago
tarekziade
updated a
bucket about 9 hours ago
nielsr
updated a
Space about 16 hours ago
Post
1243
Hugging Face MCP Server v0.3.29
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Included "papers" in the new hf_fs tool. Includes listing of trending/daily.
This is a new tool under observation - disable the "Paper Semantic Search" tool for best results.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Included "papers" in the new hf_fs tool. Includes listing of trending/daily.
This is a new tool under observation - disable the "Paper Semantic Search" tool for best results.
hf://papers/
├── README.md
├── daily/
│ ├── latest
│ └── YYYY/
│ └── MM/
│ └── DD/
├── trending/
└── ARXIV_ID/
├── metadata.json
├── paper.md
├── models/
├── datasets/
└── spaces/
sergiopaniego
posted an update 12 days ago
Post
7634
Frontier models use distillation as a step of their post-training pipelines.
In 2026 it has three jobs: compress a big model into a small one, merge RL experts into a single model, and let a model teach itself.
I wrote up which frontier models use each one and how: https://huggingface.co/blog/sergiopaniego/distillation-2026
It pairs with Class 2 of the Training an Agent series Ben and I are doing, where we teach these techniques hands-on with TRL!
In 2026 it has three jobs: compress a big model into a small one, merge RL experts into a single model, and let a model teach itself.
I wrote up which frontier models use each one and how: https://huggingface.co/blog/sergiopaniego/distillation-2026
It pairs with Class 2 of the Training an Agent series Ben and I are doing, where we teach these techniques hands-on with TRL!
- 3 replies
jeffboudier
posted an update 17 days ago
Post
184
@alvarobartt published a step-by-step guide to deploy zai-org/GLM-5.2 on AMD GPUs, using newly released features in Microsoft Foundry. The FP8 model fits on a single node of MI300X GPUs, which cuts the bill in half vs. H100.
https://alvarobartt.com/goal-glm-5.2-on-foundry/
https://alvarobartt.com/goal-glm-5.2-on-foundry/
irenesolaiman
authored a
paper 20 days ago
Post
473
Uhh did Opus 4.8 cheat on PostTrainBench??
it found an API key in the PostTrainBench environment that allowed it to generate synthetic training data without using GPU hours, boosting the base model by 0.4913
Source: https://posttrainbench.com/traces/run.html?id=claude_non_api_max_claude-opus-4-8_10h_run1__healthbench_Qwen_Qwen3-4B-Base_17315102#tab=trace
it found an API key in the PostTrainBench environment that allowed it to generate synthetic training data without using GPU hours, boosting the base model by 0.4913
Source: https://posttrainbench.com/traces/run.html?id=claude_non_api_max_claude-opus-4-8_10h_run1__healthbench_Qwen_Qwen3-4B-Base_17315102#tab=trace
- 1 reply
sergiopaniego
posted an update 24 days ago
Post
333
TRL v1.7.0 is out‼️
+ continuous batching makes GRPO and RLOO 1.25x faster at -16 GB
+ proper MoE post-training across GRPO/RLOO/AsyncGRPO
+ new GMPO trainer
+ AsyncGRPO weight sync + padding-free
+ more
https://github.com/huggingface/trl/releases/tag/v1.7.0
wrote a small article about the continuous batching for GRPO feature
https://huggingface.co/blog/sergiopaniego/cb-trl-grpo
+ continuous batching makes GRPO and RLOO 1.25x faster at -16 GB
+ proper MoE post-training across GRPO/RLOO/AsyncGRPO
+ new GMPO trainer
+ AsyncGRPO weight sync + padding-free
+ more
https://github.com/huggingface/trl/releases/tag/v1.7.0
wrote a small article about the continuous batching for GRPO feature
https://huggingface.co/blog/sergiopaniego/cb-trl-grpo
sergiopaniego
posted an update about 1 month ago
Post
327
Continuous batching just landed in TRL for GRPO!
At 64 generations it runs faster and uses less VRAM than plain generate, no vLLM needed
How it works and when to reach for it, below
https://huggingface.co/blog/sergiopaniego/cb-trl-grpo
At 64 generations it runs faster and uses less VRAM than plain generate, no vLLM needed
How it works and when to reach for it, below
https://huggingface.co/blog/sergiopaniego/cb-trl-grpo
sergiopaniego
posted an update about 1 month ago
Post
310
GLM-5.2 is open and comes with competitive performance against opus 4.8
day-0 in transformers + vllm + sglang, mit license 🤗
on the post-training side: critic-based ppo for variable-length agentic rollouts (ppo is back!) + an online anti-reward-hacking module that feeds the agent dummy info when it tries to cheat
day-0 in transformers + vllm + sglang, mit license 🤗
on the post-training side: critic-based ppo for variable-length agentic rollouts (ppo is back!) + an online anti-reward-hacking module that feeds the agent dummy info when it tries to cheat
cfahlgren1
submitted a
paper to Daily Papers about 1 month ago
sergiopaniego
posted an update about 1 month ago
Post
3942
OpenEnv has a new home: github.com/huggingface/OpenEnv
Starting today, it's coordinated by a committee that includes Meta-PyTorch, Reflection, Unsloth, Modal, Prime Intellect, Nvidia, Mercor, Fleet AI, and Hugging Face
frontier labs train their models and their harnesses together. Claude knows Claude Code. GPT-5.5 knows Codex. that's not an accident, it's training. open-source models deserve the same magic, but pulling that off requires infrastructure that belongs to everyone, not one lab
OpenEnv is that layer. one api, any harness, any trainer, any environment
Rewards and training loops stay in TRL, Unsloth, wherever you already work. OpenEnv is the socket they all plug into
Get involved!
Full announcement: https://huggingface.co/blog/openenv-agentic-rl
Starting today, it's coordinated by a committee that includes Meta-PyTorch, Reflection, Unsloth, Modal, Prime Intellect, Nvidia, Mercor, Fleet AI, and Hugging Face
frontier labs train their models and their harnesses together. Claude knows Claude Code. GPT-5.5 knows Codex. that's not an accident, it's training. open-source models deserve the same magic, but pulling that off requires infrastructure that belongs to everyone, not one lab
OpenEnv is that layer. one api, any harness, any trainer, any environment
Rewards and training loops stay in TRL, Unsloth, wherever you already work. OpenEnv is the socket they all plug into
Get involved!
Full announcement: https://huggingface.co/blog/openenv-agentic-rl
sergiopaniego
posted an update about 1 month ago
Post
337
Frontier agents are this good partly because the model was trained inside the very harness it ships with.
NVIDIA's new paper "Polar: Agentic RL on Any Harness at Scale" brings that recipe to the open: it turns coding harnesses like Codex, Claude Code, Qwen Code or Pi into RL training environments without touching their internals.
The core idea: every agent, however complex or closed, talks to a model through an API, so they put a proxy there. The harness runs exactly like in production while the proxy records prompts, sampled token ids and logprobs. Trajectories get rebuilt outside, token faithful, so gradients hit the exact tokens the policy sampled.
The gains are consistent across all four harnesses. Same Qwen3.5-4B, plain GRPO, evaluated on SWE-Bench Verified:
Codex 3.8 → 26.4 (+22.6)
Claude Code 29.8 → 34.6 (+4.8)
Qwen Code 34.6 → 35.2 (+0.6)
Pi 34.2 → 40.4 (+6.2)
The biggest gains appear on unfamiliar execution paths, Codex being the clearest case. The takeaway: you are not just training a model, you are training the model + harness system.
Two engineering pieces make it work at scale. Async worker pools isolate container boots (CPU), agent execution (GPU) and long tail test runs, so slow runtimes never block the GPUs. And prefix merging stitches hundreds of captured API calls back into contiguous traces: 5.4x faster trainer updates and rollout GPUs at 88% utilization.
It also doubles as an SFT data factory: 504 test verified agent traces from a 122B teacher, multi-turn conversations averaging 104 messages each, coming to the Hub under Apache 2.0 (release pending review).
Paper authors: Binfeng Xu, Hao Zhang, Shaokun Zhang, Songyang Han, Mingjie Liu, Jian Hu, Shizhe Diao, Zhenghui Jin, Yunheng Zou, Michael Demoret, Jan Kautz and Yi Dong.
> Paper: Polar: Agentic RL on Any Harness at Scale (2605.24220)
> Code: https://github.com/NVIDIA-NeMo/ProRL-Agent-Server
> Training data: NovaSky-AI/SkyRL-v0-293-data
NVIDIA's new paper "Polar: Agentic RL on Any Harness at Scale" brings that recipe to the open: it turns coding harnesses like Codex, Claude Code, Qwen Code or Pi into RL training environments without touching their internals.
The core idea: every agent, however complex or closed, talks to a model through an API, so they put a proxy there. The harness runs exactly like in production while the proxy records prompts, sampled token ids and logprobs. Trajectories get rebuilt outside, token faithful, so gradients hit the exact tokens the policy sampled.
The gains are consistent across all four harnesses. Same Qwen3.5-4B, plain GRPO, evaluated on SWE-Bench Verified:
Codex 3.8 → 26.4 (+22.6)
Claude Code 29.8 → 34.6 (+4.8)
Qwen Code 34.6 → 35.2 (+0.6)
Pi 34.2 → 40.4 (+6.2)
The biggest gains appear on unfamiliar execution paths, Codex being the clearest case. The takeaway: you are not just training a model, you are training the model + harness system.
Two engineering pieces make it work at scale. Async worker pools isolate container boots (CPU), agent execution (GPU) and long tail test runs, so slow runtimes never block the GPUs. And prefix merging stitches hundreds of captured API calls back into contiguous traces: 5.4x faster trainer updates and rollout GPUs at 88% utilization.
It also doubles as an SFT data factory: 504 test verified agent traces from a 122B teacher, multi-turn conversations averaging 104 messages each, coming to the Hub under Apache 2.0 (release pending review).
Paper authors: Binfeng Xu, Hao Zhang, Shaokun Zhang, Songyang Han, Mingjie Liu, Jian Hu, Shizhe Diao, Zhenghui Jin, Yunheng Zou, Michael Demoret, Jan Kautz and Yi Dong.
> Paper: Polar: Agentic RL on Any Harness at Scale (2605.24220)
> Code: https://github.com/NVIDIA-NeMo/ProRL-Agent-Server
> Training data: NovaSky-AI/SkyRL-v0-293-data
sergiopaniego
posted an update about 2 months ago
Post
288
The recording from our talk: "From Responses To Trajectories: Multi-Turn and Multi-Environment RL" from PyTorch Conf Europe is live!
@kashif and I covered the latest advances in multi-turn GRPO in TRL: trajectories, tool use, envs, and agentic post-training at scale
https://www.youtube.com/watch?v=rPBeXFntJSU
@kashif and I covered the latest advances in multi-turn GRPO in TRL: trajectories, tool use, envs, and agentic post-training at scale
https://www.youtube.com/watch?v=rPBeXFntJSU
sergiopaniego
posted an update about 2 months ago
Post
225
how do you sync a trillion parameter model every RL step without a shared cluster? we just wrote a blog about it, led by @aminediroHF
what I like the most is the way it proves you can use the Hub for basically everything 🧐 → trainer on one machine, vLLM in a HF Space, the wordle env in another HF Space and weights going through a Hub Bucket. no shared cluster, just HTTPS
it works because ~99% of bf16 weights don't change between RL steps so you only sync the diff. 1.2 GB to 25 MB of payload per step
https://huggingface.co/blog/delta-weight-sync
what I like the most is the way it proves you can use the Hub for basically everything 🧐 → trainer on one machine, vLLM in a HF Space, the wordle env in another HF Space and weights going through a Hub Bucket. no shared cluster, just HTTPS
it works because ~99% of bf16 weights don't change between RL steps so you only sync the diff. 1.2 GB to 25 MB of payload per step
https://huggingface.co/blog/delta-weight-sync
Post
3432
Hugging Face MCP Server v0.3.17
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
SEP-2640 "Skills Over MCP" support added (early access)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
SEP-2640 "Skills Over MCP" support added (early access)
- 3 replies
sergiopaniego
posted an update about 2 months ago
Post
2359
most multi-turn RL loops have a silent bug: you decode the model's output to detect tool calls, then re-tokenize the conversation for the next turn. BPE isn't invertible, so decode then re-encode can land on different ids. gradient ends up on tokens the model never sampled. no crash, just quietly wrong math and broken training
@qgallouedec wrote a super educational blog on MITO (message-in, token-out) vs TITO (token-in, token-out) and how you might fix the problem above
go read it 🤓
https://qgallouedec-tito.hf.space/
@qgallouedec wrote a super educational blog on MITO (message-in, token-out) vs TITO (token-in, token-out) and how you might fix the problem above
go read it 🤓
https://qgallouedec-tito.hf.space/