Browsing: Training

What usually stops PyTorch training from scaling out to a cluster is the container, not the model code. Every container needs the same CUDA runtime, a PyTorch build compiled against it, NCCL (NVIDIA Collective Communications Library), the Elastic Fabric Adapter (EFA) userspace libraries, the AWS OFI NCCL plugin that connects NCCL to EFA, and a…

Apollo Research, an AI safety evaluation organization, says it has caught a frontier open-weight model reasoning about something no model has been documented doing before: how its own answers today might shape the training of its future self. The model is GLM-5.3, built by the Chinese AI lab Z.ai, and the finding landed in an…

When implementing an EEG machine-learning pipeline, there are many samples obtained from a small number of subjects. One recording can be divided into multiple windows, which then can be randomly split into train and test splits. This produces a sample size that might seem very big. Yet, if the same subject’s windows are present in…

Reinforcement learning (RL) post-training is becoming a standard step in building capable language model agents. Models learn to reason and act across sequences of steps by generating trajectories, receiving rewards, and updating their policy based on outcomes. Running this at scale, across multiple nodes with hundreds of GPU-hours of rollouts per training run, requires persistent…

In their 2024 International Conference on Machine Learning paper, Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews, Weixin Liang, Zachary Izzo, and their coauthors built a statistical method for estimating how much text in a large collection had been substantially written or rewritten by a…