ossrules.md

weights-and-biases

W&B: log ML experiments, sweeps, model registry, dashboards.

  • MLOps
  • Weights And Biases
  • WandB
  • Experiment Tracking
  • Hyperparameter Tuning
  • Model Registry
  • Collaboration
  • Real-Time Visualization
  • PyTorch
  • TensorFlow
  • HuggingFace

Declared platforms: linux · macos · windows

Install
npx skills add 'https://github.com/NousResearch/hermes-agent/tree/main/optional-skills/mlops/evaluation/weights-and-biases'
Download bundle ↓
main · 24fd22b ↗Scanned 2026-09-15

Contributors

GitHub-linked commit authors for this SKILL.md at the saved revision. Co-authors and history before file renames are not included.

File history ↗

SKILL.md

SKILL.mdBrowse 4 files
View on GitHub
3,321 tokens

12,676 bytes
Token encoding: o200k_base
Snapshot 24fd22b

---name: weights-and-biasesdescription: "W&B: log ML experiments, sweeps, model registry, dashboards."version: 1.0.1author: Orchestra Researchlicense: MITdependencies: [wandb]platforms: [linux, macos, windows]metadata:  hermes:    tags: [MLOps, Weights And Biases, WandB, Experiment Tracking, Hyperparameter Tuning, Model Registry, Collaboration, Real-Time Visualization, PyTorch, TensorFlow, HuggingFace] --- # Weights & Biases: ML Experiment Tracking & MLOps ## When to Use This Skill Use Weights & Biases (W&B) when you need to:- **Track ML experiments** with automatic metric logging- **Visualize training** in real-time dashboards- **Compare runs** across hyperparameters and configurations- **Optimize hyperparameters** with automated sweeps- **Manage model registry** with versioning and lineage- **Collaborate on ML projects** with team workspaces- **Track artifacts** (datasets, models, code) with lineage **Users**: 200,000+ ML practitioners | **GitHub Stars**: 10.5k+ | **Integrations**: 100+ ## Installation ```bash# Install W&Bpip install wandb # Login (creates API key)wandb login # Or set API key programmaticallyexport WANDB_API_KEY=your_api_key_here``` ## Quick Start ### Basic Experiment Tracking ```pythonimport wandb # Initialize a runrun = wandb.init(    project="my-project",    config={        "learning_rate": 0.001,        "epochs": 10,        "batch_size": 32,        "architecture": "ResNet50"    }) # Training loopfor epoch in range(run.config.epochs):    # Your training code    train_loss = train_epoch()    val_loss = validate()     # Log metrics    wandb.log({        "epoch": epoch,        "train/loss": train_loss,        "val/loss": val_loss,        "train/accuracy": train_acc,        "val/accuracy": val_acc    }) # Finish the runwandb.finish()``` ### With PyTorch ```pythonimport torchimport wandb # Initializewandb.init(project="pytorch-demo", config={    "lr": 0.001,    "epochs": 10}) # Access configconfig = wandb.config # Training loopfor epoch in range(config.epochs):    for batch_idx, (data, target) in enumerate(train_loader):        # Forward pass        output = model(data)        loss = criterion(output, target)         # Backward pass        optimizer.zero_grad()        loss.backward()        optimizer.step()         # Log every 100 batches        if batch_idx % 100 == 0:            wandb.log({                "loss": loss.item(),                "epoch": epoch,                "batch": batch_idx            }) # Save modeltorch.save(model.state_dict(), "model.pth")wandb.save("model.pth")  # Upload to W&B wandb.finish()``` ## Core Concepts ### 1. Projects and Runs **Project**: Collection of related experiments**Run**: Single execution of your training script ```python# Create/use projectrun = wandb.init(    project="image-classification",    name="resnet50-experiment-1",  # Optional run name    tags=["baseline", "resnet"],    # Organize with tags    notes="First baseline run"      # Add notes) # Each run has unique IDprint(f"Run ID: {run.id}")print(f"Run URL: {run.url}")``` ### 2. Configuration Tracking Track hyperparameters automatically: ```pythonconfig = {    # Model architecture    "model": "ResNet50",    "pretrained": True,     # Training params    "learning_rate": 0.001,    "batch_size": 32,    "epochs": 50,    "optimizer": "Adam",     # Data params    "dataset": "ImageNet",    "augmentation": "standard"} wandb.init(project="my-project", config=config) # Access config during traininglr = wandb.config.learning_ratebatch_size = wandb.config.batch_size``` ### 3. Metric Logging ```python# Log scalarswandb.log({"loss": 0.5, "accuracy": 0.92}) # Log multiple metricswandb.log({    "train/loss": train_loss,    "train/accuracy": train_acc,    "val/loss": val_loss,    "val/accuracy": val_acc,    "learning_rate": current_lr,    "epoch": epoch}) # Log with custom x-axiswandb.log({"loss": loss}, step=global_step) # Log media (images, audio, video)wandb.log({"examples": [wandb.Image(img) for img in images]}) # Log histogramswandb.log({"gradients": wandb.Histogram(gradients)}) # Log tablestable = wandb.Table(columns=["id", "prediction", "ground_truth"])wandb.log({"predictions": table})``` ### 4. Model Checkpointing ```pythonimport torchimport wandb # Save model checkpointcheckpoint = {    'epoch': epoch,    'model_state_dict': model.state_dict(),    'optimizer_state_dict': optimizer.state_dict(),    'loss': loss,} torch.save(checkpoint, 'checkpoint.pth') # Upload to W&Bwandb.save('checkpoint.pth') # Or use Artifacts (recommended)artifact = wandb.Artifact('model', type='model')artifact.add_file('checkpoint.pth')wandb.log_artifact(artifact)``` ## Hyperparameter Sweeps Automatically search for optimal hyperparameters. ### Define Sweep Configuration ```pythonsweep_config = {    'method': 'bayes',  # or 'grid', 'random'    'metric': {        'name': 'val/accuracy',        'goal': 'maximize'    },    'parameters': {        'learning_rate': {            'distribution': 'log_uniform_values',            'min': 1e-5,            'max': 1e-1        },        'batch_size': {            'values': [16, 32, 64, 128]        },        'optimizer': {            'values': ['adam', 'sgd', 'rmsprop']        },        'dropout': {            'distribution': 'uniform',            'min': 0.1,            'max': 0.5        }    }} # Initialize sweepsweep_id = wandb.sweep(sweep_config, project="my-project")``` ### Define Training Function ```pythondef train():    # Initialize run    run = wandb.init()     # Access sweep parameters    lr = wandb.config.learning_rate    batch_size = wandb.config.batch_size    optimizer_name = wandb.config.optimizer     # Build model with sweep config    model = build_model(wandb.config)    optimizer = get_optimizer(optimizer_name, lr)     # Training loop    for epoch in range(NUM_EPOCHS):        train_loss = train_epoch(model, optimizer, batch_size)        val_acc = validate(model)         # Log metrics        wandb.log({            "train/loss": train_loss,            "val/accuracy": val_acc        }) # Run sweepwandb.agent(sweep_id, function=train, count=50)  # Run 50 trials``` ### Sweep Strategies ```python# Grid search - exhaustivesweep_config = {    'method': 'grid',    'parameters': {        'lr': {'values': [0.001, 0.01, 0.1]},        'batch_size': {'values': [16, 32, 64]}    }} # Random searchsweep_config = {    'method': 'random',    'parameters': {        'lr': {'distribution': 'uniform', 'min': 0.0001, 'max': 0.1},        'dropout': {'distribution': 'uniform', 'min': 0.1, 'max': 0.5}    }} # Bayesian optimization (recommended)sweep_config = {    'method': 'bayes',    'metric': {'name': 'val/loss', 'goal': 'minimize'},    'parameters': {        'lr': {'distribution': 'log_uniform_values', 'min': 1e-5, 'max': 1e-1}    }}``` ## Artifacts Track datasets, models, and other files with lineage. ### Log Artifacts ```python# Create artifactartifact = wandb.Artifact(    name='training-dataset',    type='dataset',    description='ImageNet training split',    metadata={'size': '1.2M images', 'split': 'train'}) # Add filesartifact.add_file('data/train.csv')artifact.add_dir('data/images/') # Log artifactwandb.log_artifact(artifact)``` ### Use Artifacts ```python# Download and use artifactrun = wandb.init(project="my-project") # Download artifactartifact = run.use_artifact('training-dataset:latest')artifact_dir = artifact.download() # Use the datadata = load_data(f"{artifact_dir}/train.csv")``` ### Model Registry ```python# Log model as artifactmodel_artifact = wandb.Artifact(    name='resnet50-model',    type='model',    metadata={'architecture': 'ResNet50', 'accuracy': 0.95}) model_artifact.add_file('model.pth')wandb.log_artifact(model_artifact, aliases=['best', 'production']) # Link to model registryrun.link_artifact(model_artifact, 'model-registry/production-models')``` ## Integration Examples ### HuggingFace Transformers ```pythonfrom transformers import Trainer, TrainingArgumentsimport wandb # Initialize W&Bwandb.init(project="hf-transformers") # Training arguments with W&Btraining_args = TrainingArguments(    output_dir="./results",    report_to="wandb",  # Enable W&B logging    run_name="bert-finetuning",    logging_steps=100,    save_steps=500) # Trainer automatically logs to W&Btrainer = Trainer(    model=model,    args=training_args,    train_dataset=train_dataset,    eval_dataset=eval_dataset) trainer.train()``` ### PyTorch Lightning ```pythonfrom pytorch_lightning import Trainerfrom pytorch_lightning.loggers import WandbLoggerimport wandb # Create W&B loggerwandb_logger = WandbLogger(    project="lightning-demo",    log_model=True  # Log model checkpoints) # Use with Trainertrainer = Trainer(    logger=wandb_logger,    max_epochs=10) trainer.fit(model, datamodule=dm)``` ### Keras/TensorFlow ```pythonimport wandbfrom wandb.integration.keras import WandbMetricsLogger, WandbModelCheckpoint # Initializewandb.init(project="keras-demo") # Add callbacks (the monolithic WandbCallback was removed;# use the dedicated callbacks from wandb.integration.keras instead)model.fit(    x_train, y_train,    validation_data=(x_val, y_val),    epochs=10,    callbacks=[        WandbMetricsLogger(),                        # Auto-logs metrics        WandbModelCheckpoint("models/model-{epoch}")  # Saves checkpoints    ])``` ## Visualization & Analysis ### Custom Charts ```python# Log custom visualizationsimport matplotlib.pyplot as plt fig, ax = plt.subplots()ax.plot(x, y)wandb.log({"custom_plot": wandb.Image(fig)}) # Log confusion matrixwandb.log({"conf_mat": wandb.plot.confusion_matrix(    probs=None,    y_true=ground_truth,    preds=predictions,    class_names=class_names)})``` ### Reports Create shareable reports in W&B UI:- Combine runs, charts, and text- Markdown support- Embeddable visualizations- Team collaboration ## Best Practices ### 1. Organize with Tags and Groups ```pythonwandb.init(    project="my-project",    tags=["baseline", "resnet50", "imagenet"],    group="resnet-experiments",  # Group related runs    job_type="train"             # Type of job)``` ### 2. Log Everything Relevant ```python# Log system metricswandb.log({    "gpu/util": gpu_utilization,    "gpu/memory": gpu_memory_used,    "cpu/util": cpu_utilization}) # Log code versionwandb.log({"git_commit": git_commit_hash}) # Log data splitswandb.log({    "data/train_size": len(train_dataset),    "data/val_size": len(val_dataset)})``` ### 3. Use Descriptive Names ```python# ✅ Good: Descriptive run nameswandb.init(    project="nlp-classification",    name="bert-base-lr0.001-bs32-epoch10") # ❌ Bad: Generic nameswandb.init(project="nlp", name="run1")``` ### 4. Save Important Artifacts ```python# Save final modelartifact = wandb.Artifact('final-model', type='model')artifact.add_file('model.pth')wandb.log_artifact(artifact) # Save predictions for analysispredictions_table = wandb.Table(    columns=["id", "input", "prediction", "ground_truth"],    data=predictions_data)wandb.log({"predictions": predictions_table})``` ### 5. Use Offline Mode for Unstable Connections ```pythonimport os # Enable offline modeos.environ["WANDB_MODE"] = "offline" wandb.init(project="my-project")# ... your code ... # Sync later# wandb sync <run_directory>``` ## Team Collaboration ### Share Runs ```python# Runs are automatically shareable via URLrun = wandb.init(project="team-project")print(f"Share this URL: {run.url}")``` ### Team Projects - Create team account at wandb.ai- Add team members- Set project visibility (private/public)- Use team-level artifacts and model registry ## Pricing - **Free**: Unlimited public projects, 100GB storage- **Academic**: Free for students/researchers- **Teams**: $50/seat/month, private projects, unlimited storage- **Enterprise**: Custom pricing, on-prem options ## Resources - **Documentation**: https://docs.wandb.ai- **GitHub**: https://github.com/wandb/wandb (10.5k+ stars)- **Examples**: https://github.com/wandb/examples- **Community**: https://wandb.ai/community- **Discord**: https://wandb.me/discord ## See Also - `references/sweeps.md` - Comprehensive hyperparameter optimization guide- `references/artifacts.md` - Data and model versioning patterns- `references/integrations.md` - Framework-specific examples
Discovery context

Discovered by repository scan. No exact path reference found in the snapshot’s root AGENTS.md.

← All Hermes Agent skillsProject instructions →