Product Website

Cosmos-Curate

A powerful video curation system that processes, analyzes, and organizes video content using advanced AI models and distributed computing.

Overview

Cosmos-Curate is a comprehensive solution for video processing and curation using state-of-the-art AI models,
which powers the training data generation for Cosmos at NVIDIA.
It is built on top of a framework optimized for GPU-accelerated streaming pipeline,
which is now open-sourced independently as Cosmos-Xenna.

Features

• Video Processing: Efficient video splitting, annotation, filtering, deduplication, and dataset generation
• AI-Powered Analysis: Advanced video analysis using multiple model families
• Distributed Computing: Scalable processing using Cosmos-Xenna built on top of Ray
• Cloud Integration: Support for various platforms
• Pipeline System: Modular and extensible pipeline architecture

Documentation

Comprehensive documentation is available under docs/ directory.

User Documentation

• End User Guide - For user to run video processing pipelines
• Reference Pipelines Guide - For user to configure video processing pipelines

Developer Documentation

• Developer Guide - For contributors
• Architecture Guide - For developer to understand the core architecture
• Pipeline Design Guide - For developer to add or modify pipelines

Directory Structure

cosmos-curate/
├── cosmos_curate/         # Curate implementation
│   ├── client              # CLI to run locally
│       ├── image_cli       # Docker image management
│       ├── local_cli       # Launch pipelines by running local container
│       ├── nvcf_cli        # Launch pipelines on NVIDIA cloud function
│       ├── slurm_cli       # Launch pipelines on Slurm cluster
│       ├── utils           # Common utilities for various CLI apps
│   ├── core/               # Core functionality
│       ├── cf              # Service entry point for a cloud function deployment
│       ├── interfaces      # Core base class to integrate model and define new pipelines
│       ├── managers        # CLIs to run inside the container to manage models, databases, etc.
│       ├── utils           # Common utilities for pipelines
│   ├── models/             # AI model inference implementations
│   ├── pipelines/          # Pipeline implementations
│       ├── examples/       # Minimal example pipelines to help understand the framework
│       ├── video/          # Reference pipelines for video curation
│   ├── scripts/            # Startup scripts in various deployment environments
├── cosmos-xenna            # Git submodule for https://github.com/nvidia-cosmos/cosmos-xenna
├── packages                # Dockerfiles and scripts related to packaging
│   ├── cosmos_curate      # Dockerfile template and conda environment recipes for building cosmos_curate image
├── tests                   # Tests for testing
│   ├── cosmos_curate             
│       ├── pipelines       # Tests for models and pipeline stages for cosmos_curate
│       ├── client          # Tests for client CLIs
├── examples                # Example configuration files and scripts

Support

For support and questions:

• Check the documentation
• Open an issue on GitHub

Acknowledgments

• cosmos-xenna team for the core library
• All contributors and users of the project

Responsible Use of AI Models

Responsible Use

License and Contact

This project will download and install additional third-party open source software projects. Review the license terms of these open source projects before use.

NVIDIA Cosmos source code is released under the Apache 2 License.

NVIDIA Cosmos models are released under the NVIDIA Open Model License. For a custom license, please contact [email protected].