[DRAFT] Feat/hls source support

[cotid-qualabs]

No description provided.

[cotid-qualabs]

/gemini summary

[gemini-code-assist]

Summary of Changes

This pull request significantly expands the media ingestion capabilities by integrating HLS (HTTP Live Streaming) as a primary source. It establishes a robust pipeline for consuming HLS streams, converting them into CMAF segments, and broadcasting them via MoQ. Key improvements include interactive controls for HLS resolution selection, enhanced UI elements in the demo for latency and quality management, and critical refinements to CMAF timestamp handling to ensure smooth and reliable playback from diverse HLS/CMAF sources.

Highlights

• HLS Source Support: Introduced comprehensive support for HLS (HTTP Live Streaming) as a new media source, enabling the system to ingest HLS master and media playlists and publish them as MoQ broadcasts.
• Interactive HLS Resolution Selection: Added a feature for interactive selection of HLS video resolutions using ffprobe, allowing users to choose specific renditions to ingest via the CLI.
• Demo UI Enhancements: The hang-demo application now includes user interface elements for adjusting playback latency and manually selecting video quality, providing better control and testing capabilities.
• CMAF Timestamp Normalization: Improved CMAF import logic to normalize timestamps per track, ensuring playback starts near zero and remains monotonic, which enhances robustness for HLS/CMAF sources with large timestamp offsets or jitter.
• Dependency Updates: Incorporated new and updated Rust dependencies, including m3u8-rs for HLS playlist parsing, reqwest for HTTP fetching, and various compression and TLS-related crates to support the new HLS ingestion pipeline.

Changelog

• js/hang-demo/src/index.html
  • Added 'Advanced options' section with latency slider and quality selector.
  • Updated hang-watch element with id="watch" and latency="5000".
• js/hang-demo/src/index.ts
  • Implemented JavaScript logic to wire up the new latency slider and quality selector.
  • Added dynamic population of quality options based on video catalog changes.
  • Introduced Effect from @kixelated/signals for reactive updates.
• js/hang/src/publish/video/encoder.ts
  • Changed Moq and Catalog imports to type * as for type-only imports.
  • Introduced isH264 variable to simplify codec checks.
  • Removed a blank line.
• js/hang/src/watch/audio/ring-buffer.ts
  • Added #origin field for timestamp normalization.
  • Modified write method to use a local origin for timestamps, improving robustness for HLS/CMAF sources.
  • Updated read method to stay in refill mode until enough samples are available, preventing early underflows.
• js/hang/src/watch/video/source.ts
  • Added #manualSelection signal for manual rendition override.
  • Introduced setActiveRendition method for external rendition selection.
  • Modified rendition selection logic to prioritize manual selection.
• justfile
  • Added ingest-hls recipe for ingesting live HLS media playlists.
  • Added ingest-hls-interactive recipe for interactive ffprobe-based resolution selection for HLS.
• rs/Cargo.lock
  • Added new dependencies: async-compression, compression-codecs, compression-core, crc32fast, flate2, m3u8-rs, hyper-rustls, simd-adler32, webpki-roots, reqwest.
  • Updated mp4-atom version from 0.9.0 to 0.9.1.
• rs/hang-cli/Cargo.toml
  • Added bytes, m3u8-rs, reqwest dependencies.
• rs/hang-cli/src/client.rs
  • Imported HlsConfig.
  • Modified client function to accept hls: Option<HlsConfig>.
  • Updated Import::new and import.init_from/import.read_from calls to handle Hls import type and hls configuration.
• rs/hang-cli/src/hls.rs
  • New file: Implements HLS ingestion logic for the CLI.
  • Defines ReqwestHlsFetcher for fetching HLS segments using reqwest.
  • Introduces HlsImporter to manage the HLS ingest loop.
  • Provides HlsResolution struct and FromStr implementation for parsing resolutions.
  • Includes run_ffprobe_and_select function for interactive resolution selection using ffprobe.
• rs/hang-cli/src/import.rs
  • Added Hls variant to ImportType enum.
  • Modified Import::new to accept hls: Option<HlsConfig> and instantiate HlsImporter for Hls type.
  • Updated init_from and read_from methods to handle Hls import type, passing None for input stream as HLS handles its own fetching.
• rs/hang-cli/src/main.rs
  • Imported hls module and HlsResolution.
  • Added HlsArgs struct for CLI arguments related to HLS.
  • Modified Command::Serve and Command::Publish to include HlsArgs.
  • Integrated run_ffprobe_and_select for interactive HLS resolution selection.
• rs/hang-cli/src/server.rs
  • Imported HlsConfig.
  • Modified server function to accept hls: Option<HlsConfig>.
  • Updated Import::new and import.init_from/import.read_from calls to handle Hls import type and hls configuration.
• rs/hang/Cargo.toml
  • Added url and m3u8-rs dependencies.
  • Removed url from dev-dependencies.
• rs/hang/examples/video.rs
  • Imported hang::catalog.
  • Used fully qualified catalog::VideoConfig and catalog::H264.
• rs/hang/src/catalog/video/h264.rs
  • Improved parsing of H264 codec strings by explicitly checking the 'avc1' prefix.
• rs/hang/src/cmaf/import.rs
  • Added AtomicU32 for NEXT_VIDEO_TRACK_ID and NEXT_AUDIO_TRACK_ID to generate unique track names.
  • Introduced #base_timestamp for each track to normalize timestamps.
  • Added with_catalog and catalog methods to Import for reusing existing catalogs.
  • Modified init to merge new renditions into the shared catalog.
  • Updated init_video and init_audio to use the new atomic track IDs.
  • Refactored H264 initialization into init_h264 helper function.
  • Improved audio keyframe logic to ensure a keyframe is always sent for the first audio sample in a fragment.
  • Enhanced timestamp difference logging to handle out-of-order timestamps.
• rs/hang/src/error.rs
  • Updated documentation to include 'ingest pipelines'.
  • Added Hls(String) error variant for HLS-specific errors.
• rs/hang/src/hls.rs
  • New file: Core HLS ingestion logic.
  • Defines HlsConfig, StepOutcome, and HlsFetcher trait.
  • Implements HlsIngest to manage fetching playlists, selecting variants, consuming segments, and feeding them to CMAF importers.
  • Includes logic for selecting video variants (preferring H.264) and audio renditions from master playlists.
  • Provides utility functions for URI resolution.
• rs/hang/src/lib.rs
  • Updated crate documentation to mention 'HLS Import'.
  • Added pub mod hls;.
• rs/justfile
  • Added ingest-hls and ingest-hls-interactive recipes, mirroring the justfile changes.

Activity

• The author cotid-qualabs requested a Gemini summary.