Data & context
Data-Juicer
Uses reusable operators and versionable recipes across multimodal model-data preparation workflows.
Overview
Research summary
Data-Juicer turns data preparation into composable operator pipelines. Its documented operators cover text, image, audio, video and multimodal data, with workflows for filtering, deduplication, synthesis and analysis. YAML recipes and a Python interface make preprocessing reproducible and customizable. The project targets both local experimentation and distributed execution for larger corpora. Its stated use cases span pretraining, fine-tuning, evaluation data, agent traces and retrieval-oriented processing, with performance dependent on the selected operators and execution environment.
Repository summary
- Stars
- 7,110
- Open issues
- Unavailable
- Last push
- 2026-09-28
- Commits, 90 days
- Unavailable
- Repository activity
- Not scored
- Version
- Unavailable
Recorded catalogue figures. View repository data and provenance →
Classification
Pricing & services
Paid services unknown
Whether the provider offers paid products or services has not been established.
Licence scope
Implementation
Recorded implementation details and interfaces for Data-Juicer.
Implementation details
Python library and supporting tools
- Languages
- Python
- Repository type
- source
Recorded interfaces and capabilities
Licence scope
Repository
Repository snapshots, release information and recorded maintenance signals.
Repository snapshot
- Stars
- 7,110
- Open issues
- Unavailable
- Last push
- 2026-09-28
- Commits, 90 days
- Unavailable
- Repository activity
- Not scored
- Archived
- Not recorded
Repository activity is a snapshot, not a quality or popularity ranking. It combines recent-push freshness (50%), 90-day commits (30%) and issue pressure (20%).
Maintenance and provenance
- Catalogue snapshot
- 2026-10-06
- Stars source
- Recorded fallback
- Last push source
- Recorded fallback
Documentation
Recorded references and research provenance for this entry.
Recorded sources 5
- https://datajuicer.github.io/data-juicer/ Project page · Documentation
- https://github.com/datajuicer/data-juicer Linked repository · Research reference
- https://api.github.com/repos/datajuicer/data-juicer Research reference
- https://github.com/datajuicer/data-juicer/blob/main/README.md Research reference
- https://github.com/datajuicer/data-juicer/blob/main/LICENSE Research reference
Research metadata
- Research date
- 2026-10-02
- Catalogue snapshot
- 2026-10-06
Community & social 1 channels
Communities
- GitHub Discussionsgithub.com/datajuicer/data-juicer/discussionsOfficial
Link details for GitHub Discussions
- Last checked