Datasets

Training datasets with quantum-safe provenance

aline-gassenn/MedDialog-Audio HF Unverified

MedDialogue-Audio English Medical Dialogue Corpus for Speech Recognition Research. This repository contains MedDialogue-Audio, an English audio corpus designed for research in Automatic Speech Recognition (ASR) in the healthcare domain. The dataset was published in the proceedings of the 7th SBBD Dataset Showcase Workshop, and is available online at the following link: https://sol.sbc.org.br/index.php/dsw/article/view/37199 Dataset Description MedDialogue-Audio is… See the full description on the dataset page: https://huggingface.co/datasets/aline-gassenn/MedDialog-Audio.

Task_categories:automatic-Speech-RecognitionLanguage:enSize_categories:100K<n<1MDoi:10.57967/hf/5889Medical
aps/super_glue HF Unverified

Dataset Card for "super_glue" Dataset Summary SuperGLUE (https://super.gluebenchmark.com/) is a new benchmark styled after GLUE with a new set of more difficult language understanding tasks, improved resources, and a new public leaderboard. Supported Tasks and Leaderboards More Information Needed Languages More Information Needed Dataset Structure Data Instances axb Size of downloaded dataset files: 0.03 MB Size of… See the full description on the dataset page: https://huggingface.co/datasets/aps/super_glue.

Task_categories:text-ClassificationTask_categories:token-ClassificationTask_categories:question-AnsweringTask_ids:natural-Language-InferenceTask_ids:word-Sense-DisambiguationTask_ids:coreference-Resolution
InternRobotics/OmniWorld HF Unverified

[ICLR 2026] OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modeling         🎉NEWS [2026.3.21] 🔥 OmniWorld-Game with Metric Scale is now released! Check out our latest model Pi3X (an enhanced version of Pi3), which leverages this data to achieve better performance! [2026.1.26] 🎉 OmniWorld was accepted by ICLR 2026! [2026.1.7] Update OmniWorld-Game, release RH20T-Robot, RH20T-Human, Ego-Exo4D, EgoDex, Epic-Kitchens. [2025.11.11] The OmniWorld is… See the full description on the dataset page: https://huggingface.co/datasets/InternRobotics/OmniWorld.

Task_categories:text-To-VideoTask_categories:image-To-VideoTask_categories:image-To-3dTask_categories:roboticsTask_categories:otherLanguage:en
aoiandroid/android-times-articles HF Unverified

Android Times — Articles Dataset Archive Private dataset containing synthesized and processed article archives, multi-language transcripts, metadata, and editorial assets for Android Times. Dataset Structure articles/ ├── en-US/ # English (United States) localized articles & scripts ├── ja-JP/ # Japanese localized articles & scripts ├── en-AU/ # Australian localized articles ├── en-CN/ # China localized English… See the full description on the dataset page: https://huggingface.co/datasets/aoiandroid/android-times-articles.

Task_categories:text-GenerationTask_categories:summarizationTask_categories:translationLanguage:enLanguage:jaLanguage:zh
OpenGVLab/GUI-Odyssey HF Unverified

Dataset Card for GUI Odyssey News⭐️ A new and improved version of the GUIOdyssey dataset has been released! 🎉🎉 👉 Please use the latest version and refer to the updated README for the most up-to-date information. We highly recommend using the new version for all training and evaluation! Repository: https://github.com/OpenGVLab/GUI-Odyssey Latest Version of Dataset: hflqf88888/GUIOdyssey Paper: https://arxiv.org/pdf/2406.08451 Introduction GUI Odyssey is… See the full description on the dataset page: https://huggingface.co/datasets/OpenGVLab/GUI-Odyssey.

Language:enSize_categories:1K<n<10KFormat:jsonModality:imageModality:tabularModality:text
PleIAs/common_corpus HF Unverified

Common Corpus Full paper - ICLR 2026 oral Common Corpus is the largest open and permissible licensed text dataset, comprising 2.27 trillion tokens (2,267,302,720,836 tokens). It is a diverse dataset, consisting of books, newspapers, scientific articles, government and legal documents, code, and more. Common Corpus has been created by Pleias in association with several partners. Common Corpus differs from existing open datasets in that it is: Truly Open: contains only data that… See the full description on the dataset page: https://huggingface.co/datasets/PleIAs/common_corpus.

Language:enLanguage:frLanguage:deLanguage:zhLanguage:itLanguage:es
HPLT/HPLT2.0_cleaned HF Unverified

NB: HPLT2.0 is now superseded by a newer release: HPLT3.0 We recommed switching to v3.0, unless you have a compelling reason to stay on 2.0. This is a large-scale collection of web-crawled documents in 191 world languages, produced by the HPLT project. The source of the data is mostly Internet Archive with some additions from Common Crawl. For a detailed description of the dataset, please refer to our website and our pre-print. The Cleaned variant of HPLT Datasets v2.0 This is… See the full description on the dataset page: https://huggingface.co/datasets/HPLT/HPLT2.0_cleaned.

Task_categories:fill-MaskTask_categories:text-GenerationTask_ids:language-ModelingMultilinguality:multilingualLanguage:aceLanguage:af
AI-MO/NuminaMath-CoT HF Unverified

Dataset Card for NuminaMath CoT Dataset Summary Approximately 860k math problems, where each solution is formatted in a Chain of Thought (CoT) manner. The sources of the dataset range from Chinese high school math exercises to US and international mathematics olympiad competition problems. The data were primarily collected from online exam paper PDFs and mathematics discussion forums. The processing steps include (a) OCR from the original PDFs, (b) segmentation… See the full description on the dataset page: https://huggingface.co/datasets/AI-MO/NuminaMath-CoT.

Task_categories:text-GenerationLanguage:enSize_categories:100K<n<1MFormat:parquetModality:textLibrary:datasets
SwayStar123/preprocessed_commoncatalog-cc-by HF Unverified

I also seperately provide just the prompts in prompts.json keys are the image_id, and the values are the captions generated Captions generated by moondream: vikhyatk/moondream2 Latents generated by SDXL VAE: madebyollin/sdxl-vae-fp16-fix Embeddings generated by SigLIP: hf-hub:timm/ViT-SO400M-14-SigLIP-384 Original dataset: common-canvas/commoncatalog-cc-by Latents f32 and embeddings are f16 bytes Compute cost: 16x3090 for 3 day. Approximately.

Language:enSize_categories:10M<n<100MFormat:parquetModality:textLibrary:datasetsLibrary:dask
secemp9/arxiv-complete HF Unverified

arXiv Complete Corpus A snapshot of arXiv's metadata, version history, submission files and rendered documents. It covers 3,148,796 papers and includes file contents, paths, sizes and SHA-256 digests. Metadata comes from arXiv's OAI-PMH arXivRaw interface; files come from the GCS mirror, S3 source archives and direct PDF fetches. This release holds a PDF for 99.47% of papers and 99.54% of versions reported with a non-zero submission size. It is a one-off snapshot; coverage gaps… See the full description on the dataset page: https://huggingface.co/datasets/secemp9/arxiv-complete.

Task_categories:text-GenerationTask_categories:text-RetrievalLanguage:enSize_categories:100M<n<1BFormat:parquetModality:tabular
HuggingFaceM4/the_cauldron HF Unverified

Dataset Card for The Cauldron Dataset description The Cauldron is part of the Idefics2 release. It is a massive collection of 50 vision-language datasets (training sets only) that were used for the fine-tuning of the vision-language model Idefics2. Load the dataset To load the dataset, install the library datasets with pip install datasets. Then, from datasets import load_dataset ds = load_dataset("HuggingFaceM4/the_cauldron", "ai2d") to download and load the… See the full description on the dataset page: https://huggingface.co/datasets/HuggingFaceM4/the_cauldron.

Size_categories:1M<n<10MFormat:parquetModality:imageModality:textLibrary:datasetsLibrary:dask
roneneldan/TinyStories HF Unverified

Dataset containing synthetically generated (by GPT-3.5 and GPT-4) short stories that only use a small vocabulary. Described in the following paper: https://arxiv.org/abs/2305.07759. The models referred to in the paper were trained on TinyStories-train.txt (the file tinystories-valid.txt can be used for validation loss). These models can be found on Huggingface, at roneneldan/TinyStories-1M/3M/8M/28M/33M/1Layer-21M. Additional resources: tinystories_all_data.tar.gz - contains a superset of… See the full description on the dataset page: https://huggingface.co/datasets/roneneldan/TinyStories.

Task_categories:text-GenerationLanguage:enSize_categories:1M<n<10MFormat:parquetModality:textLibrary:datasets
isaacus/open-australian-legal-corpus HF Unverified

Open Australian Legal Corpus ‍⚖️ The Open Australian Legal Corpus by Isaacus, a foundational legal AI research company, is the first and only multijurisdictional open corpus of Australian legislative and judicial documents. Comprised of 229,122 texts totalling over 60 million lines and 1.4 billion tokens, the Corpus includes every in force statute and regulation in the Commonwealth, New South Wales, Queensland, Western Australia, South Australia, Tasmania and Norfolk Island, in… See the full description on the dataset page: https://huggingface.co/datasets/isaacus/open-australian-legal-corpus.

Task_categories:text-GenerationTask_categories:fill-MaskTask_categories:text-RetrievalTask_ids:language-ModelingTask_ids:masked-Language-ModelingTask_ids:document-Retrieval
configinc/HABIT HF Unverified

HABIT: Human-Aware Behavior and Interaction Training Dataset for Robot Manipulation TL;DR: HABIT is a large-scale robot demonstration dataset for human-present environments, designed to teach robot policies human-aware behaviors. Keywords: Robot Manipulation Dataset, Human-Robot Interaction, Vision-Language-Action Model Overview Data-driven approaches have emerged as a promising direction for training robotic manipulation policies. Recent robot datasets have… See the full description on the dataset page: https://huggingface.co/datasets/configinc/HABIT.

Task_categories:roboticsLanguage:enSize_categories:1M<n<10MFormat:parquetModality:tabularModality:text
nebius/SWE-rebench-V2-PRs HF Unverified

SWE-rebench-V2-PRs Dataset Summary SWE-rebench-V2-PRs is a large-scale dataset of real-world GitHub pull requests collected across multiple programming languages, intended for training and evaluating code-generation and software-engineering agents. The dataset contains 126,300 samples covering Go, Python, JavaScript, TypeScript, Rust, Java, C, C++, Julia, Elixir, Kotlin, PHP, Scala, Clojure, Dart, OCaml, and other languages. For log parser functions, base Dockerfiles, and… See the full description on the dataset page: https://huggingface.co/datasets/nebius/SWE-rebench-V2-PRs.

Task_categories:text-GenerationLanguage:enSize_categories:100K<n<1MFormat:parquetModality:textLibrary:datasets
Voxel51/OxfordFlowers102 HF Unverified

Dataset Card for Oxford Flowers 102 This is a FiftyOne dataset with 8189 samples. Installation If you haven't already, install FiftyOne: pip install -U fiftyone Usage import fiftyone as fo import fiftyone.utils.huggingface as fouh # Load the dataset # Note: other available arguments include 'max_samples', etc dataset = fouh.load_from_hub("Voxel51/OxfordFlowers102") # Launch the App session = fo.launch_app(dataset) Dataset Details… See the full description on the dataset page: https://huggingface.co/datasets/Voxel51/OxfordFlowers102.

Task_categories:image-ClassificationTask_categories:image-SegmentationLanguage:enSize_categories:10K<n<100KFormat:imagefolderModality:image
mlfoundations/MINT-1T-PDF-CC-2024-18 HF PQC Verified

🍃 MINT-1T:Scaling Open-Source Multimodal Data by 10x: A Multimodal Dataset with One Trillion Tokens 🍃 MINT-1T is an open-source Multimodal INTerleaved dataset with 1 trillion text tokens and 3.4 billion images, a 10x scale-up from existing open-source datasets. Additionally, we include previously untapped sources such as PDFs and ArXiv papers. 🍃 MINT-1T is designed to facilitate research in multimodal pretraining. 🍃 MINT-1T is created by a team from the University of Washington in… See the full description on the dataset page: https://huggingface.co/datasets/mlfoundations/MINT-1T-PDF-CC-2024-18.

Task_categories:image-To-TextTask_categories:text-GenerationLanguage:enSize_categories:100B<n<1TMultimodal
google/IFEval HF Unverified

Dataset Card for IFEval Dataset Summary This dataset contains the prompts used in the Instruction-Following Eval (IFEval) benchmark for large language models. It contains around 500 "verifiable instructions" such as "write in more than 400 words" and "mention the keyword of AI at least 3 times" which can be verified by heuristics. To load the dataset, run: from datasets import load_dataset ifeval = load_dataset("google/IFEval") Supported Tasks and… See the full description on the dataset page: https://huggingface.co/datasets/google/IFEval.

Task_categories:text-GenerationLanguage:enSize_categories:n<1KFormat:jsonModality:textLibrary:datasets
allenai/openbookqa HF Unverified

Dataset Card for OpenBookQA Dataset Summary OpenBookQA aims to promote research in advanced question-answering, probing a deeper understanding of both the topic (with salient facts summarized as an open book, also provided with the dataset) and the language it is expressed in. In particular, it contains questions that require multi-step reasoning, use of additional common and commonsense knowledge, and rich text comprehension. OpenBookQA is a new kind of… See the full description on the dataset page: https://huggingface.co/datasets/allenai/openbookqa.

Task_categories:question-AnsweringTask_ids:open-Domain-QaAnnotations_creators:crowdsourcedAnnotations_creators:expert-GeneratedLanguage_creators:expert-GeneratedMultilinguality:monolingual
Voxel51/Food101 HF Unverified

Dataset Card for Food-101 This is a FiftyOne dataset with 35000 samples. Note: This dataset is subset of the full Food101 dataset. The recipe notebook for creating this dataset can be found here Installation If you haven't already, install FiftyOne: pip install -U fiftyone Usage import fiftyone as fo import fiftyone.utils.huggingface as fouh # Load the dataset # Note: other available arguments include 'max_samples', etc dataset =… See the full description on the dataset page: https://huggingface.co/datasets/Voxel51/Food101.

Task_categories:image-ClassificationLanguage:enSize_categories:10K<n<100KFormat:imagefolderModality:imageLibrary:datasets
Showing 20 of 341 datasets (page 6 of 18)