Curated list of datasets
Curated list of datasets — notes by Théo Morales
Requirements
In my research on joint hand-object pose estimation, and because my focus is on object-specific adaptation, I find myself in the need of different datasets than the typical use. There are various hand-object interaction datasets out there, but only a few meet my requirements:
- Provide the ground-truth for the 2D or 3D hand joint poses (21 joints)
- Provide the ground-truth for the 6D object pose
- Contain several object categories/shapes; the more the better
- Provide RGB images
For my current experiments with object-specific adaptation with meta-learning, I need datasets that contain as many different objects as possible, in order to exploit the advantage of meta-learning methods for few-shot learning, since the goal is to demonstrate the potential increase in the ability of a model to generalize to unseen objects.
Curated list of datasets
| Dataset | # subjects | # frames | # objects | Nature | Year |
|---|---|---|---|---|---|
| SynthHands | 2 | 63,350 | 7 | Synthetic | 2017 |
| FPHAD | 6 | 100,000 | 4 | Real | 2018 |
| FHAB | 6 | 105,459 | 26 (4 annotated) | Real | 2018 |
| ObMan | N/A | 150,000 | 8 | Synthetic | 2019 |
| ContactPose | 50 | 2.9M | 25 | Hybrid | 2020 |
| GRAB | 10 | 1.6M | 51 | Mocap | 2020 |
| HO3D | 10 | 77,558 | 10 | Real | 2021 |
| H2O | 4 | 571,645 | 8 | Real | 2021 |
| DexYCB | 10 | 582,000 | 20 | Real | 2021 |
| OakInk | 12 | 230,000 | 100 | Real | 2022 |
| HOI4D | 9 | 2.4M | 800 | Real egocentric | 2022 |
| ARCTIC | 10 | 2.1M | 11 | Real | 2023 |
Related datasets not meeting the requirements
| Dataset | # subjects | # frames | # objects | Nature | Year | Reason | | —————————————————————————————– | ———- | ——– | ——— | ——— | —- | ———————————————————- | | Dexter+Object | 4 | 3,014 | N/A | Real | 2016 | Only finger tip annotations. | | HUST Dataset | 30 | N/A | N/A | Real | 2016 | Cyberglove used, no object annotations, only joint angles. | | TUB Dataset | 17 | N/A | 25 | Real | 2016 | Cyberglove used, no object annotations, only joint angles. | | EgoDexter | 4 | 1,485 | N/A | Real | 2017 | Only finger tip annotations. | | Rendered Hand Pose (RHD) | 20 | 44,700 | N/A | Synthetic | 2017 | No object annotations. | | UNIPI Dataset | 6 | N/A | 21 | Real | 2018 | No object annotations, only joint angles. | | Partially Occluded Hands | N/A | 11,840 | 148 | Real | 2018 | No object annotations. | | FreiHAND | N/A | 130,3960 | N/A | Real | 2019 | No object annotations. | | ContactDB | 50 | 375K | 50 | Real | 2019 | No joint annotations. |
