Published June 16, 2023
| Version 1.0.0
Dataset
Open
Synthetic Multimodal Dataset for Daily Life Activities
Creators
- 1. Fujitsu
- 2. National Institute of Advanced Industrial Science and Technology
- 3. Osaka Electro-Communication University
- 4. National Agriculture and Food Research Organization
Contributors
Project leader:
Project members:
- 1. National Institute of Advanced Industrial Science and Technology
- 2. Fujitsu
Description
Outline
- This dataset is originally created for the Knowledge Graph Reasoning Challenge for Social Issues (KGRC4SI)
- Video data that simulates daily life actions in a virtual space from Scenario Data.
- Knowledge graphs, and transcriptions of the Video Data content ("who" did what "action" with what "object," when and where, and the resulting "state" or "position" of the object).
- Knowledge Graph Embedding Data are created for reasoning based on machine learning
- This data is open to the public as open data
Details
-
- mp4 format
- 203 action scenarios
- For each scenario, there is a character rear view (file name ending in 0), an indoor camera switching view (file name ending in 1), and a fixed camera view placed in each corner of the room (file name ending in 2-5). Also, for each action scenario, data was generated for a minimum of 1 to a maximum of 7 patterns with different room layouts (scenes). A total of 1,218 videos
- Videos with slowly moving characters simulate the movements of elderly people.
-
- RDF format
- 203 knowledge graphs corresponding to the videos
- Includes schema and location supplement information
- The schema is described below
- SPARQL endpoints and query examples are available
-
- txt format
- Data provided to VirtualHome2KG to generate videos and knowledge graphs
- Includes the action title and a brief description in text format.
- Embedding
- Embedding Vectors in TransE, ComplEx, and RotatE. Created with DGL-KE (https://dglke.dgl.ai/doc/)
- Embedding Vectors created with jRDF2vec (https://github.com/dwslab/jRDF2Vec).
Specification of Ontology
- Please refer to the specification for descriptions of all classes, instances, and properties: https://aistairc.github.io/VirtualHome2KG/vh2kg_ontology.htm
Related Resources
Files
Embeddings.zip
Files
(3.0 GB)
| Name | Size | Download all |
|---|---|---|
|
md5:bdde0d363c9c72c7ca52c3a8a97062a9
|
597.0 MB | Preview Download |
|
md5:29c95c0af82dc99dac5d8142cade926d
|
2.2 GB | Preview Download |
|
md5:75d888631b128c335644c021551d98b8
|
112.2 kB | Preview Download |
|
md5:41cba971297591e0219584ff88bbf912
|
6.9 MB | Preview Download |
|
md5:9db65e76e62efeda07308db9968b6043
|
196.4 MB | Download |
Additional details
References
- Egami, S., Nishimura, S., Fukuda, K.: A Framework for Constructing and Augmenting Knowledge Graphs using Virtual Space: Towards Analysis of Daily Activities. Proceedings of the 33rd IEEE International Conference on Tools with Artificial Intelligence. pp.1226-1230 (2021)
- Egami, S., Nishimura, S., Fukuda, K.: VirtualHome2KG: Constructing and Augmenting Knowledge Graphs of Daily Activities Using Virtual Space. Proceedings of the ISWC 2021 Posters, Demos and Industry Tracks: From Novel Ideas to Industrial Practice, co-located with 20th International Semantic Web Conference. CEUR, Vol.2980 (2021)
- Egami, S., Ugai, T., Oono, M., Kitamura, K., Fukuda.: Synthesizing Event-centric Knowledge Graphs of Daily Activities using Virtual Space. IEEE Access, Vol. 11, pp.23857-23873
- Ugai, T., Egami, S., Swe Nwe Nwe Htun, Kozaki, K, Kawamura, T., Fukuda, K.: Synthetic Multimodal Dataset for Empowering Safety and Well-being in Home Environments. arXiv cs.AI 2401.14743