Learning human activities and object affordances from RGB-D videos
Abstract
Understanding human activities and object affordances are two very important skills, especially for personal robots which operate in human environments. In this work, we consider the problem of extracting a descriptive labeling of the sequence of sub-activities being performed by a human, and more importantly, of their interactions with the objects in the form of associated affordances. Given a RGB-D video, we jointly model the human activities and object affordances as a Markov random field where the nodes represent objects and sub-activities, and the edges represent the relationships between object affordances, their relations with sub-activities, and their evolution over time. We formulate the learning problem using a structural support vector machine (SSVM) approach, where labelings over various alternate temporal segmentations are considered as latent variables. We tested our method on a challenging dataset comprising 120 activity videos collected from 4 subjects, and obtained an accuracy of 79.4% for affordance, 63.4% for sub-activity and 75.0% for high-level activity labeling. We then demonstrate the use of such descriptive labeling in performing assistive tasks by a PR2 robot.
References
Aggarwal JK, Ryoo MS (2011) Human activity analysis: A review. ACM Computer Surveys 43(3): 16.
Aksoy E, Abramov A, Worgotter F, Dellen B (2010) Categorizing object–action relations from semantic scene graphs. In: Proceedings of ICRA.
Aksoy EE, Abramov A, Dörr J, Ning K, Dellen B, Wörgötter F (2011) Learning the semantics of object-action relations by observation. The International Journal of Robotics Research 30: 1229–1249.
Aldoma A, Tombari F, Vincze M (2012) Supervised learning of hidden and non-hidden 0-order affordances and detection in real scenes. In: Proceedings of ICRA.
Anand A, Koppula HS, Joachims T, Saxena A (2012) Contextually guided semantic labeling and search for 3D point clouds. The International Journal of Robotics Research, in press.
Bollini M, Tellex S, Thompson T, Roy N, Rus D (2012) Interpreting and executing recipes with a cooking robot. In: Proceedings of ISER.
Choi C, Christensen HI (2012) Robust 3D visual tracking using particle filtering on the special Euclidean group: A combined approach of keypoint and edge features. The International Journal of Robotics Research 31: 498–519.
Collet A, Martinez M, Srinivasa SS (2011) The MOPED framework: object recognition and pose estimation for manipulation. The International Journal of Robotics Research 30: 1284–1306.
Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In: Proceedings of CVPR.
Diankov R (2010) Automated Construction of Robotic Manipulation Programs. PhD thesis, Carnegie Mellon University, Robotics Institute. http://www.programmingvision.com/rosen_diankov_thesis.pdf.
Felzenszwalb PF, Huttenlocher D (2004) Efficient graph-based image segmentation. International Journal of Computer Vision 59: 167–181.
Finley T, Joachims T (2008) Training structural svms when exact inference is intractable. In: Proceedings of ICML.
Fox E, Sudderth E, Jordan M, Willsky A (2011) Bayesian nonparametric inference of switching dynamic linear models. IEEE Transactions on Signal Processing 59: 1569–1585.
Gall J, Fossati A, van Gool L (2011) Functional categorization of objects using real-time markerless motion capture. In: Proceedings of CVPR.
Gibson J (1979) The Ecological Approach to Visual Perception. Houghton Mifflin.
Gupta A, Kembhavi A, Davis L (2009) Observing human–object interactions: Using spatial and functional compatibility for recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 31: 1775–1789.
Hammer P, Hansen P, Simeone B (1984) Roof duality, complementation and persistency in quadratic 0–1 optimization. Mathematical Programming 28: 121–155.
Henry P, Krainin M, Herbst E, Ren X, Fox D (2012) RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments. The International Journal of Robotics Research 31: 647–663.
Hermans T, Rehg JM, Bobick A (2011) Affordance prediction via learned object attributes. In: ICRA: Workshop on Semantic Perception, Mapping, and Exploration.
Hoai M, De la, Torre F (2012) Maximum margin temporal clustering. In: Proceedings of International Conference on Artificial Intelligence and Statistics.
Hoai M, Lan Z, De la, Torre F (2011) Joint segmentation and classification of human actions in video. In: Proceedings of CVPR.
Jiang Y, Li Z, Chang S (2011a) Modeling scene and object contexts for human action retrieval with few examples. IEEE Transactions on Circuits and Systems for Video Technology 21: 674–681.
Jiang Y, Lim M, Saxena A (2012a) Learning object arrangements in 3D scenes using human context. In: Proceedings of ICML.
Jiang Y, Lim M, Zheng C, Saxena A (2012b) Learning to place new objects in a scene. The International Journal of Robotics Research 31: 1021–1043.
Jiang Y, Moseson S, Saxena A (2011b) Efficient grasping from rgbd images: Learning using a new rectangle representation. In: Proceedings of ICRA.
Jiang Y, Saxena A (2012) Hallucinating humans for learning robotic placement of objects. In: Proceedings of ISER.
Joachims T, Finley T, Yu C (2009) Cutting-plane training of structural SVMs. Machine Learning 77: 27–59.
Kjellström H, Romero J, Kragic D (2011) Visual object-action recognition: Inferring object affordances from human demonstration. Computer Vision and Image Understanding 115: 81–90.
Koller D, Friedman N (2009) Probabilistic Graphical Models: Principles and Techniques. Cambridge, MA: MIT Press.
Konidaris G, Kuindersma S, Grupen R, Barto A (2012) Robot learning from demonstration by constructing skill trees. The International Journal of Robotics Research 31: 360–375.
Koppula H, Anand A, Joachims T, Saxena A (2011) Semantic labeling of 3D point clouds for indoor scenes. In: Proceedings of NIPS.
Koppula HS, Gupta R, Saxena A (2012) Human activity learning using object affordances from RGB-D videos. CoRR abs/1208.0967.
Kormushev P, Calinon S, Caldwell DG (2010) Robot motor skill coordination with EM-based reinforcement learning. In: Proceedings of IROS.
Krainin M, Henry P, Ren X, Fox D (2011) Manipulator and object tracking for in-hand 3D object modeling. The International Journal of Robotics Research 30: 1311–1327.
Lai K, Bo L, Ren X, Fox D (2011a) A large-scale hierarchical multi-view RGB-D object dataset. In: Proceedings of ICRA.
Lai K, Bo L, Ren X, Fox D (2011b) Sparse distance learning for object recognition combining RGB and depth information. In: Proceedings of ICRA.
Laptev I, Marszalek M, Schmid C, Rozenfeld B (2008) Learning realistic human actions from movies. In: Proceedings of CVPR.
Li C, Kowdle A, Saxena A, Chen T (2012) Towards holistic scene understanding: Feedback enabled cascaded classification models. IEEE Transactions on Pattern Analysis and Machine Intelligence 34: 1394–1408.
Li W, Zhang Z, Liu Z (2010) Action recognition based on a bag of 3D points. In: Workshop on CVPR for Human Communicative Behavior Analysis.
Liu J, Luo J, Shah M (2009) Recognizing realistic actions from videos “in the wild”. In: Proceedings of CVPR.
Lopes M, Santos-Victor J (2005) Visual learning by imitation with motor representations. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics 35: 438–449.
Matikainen P, Sukthankar R, Hebert M (2012) Model recommendation for action recognition. In: Proceedings of CVPR.
Miller S, van den Berg J, Fritz M, Darrell T, Goldberg K, Abbeel P (2011) A geometric approach to robotic laundry folding. The International Journal of Robotics Research 31: 249–267.
Moldovan B, van Otterlo M, Moreno P, Santos-Victor J, De Raedt L (2012) Statistical relational learning of object affordances for robotic manipulation. In: Latest Advances in Inductive Logic Programming,.
Montesano L, Lopes M, Bernardino A, Santos-Victor J (2008) Learning object affordances: From sensory–motor coordination to imitation. IEEE Transactions on Robotics 24: 15–26.
Ni B, Wang G, Moulin P (2011) Rgbd-hudaact: A color-depth video database for human daily activity recognition. In: ICCV Workshop on Consumer Depth Cameras for Computer Vision.
Panangadan A, Mataric MJ, Sukhatme GS (2010) Tracking and modeling of human activity using laser rangefinders. International Journal of Social Robotics 2: 95–107.
Pandey A, Alami R (2012) Taskability graph: Towards analyzing effort based agent-agent affordances. In: IEEE RO-MAN, pp. 791–796.
Pandey AK, Alami R (2010) Mightability maps: A perceptual level decisional framework for co-operative and competitive human–robot interaction. In: Proceedings of IROS.
Pele O, Werman M (2008) A linear time histogram metric for improved SIFT matching. In: Proceedings of ECCV.
Pirsiavash H, Ramanan D (2012) Detecting activities of daily living in first-person camera views. In: Proceedings of CVPR.
Ridge B, Skočaj D, Leonardis A (2009) Unsupervised learning of basic object affordances from object properties. In: Proceedings of the Fourteenth Computer Vision Winter Workshop (CVWW).
Rohrbach M, Amin S, Andriluka M, Schiele B (2012) A database for fine grained activity detection of cooking activities. In: Proceedings of CVPR.
Rosman B, Ramamoorthy S (2011) Learning spatial relationships between objects. The International Journal of Robotics Research 30: 1328–1342.
Rother C, Kolmogorov V, Lempitsky V, Szummer M (2007) Optimizing binary MRFs via extended roof duality. In: Proceedings of CVPR.
Rusu R, Bradski G, Thibaux R, Hsu J (2010) Fast 3D recognition and pose using the viewpoint feature histogram. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
Rusu RB, Blodow N, Marton ZC, Beetz M (2009) Close-range scene segmentation and reconstruction of 3D point cloud maps for mobile manipulation in domestic environments. In: Proceedings of IROS.
Sadanand S, Corso J (2012) Action bank: A high-level representation of activity in video. In: Proceedings of CVPR.
Saxena A, Driemeyer J, Ng A (2008) Robotic grasping of novel objects using vision. The International Journal of Robotics Research 27: 157.
Saxena A, Sun M, Ng AY (2009) Make3d: Learning 3d scene structure from a single still image. IEEE Transactions on Pattern Analysis and Machine Intelligence 31: 824–840.
Shi Q, Wang L, Cheng L, Smola A (2011) Human action segmentation and recognition using discriminative semi-Markov models. International Journal of Computer Vision 93: 22–32.
Shotton J, Fitzgibbon A, Cook M, et al. (2011) Real-time human pose recognition in parts from single depth images. In: Proceedings of CVPR.
Sun J, Moore JL, Bobick A, Rehg JM (2009) Learning visual object categories for robot affordance prediction. The International Journal of Robotics Research 29: 174–197.
Sung J, Ponce C, Selman B, Saxena A (2012) Unstructured human activity detection from RGBD images. In: Proceedings of ICRA.
Sung JY, Ponce C, Selman B, Saxena A (2011) Human activity detection from rgbd images. In: AAAI workshop on Pattern, Activity and Intent Recognition (PAIR).
Tang K, Fei-Fei L, Koller D (2012) Learning latent temporal structure for complex event detection. In: Proceedings of CVPR.
Taskar B, Chatalbashev V, Koller D (2004) Learning associative Markov networks. In: Proceedings of ICML.
Tsochantaridis I, Hofmann T, Joachims T, Altun Y (2004) Support vector machine learning for interdependent and structured output spaces. In: Proceedings of ICML.
Yang W, Wang Y, Mori G (2010) Recognizing human actions from still images with latent poses. In: Proceedings of CVPR.
Yao B, Fei-Fei L (2010) Modeling mutual context of object and human pose in human–object interaction activities. In: Proceedings of CVPR.
Yao B, Jiang X, Khosla A, Lin A, Guibas L, Fei-Fei L (2011) Action recognition by learning bases of action attributes and parts. In: Proceedings of ICCV.
Yu C, Joachims T (2009) Learning structural svms with latent variables. In: Proceedings of ICML.
Zhang H, Parker LE (2011) 4-dimensional local spatio-temporal features for human activity recognition. In: Proceedings of IROS.
Cite article
Cite article
Cite article
OR
Download to reference manager
If you have citation software installed, you can download article citation data to the citation manager of your choice
Information, rights and permissions
Information
Published In
Article first published online: July 11, 2013
Issue published: July 2013
Keywords
Rights and permissions
© The Author(s) 2013.
This article is distributed under the terms of the Creative Commons Attribution-Non Commercial 3.0 License (http://www.creativecommons.org/licenses/by-nc/3.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access page(http://www.uk.sagepub.com/aboutus/openaccess.htm).
Request permissions for this article.
Authors
Metrics and citations
Metrics
Journals metrics
This article was published in The International Journal of Robotics Research.
VIEW ALL JOURNAL METRICSArticle usage*
Total views and downloads: 4239
*Article usage tracking started in December 2016
Altmetric
See the impact this article is making through the number of times it’s been read, and the Altmetric Score.
Learn more about the Altmetric Scores
Articles citing this one
Receive email alerts when this article is cited
Web of Science: 433 view articles Opens in new tab
Crossref: 474
-
Grounded Affordance from Exocentric View
-
Interactor detection in multiview videos of parent-child play
-
OASNet: Object Affordance State Recognition Network With Joint Visual ...
-
AI Trainer : Video-Based Squat Analysis
-
ADOSMNet: a novel visual affordance detection network with object shap...
-
IKEA Ego 3D Dataset: Understanding furniture assembly actions from ego...
-
Multi-Label Action Anticipation for Real-World Videos With Scene Under...
-
Multimodal action recognition: a comprehensive survey on temporal mode...
-
Depth Image Restoration for High-Level Vision Tasks
-
Task parse tree: Learning task policy from videos with task-irrelevant...
-
Human activity recognition in an end-of-life consumer electronics disa...
-
A Method to Detect Human Hands Moving Objects
-
Exploring Spatio–Temporal Graph Convolution for Video-Based Human–Obje...
-
MECCANO: A multimodal egocentric dataset for humans behavior understan...
-
The multi-angle extended three-dimensional activities (META) stimulus ...
-
Open Set Video HOI detection from Action-centric Chain-of-Look Prompti...
-
Grounding 3D Object Affordance from 2D Interactions in Images
-
Learning Robot Manipulation Skills From Human Demonstration Videos Usi...
-
RPLNet: Object-Object Affordance Recognition via Relational Phrase Lea...
-
GFIE: A Dataset and Baseline for Gaze-Following from 2D to 3D in Indoo...
-
LOCATE: Localize and Transfer Object Parts for Weakly Supervised Affor...
-
Putting People in Their Place: Affordance-Aware Human Insertion into S...
-
Few-shot human–object interaction video recognition with transformers
-
PACE: Data-Driven Virtual Agent Interaction in Dense and Cluttered Env...
-
CholecTriplet2021: A benchmark challenge for surgical action triplet r...
-
A Multi-modal Framework for Robots to Learn Manipulation Tasks from Hu...
-
Toward human activity recognition: a survey
-
Toyota Smarthome Untrimmed: Real-World Untrimmed Videos for Activity D...
-
A Step Towards Automated Functional Assessment of Activities of Daily ...
-
SSRT: A Sequential Skeleton RGB Transformer to Recognize Fine-Grained ...
-
Learning Scene-Aware Spatio-Temporal GNNs for Few-Shot Early Action Pr...
-
A Comprehensive Survey of RGB-Based and Skeleton-Based Human Action Re...
-
Applications of Deep Learning-Based Methods on Surveillance Video Stre...
-
Partial Alignment of Time Series for Action and Activity Prediction
-
Fine-grained Affordance Annotation for Egocentric Hand-Object Interact...
-
Placing Human Animations into 3D Scenes by Learning Interaction- and G...
-
Skew-Robust Human-Object Interactions in Videos
-
Contact Part Detection From 3D Human Motion Data Using Manually Labele...
-
The VISTA datasets, a combination of inertial sensors and depth camera...
-
A Survey of Human Action Recognition and Posture Prediction
-
Language-guided graph parsing attention network for human-object inter...
-
Online suspicious event detection in a constrained environment with RG...
-
Distillation of human–object interaction contexts for action recogniti...
-
Recognition of Human-object Interaction in Video through a Two-stream ...
-
STIT: Spatio-Temporal Interaction Transformers for Human-Object Intera...
-
Human activity recognition in artificial intelligence framework: a nar...
-
Spatial Parsing and Dynamic Temporal Pooling networks for Human-Object...
-
Deep learning and RGB-D based human action, human–human and human–obje...
-
A Versatile Affordance Modeling Framework Using Screw Primitives to In...
-
One-shot Video Graph Generation for Explainable Action Reasoning
-
Online human action detection and anticipation in videos: A survey
-
Understanding 3D Object Articulation in Internet Videos
-
Learning Affordance Grounding from Exocentric Images
-
Complex Video Action Reasoning via Learnable Markov Logic Network
-
Human Action Recognition and Prediction: A Survey
-
ChaLearn Looking at People: IsoGD and ConGD Large-Scale RGB-D Gesture ...
-
3D Human Action Recognition: Through the eyes of researchers
-
Visual Affordance and Function Understanding
-
Spatio-temporal analysis and comparison of 3D videos
-
ANew VR Kitchen Environment for Recording Well Annotated Object Intera...
-
DTR-HAR: deep temporal residual representation for human activity reco...
-
A semi-supervised deep learning based video anomaly detection framewor...
-
Skeleton Graph-Neural-Network-Based Human Action Recognition: A Survey
-
A Vision-Based Measure of Environmental Effects on Inferring Human Int...
-
Improved human-object interaction detection through skeleton-object re...
-
Egocentric Activity Recognition and Localization on a 3D Map
-
Human-Object Interaction Detection: A Survey of Deep Learning-Based Me...
-
Graphing the Future: Activity and Next Active Object Prediction Using ...
-
Human Intent Prediction for Human-Robot Collaboration
-
Learning for action-based scene understanding
-
Task Recognition in Human-Robot Collaboration for Consumer Electronics...
-
Angular Features-Based Human Action Recognition System for a Real Appl...
-
We Know Where They Are Looking at From the RGB-D Camera: Gaze Followin...
-
Human Action Recognition From Various Data Modalities: A Review
-
Geometric Features Informed Multi-person Human-Object Interaction Reco...
-
Hallucinating Pose-Compatible Scenes
-
NTU-DensePose: A New Benchmark for Dense Pose Action Recognition
-
Switching Structured Prediction for Simple and Complex Human Activity ...
-
Action Recognition with Fusion of Multiple Graph Convolutional Network...
-
Modeling Object’s Affordances via Reward Functions
-
Spatio-Temporal Interaction Graph Parsing Networks for Human-Object In...
-
RGB+D and deep learning-based real-time detection of suspicious event ...
-
Detecting Human-Object Relationships in Videos
-
Seeing the Unseen: Predicting the First-Person Camera Wearer’s Locatio...
-
Inferring object properties from human interaction and transferring th...
-
ST-HOI: A Spatial-Temporal Baseline for Human-Object Interaction Detec...
-
HOIsim: Synthesizing Realistic 3D Human-Object Interaction Data for Hu...
-
A Generalized Earley Parser for Human Activity Parsing and Prediction
-
The KIT Bimanual Manipulation Dataset
-
Scene-Perception Graph Convolutional Networks for Human Action Predict...
-
Deep learning approaches for human-centered IoT applications in smart ...
-
M&M: Recognizing Multiple Co-evolving Activities From Multi-source Vid...
-
Exploring the usefulness of Object Affordances through a Knowledge bas...
-
Robotics Dexterous Grasping: The Methods Based on Point Cloud and Deep...
-
Skeleton-based structured early activity prediction
-
Variable structure Human Intention Estimator with mobility and vision ...
-
3D AffordanceNet: A Benchmark for Visual Object Affordance Understandi...
-
Synthesizing Long-Term 3D Human Motion and Interaction in 3D Scenes
-
Learning Asynchronous and Sparse Human-Object Interaction in Videos
-
Two-stream 2D/3D Residual Networks for Learning Robot Manipulations fr...
-
Spatial–Temporal Relation Reasoning for Action Prediction in Videos
-
Action recognition using kinematics posture feature on 3D skeleton joi...
-
Activity recognition in video sequences over qualitative abstracts of ...
-
Endowing Robots with Longer-term Autonomy by Recovering from External ...
-
Spatio-temporal multi-factor model for individual identification from ...
-
Deep Full-Body HPE for Activity Recognition from RGB Frames Only
-
Inferring Tasks and Fluents in Videos by Learning Causal Relations
-
Late Fusion of Bayesian and Convolutional Models for Action Recognitio...
-
IPT: A Dataset for Identity Preserved Tracking in Closed Domains
-
Video Activity Recognition Based on Objects Detection Using Recurrent ...
-
Visual Methods for Sign Language Recognition: A Modality-Based Review
-
Action Prediction During Human-Object Interaction Based on DTW and Ear...
-
Inferring Semantic Object Affordances from Videos
-
An Robot Vision Grasping Network Based on Inception-Lite
-
Joint Object Affordance Reasoning and Segmentation in RGB-D Videos
-
Predicting Task-Driven Attention via Integrating Bottom-Up Stimulus an...
-
Exploration of Spatial and Temporal Modeling Alternatives for HOI
-
Evaluation and Optimization of Dual-Arm Robot Path Planning for Human–...
-
Video Action Understanding
-
OSD: An Occlusion Skeleton Dataset for Action Recognition
-
Automatic Employability Test for Factory Workers using Collaborative F...
-
Sensor-based and vision-based human activity recognition: A comprehens...
-
Predicting human navigation goals based on Bayesian inference and acti...
-
Human-in-the-Loop Robot Control for Human-Robot Collaboration: Human I...
-
Sustainable Wearable System: Human Behavior Modeling for Life-Logging ...
-
A knowledge-driven layered inverse reinforcement learning approach for...
-
EV-Action: Electromyography-Vision Multi-Modal Action Dataset
-
Rate-Invariant Modeling in Lie Algebra for Activity Recognition
-
ETRI-Activity3D: A Large-Scale RGB-D Dataset for Robots to Recognize D...
-
LIGHTEN
-
FPHA-Afford: A Domain-Specific Benchmark Dataset for Occluded Object A...
-
NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understan...
-
Estimation of Sweet Pepper Crop Fresh Weight with Convolutional Neural...
-
Object affordance detection with relationship-aware network
-
Thermal comfort measurement using thermal-depth images for robotic mon...
-
Prediction of Human Activities Based on a New Structure of Skeleton Fe...
-
Anticipating Human Intention for Full-Body Motion Prediction in Object...
-
Symbolic Learning and Reasoning With Noisy Data for Probabilistic Anch...
-
Hierarchical and parameterized learning of pick-and-place manipulation...
-
Learning to infer human attention in daily activities
-
Fine-grained action plausibility rating
-
An Ensemble of Knowledge Sharing Models for Dynamic Hand Gesture Recog...
-
A Survey of Human Action Analysis in HRI Applications
-
Generating 3D People in Scenes Without People
-
Context-Aware Human Motion Prediction
-
Beyond the Self: Using Grounded Affordances to Interpret and Describe ...
-
Roombots extended: Challenges in the next generation of self-reconfigu...
-
Simultaneous Learning from Human Pose and Object Cues for Real-Time Ac...
-
KETO: Learning Keypoint Representations for Tool Manipulation
-
Fine-Grained Action Recognition by Motion Saliency and Mid-Level Patch...
-
Learning task-oriented grasping for tool manipulation from simulated s...
-
Vision-based human action recognition: An overview and real world chal...
-
A multimodal approach for human activity recognition based on skeleton...
-
Semantic Relational Object Tracking
-
Stacked Spatio-Temporal Graph Convolutional Networks for Action Segmen...
-
Double-layer conditional random fields model for human action recognit...
-
Action Recognition Based on the Fusion of Graph Convolutional Networks...
-
Text Like Classification of Skeletal Sequences for Human Action Recogn...
-
Human Activity Understanding
-
Social Activity Recognition on Continuous RGB-D Video Sequences
-
Dynamic graph convolutional networks
-
Prognosing Human Activity Using Actions Forecast and Structured Databa...
-
Learning Dynamic Spatio-Temporal Relations for Human Activity Recognit...
-
Learning Object-Action Relations from Bimanual Human Demonstration Usi...
-
A Comparative Review of Recent Kinect-Based Action Recognition Algorit...
-
Human Intention Estimation using Fusion of Pupil and Hand Motion
-
Representation Learning on Visual-Symbolic Graphs for Video Understand...
-
LEMMA: A Multi-view Dataset for L Earning Multi-agent Multi-task Activ...
-
Learning attentive dynamic maps (ADMs) for Understanding Human Actions
-
Motion Segmentation via Generalized Curvatures
-
Learning and Comfort in Human–Robot Interaction: A Review
-
A weighting scheme for mining key skeletal joints for human action rec...
-
Simitate: A Hybrid Imitation Learning Benchmark
-
Improving Robot Success Detection using Static Object Data
-
Towards Action Prediction Applying Deep Learning
-
Early Action Prediction by Soft Regression
-
Explainable Video Action Reasoning via Prior Knowledge and State Trans...
-
Resolving 3D Human Pose Ambiguities With 3D Scene Constraints
-
YogaNet: 3-D Yoga Asana Recognition Using Joint Angular Displacement M...
-
A Survey on Predicting Resident Intentions Using Contextual Modalities...
-
An Ontology for Human-Human Interactions and Learning Interaction Beha...
-
A New Bayesian Modeling for 3D Human-Object Action Recognition
-
Human Motion Prediction Based on Object Interactions
-
Help by Predicting What to Do
-
Compositional Learning of Human Activities With a Self-Organizing Neur...
-
Affordance-based altruistic robotic architecture for human–robot colla...
-
A Survey of Knowledge Representation in Service Robotics
-
Two-Stream Network with 3D Common-Specific Framework for RGB-D Action ...
-
A benchmark image dataset for industrial tools
-
Speeding Up Affordance Learning for Tool Use, Using Proprioceptive and...
-
Activities Prediction Using Structured Data Base
-
Benchmarking Search and Annotation in Continuous Human Skeleton Sequen...
-
H+O: Unified Egocentric Recognition of 3D Hand-Object Poses and Intera...
-
Putting Humans in a Scene: Learning Affordance in 3D Indoor Environmen...
-
Predicting the What and How - a Probabilistic Semi-Supervised Approach...
-
Who Takes What: Using RGB-D Camera and Inertial Sensor for Unmanned Mo...
-
Complex Human–Object Interactions Analyzer Using a DCNN and SVM Hybrid...
-
Learning a Generative Model for Multi‐Step Human‐Object Interactions f...
-
Discriminative bit selection hashing in RGB-D based object recognition...
-
Skeleton-Based Human Action Recognition by Pose Specificity and Weight...
-
Facilitating Human–Robot Collaborative Tasks by Teaching-Learning-Coll...
-
Effective Behavioural Dynamic Coupling through Echo State Networks
-
Predicting Human Actions Taking into Account Object Affordances
-
Fusing depth and colour information for human action recognition
-
Accurate detection of sitting posture activities in a secure IoT based...
-
COSMO: Contextualized scene modeling with Boltzmann Machines
-
RGB-D Action Recognition Using Multimodal Correlative Representation L...
-
A Comprehensive Survey of Vision-Based Human Action Recognition Method...
-
Collaborative multimodal feature learning for RGB-D action recognition
-
Perceiving the person and their interactions with the others for socia...
-
Multi-Scale Guided Mask Refinement for Coarse-to-Fine RGB-D Perception
-
I-Planner: Intention-aware motion planning using learning-based human ...
-
Predicting Action Tubes
-
RGB-D Sensors Data Quality Assessment and Improvement for Advanced App...
-
Assistive Humanoid Robots for the Elderly with Mild Cognitive Impairme...
-
Extended histogram: probabilistic modelling of video content temporal ...
-
Skeleton-Based Action Recognition With Key-Segment Descriptor and Temp...
-
Recognition of Assembly Tasks Based on the Actions Associated to the M...
-
Toward Computer Vision Systems That Understand Real-World Assembly Pro...
-
Advances in description of 3D human motion
-
Learning Complex Spatio-Temporal Configurations of Body Joints for Onl...
-
Joint Deep Learning for RGB-D Action Recognition
-
Three-dimensional convolutional restricted Boltzmann machine for human...
-
Human activity learning for assistive robotics using a classifier ense...
-
Deep-Temporal LSTM for Daily Living Action Recognition
-
Learning Under-Specified Object Manipulations from Human Demonstration...
-
Human activity recognition using dynamic representation and matching o...
-
Operator Awareness in Human–Robot Collaboration Through Wearable Vibro...
-
A self-organizing neural network architecture for learning human-objec...
-
Object Affordance Driven Inverse Reinforcement Learning Through Concep...
-
场景知觉过程中的动作意图识别
-
Structured RNN for human interaction
-
Skeleton-based bio-inspired human activity prediction for real-time hu...
-
Video scene analysis: an overview and challenges on deep learning algo...
-
Action recognition in depth videos using hierarchical gaussian descrip...
-
Simultaneous joint and object trajectory templates for human activity ...
-
CNN+RNN Depth and Skeleton based Dynamic Hand Gesture Recognition
-
Data-Driven, 3-D Classification of Person-Object Relationships and Sem...
-
Robust 3D Action Recognition Through Sampling Local Appearances and Gl...
-
Learning Stable Movement Primitives by Finding a Suitable Fuzzy Lyapun...
-
Action-Attending Graphic Neural Network
-
Gaze and motion information fusion for human intention inference
-
Robot action planning by online optimization in human–robot collaborat...
-
Weakly and Semi Supervised Human Body Part Parsing via Pose-Guided Kno...
-
Demo2Vec: Reasoning Object Affordances from Online Videos
-
From Lifestyle Vlogs to Everyday Interactions
-
Human Action Recognition from Motion Trajectory using Fourier Temporal...
-
Human-Robot Interaction Control Through Demonstration
-
Exploiting ability for human adaptation to facilitate improved human-r...
-
Functional Object-Oriented Network: Construction & Expansion
-
Fusing Object Context to Detect Functional Area for Cognitive Robots
-
Object-Centric Approach to Prediction and Labeling of Manipulation Tas...
-
Teaching a Robot the Semantics of Assembly Tasks
-
Structure-Aware Multimodal Feature Fusion for RGB-D Scene Classificati...
-
Representation, Analysis, and Recognition of 3D Humans
-
Human Weapon-Activity Recognition in Surveillance Videos Using Structu...
-
Unsupervised early prediction of human reaching for human–robot collab...
-
Low-Cost Automatic Ambient Assisted Living System
-
Affordances in Psychology, Neuroscience, and Robotics: A Survey
-
Bootstrapping Relational Affordances of Object Pairs Using Transfer
-
End-to-End Fine-Grained Action Segmentation and Recognition Using Cond...
-
Instance-Aware Detailed Action Labeling in Videos
-
Structural Recurrent Neural Network (SRNN) for Group Activity Analysis
-
An Unsupervised Machine Learning Approach to Assessing Designer Perfor...
-
Distances evolution analysis for online and off-line human object inte...
-
Watch-n-Patch: Unsupervised Learning of Actions and Relations
-
Two‐person activity recognition using skeleton data
-
Deep Bilinear Learning for RGB-D Action Recognition
-
Learning Human-Object Interactions by Graph Parsing Neural Networks
-
Neural Graph Matching Networks for Fewshot 3D Action Recognition
-
Graph Distillation for Action Detection with Privileged Modalities
-
Human Activities Transfer Learning for Assistive Robotics
-
A Large Scale RGB-D Dataset for Action Recognition
-
Bidirectional invariant representation of rigid body motions and its a...
-
Integrating multi-purpose natural language understanding, robot’s memo...
-
Multimodal Trip Hazard Affordance Detection on Construction Sites
-
Human Motion Segmentation via Robust Kernel Sparse Subspace Clustering
-
Motion Trajectory for Human Action Recognition Using Fourier Temporal ...
-
Robust echo state networks based on correntropy induced loss function
-
Affordance learning and inference based on vision-speech association i...
-
Affordance triggering for arbitrary states based on robot exploring
-
Emergent Structuring of Interdependent Affordance Learning Tasks Using...
-
A human activity recognition framework using max-min features and key ...
-
A dynamic Markov model for n th -order mov...
-
Jointly Learning Heterogeneous Features for RGB-D Activity Recognition
-
Computational models of affordance in robotics: a taxonomy and systema...
-
Scene Semantic Reconstruction from Egocentric RGB-D-Thermal Videos
-
Predicting Human Activities Using Stochastic Grammar
-
Jointly Recognizing Object Fluents and Tasks in Egocentric Videos
-
Probabilistic Structure from Motion with Objects (PSfMO)
-
Adaptive Binarization for Weakly Supervised Affordance Segmentation
-
Use of Thermal Point Cloud for Thermal Comfort Measurement and Human P...
-
Learning to Recognize Human Activities Using Soft Labels
-
Human‐action recognition using a multi‐layered fusion scheme of Kinect...
-
Sentence Directed Video Object Codiscovery
-
An affordances based approach to assisted teleoperation
-
Structured prediction with short/long-range dependencies for human act...
-
Two-stream RNN/CNN for action recognition in 3D videos
-
Cooking Behavior Recognition Using Egocentric Vision for Cooking Navig...
-
Structured LSTM for human-object interaction detection and anticipatio...
-
Action recognition based on a mixture of RGB and depth based skeleton
-
Human Action Recognition with RGB-D Sensors
-
ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes
-
SCC: Semantic Context Cascade for Efficient Action Detection
-
Learning and Refining of Privileged Information-Based RNNs for Action ...
-
Weakly Supervised Affordance Detection
-
The Impact of Typicality for Informative Representative Selection
-
Non-uniform Subset Selection for Active Learning in Structured Data
-
Key-frame Extraction With Semantic Graphs in Assembly Processes
-
Action prediction based on physically grounded object affordances in h...
-
Learning Human Activities for Assisted Living Robotics
-
Detecting Biological Motion for Human–Robot Interaction: A Link betwee...
-
Human activity recognition using segmented body part and body joint fe...
-
Modeling 4D Human-Object Interactions for Joint Event Segmentation, Re...
-
The toronto rehab stroke pose dataset to detect compensation during st...
-
Space-time representation of people based on 3D skeletal data: A revie...
-
Generation of action description from classification of motion and obj...
-
EGGNOG: A Continuous, Multi-modal Data Set of Naturally Occurring Gest...
-
The DAily Home LIfe Activity Dataset: A High Semantic Activity Dataset...
-
What can i do around here? Deep functional scene understanding for cog...
-
Semantic analysis of manipulation actions using spatial relations
-
An overview of action recognition in videos
-
Healthy human sitting posture estimation in RGB-D scenes using object ...
-
Human Intention Inference Using Expectation-Maximization Algorithm Wit...
-
Automatic Learning of Articulated Skeletons Based on Mean of 3D Joints...
-
Predicting human activities in sequences of actions in RGB-D videos
-
Semantic Decomposition and Recognition of Long and Complex Manipulatio...
-
A novel local feature descriptor based on energy information for human...
-
A Multilevel Body Motion-Based Human Activity Analysis Methodology
-
Context-Associative Hierarchical Memory Model for Human Activity Recog...
-
RGB-D datasets using microsoft kinect or similar sensors: a survey
-
Learning Actions to Improve the Perceptual Anchoring of Objects
-
O-PrO: An Ontology for Object Affordance Reasoning
-
Assistive Humanoid Robots for the Elderly with Mild Cognitive Impairme...
-
Physics Simulation Games
-
Intention Inference for Human-Robot Collaboration in Assistive Robotic...
-
Visual Information-Based Activity Recognition and Fall Detection for A...
-
Motion segment decomposition of RGB-D sequences for human behavior und...
-
Facial Expression Recognition Utilizing Local Direction-Based Robust F...
-
Multimodal Gesture Recognition Using 3-D Convolution and Convolutional...
-
Feature-Based Resource Allocation for Real-Time Stereo Disparity Estim...
-
Tracking a Subset of Skeleton Joints: An Effective Approach towards Co...
-
Human gesture recognition performance evaluation for service robots
-
Hybrid Multi-modal Fusion for Human Action Recognition
-
RGB-D-based action recognition datasets: A survey
-
Recent Data Sets on Object Manipulation: A Survey
-
A review of RGB-D sensor calibration for night vision
-
A probabilistic graphical model approach for human activity recognitio...
-
Training Agents With Interactive Reinforcement Learning and Contextual...
-
Affordance Research in Developmental Robotics: A Survey
-
Toward Simple Strategy for Optimal Tracking and Localization of Robots...
-
Biological movement detector enhances the attentive skills of humanoid...
-
Learning human-robot handovers through π-STAM: Policy improvement with...
-
Human pose recognition and tracking using RGB-D camera
-
Affordance processing in segregated parieto-frontal dorsal stream sub-...
-
Human intent forecasting using intrinsic kinematic constraints
-
Human activity recognition based on weighted limb features
-
Modeling 3D Environments through Hidden Human Context
-
Human Activity Recognition from automatically labeled data in RGB-D vi...
-
Optimal depth recovery using image guided TGV with depth confidence fo...
-
Modeling spatial layout of features for real world scenario RGB-D acti...
-
RGB-D based daily activity recognition for service robots using cluste...
-
Simplified industrial robot programming: Effects of errors on multimod...
-
PiGraphs
-
Humanoid infers Archimedes' principle: understanding phy...
-
Action Recognition Based on Efficient Deep Feature Learning in the Spa...
-
A depth video-based facial expression recognition system utilizing gen...
-
A Deep Structured Model with Radius–Margin Bound for 3D Human Activity...
-
Multi-modal RGB–Depth–Thermal Human Body Segmentation
-
NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis
-
3D Semantic Parsing of Large-Scale Indoor Spaces
-
Structural-RNN: Deep Learning on Spatio-Temporal Graphs
-
Learning Action Maps of Large Environments via First-Person Vision
-
RGBD Datasets: Past, Present and Future
-
Understanding Human Behaviors with an Object Functional Role Perspecti...
-
Deep metric learning autoencoder for nonlinear temporal alignment of h...
-
Efficient, dense, object-based segmentation from RGBD video
-
Watch-Bot: Unsupervised learning for reminding humans of forgotten act...
-
Recurrent Neural Networks for driver activity anticipation via sensory...
-
From pose to activity: Surveying datasets and introducing CONVERSE
-
Human action recognition using multi-layer codebooks of key poses and ...
-
Improving face detection with depth
-
Deep recursive and hierarchical conditional random fields for human ac...
-
Tell me Dave: Context-sensitive grounding of natural language to manip...
-
Anticipatory Planning for Human-Robot Teams
-
Introduction
-
Attribute Based Affordance Detection from Human-Object Interaction Ima...
-
Automatic Human Activity Segmentation and Labeling in RGBD Videos
-
Latent Force Models for Human Action Recognition
-
Recognition of Transitive Actions with Hierarchical Neural Network Lea...
-
Real-Time RGB-D Activity Prediction by Soft Regression
-
Segmental Spatiotemporal CNNs for Fine-Grained Action Segmentation
-
A Multi-scale CNN for Affordance Segmentation in RGB Images
-
Predicting the Intention of Human Activities for Real-Time Human-Robot...
-
STAM: A Framework for Spatio-Temporal Affordance Maps
-
A 3D Human Posture Approach for Activity Recognition Based on Depth Ca...
-
A probabilistic approach to workspace sharing for human–robot cooperat...
-
On the Use of Multi-Depth-Camera Based Motion Tracking Systems in Prod...
-
Anticipating Human Activities Using Object Affordances for Reactive Ro...
-
Large Displacement 3D Scene Flow with Occlusion Reasoning
-
Affordance matching from the shared information in multi-robot
-
Human action recognition using key poses and atomic motions
-
Human observation-based calibration of multiple RGB-D cameras for inte...
-
Latent Hierarchical Model for Activity Recognition
-
Multimodal Human Activity Recognition for Industrial Manufacturing Pro...
-
Activity-centric scene synthesis for functional 3D scene modeling
-
Recognizing complex instrumental activities of daily living using scen...
-
Towards efficient support relation extraction from RGBD images
-
Human activity modeling and prediction for assisting appliance operati...
-
Human Activity Recognition Process Using 3-D Posture Data
-
Fuzzy Temporal Segmentation and Probabilistic Recognition of Continuou...
-
Model-free incremental learning of the semantics of manipulation actio...
-
Fine manipulative action recognition through sensor fusion
-
A framework for unsupervised online human reaching motion recognition ...
-
A hierarchical representation for human activity recognition with nois...
-
Semantic parsing of human manipulation activities using on-line learne...
-
Interactive affordance map building for a robotic task
-
Affordance Learning Based on Subtask's Optimal Strategy
-
A Survey of Applications and Human Motion Recognition with Microsoft K...
-
Human Activity-Understanding: A Multilayer Approach Combining Body Mov...
-
Human robot interaction can boost robot's affordance learning: A proof...
-
Cognitive Learning, Monitoring and Assistance of Industrial Workflows ...
-
Self-organizing neural integration of pose-motion features for human a...
-
Discriminative key-component models for interaction detection and reco...
-
SUN RGB-D: A RGB-D scene understanding benchmark suite
-
Mining semantic affordances of visual object categories
-
Jointly learning heterogeneous features for RGB-D activity recognition
-
Staged Development of Robot Skills: Behavior Formation, Affordance Lea...
-
Fusing Multiple Features for Depth-Based Action Recognition
-
PlanIt: A crowdsourcing approach for learning to plan paths from large...
-
Decision making under uncertain segmentations
-
Affordance detection of tool parts from geometric features
-
3D Reasoning from Blocks to Stability
-
Concept and Functional Structure of a Service Robot
-
Characterizing Predicate Arity and Spatial Structure for Inductive Lea...
-
Qualitative and Quantitative Spatio-temporal Relations in Daily Living...
-
Discriminative Dictionary Learning for Skeletal Action Recognition
-
Physics Simulation Games
-
Learning Dictionaries of Sparse Codes of 3D Movements of Body Joints f...
-
A new benchmark for pose estimation with ground truth from virtual rea...
-
Human activity recognition in the context of industrial human-robot in...
-
SceneGrok
-
Human Activity Recognition in Images using SVMs and Geodesics on Smoot...
-
3D Human Activity Recognition with Reconfigurable Convolutional Neural...
-
An Efficient Local Feature-Based Facial Expression Recognition System
-
Mining Mid-Level Features for Action Recognition Based on Effective Sk...
-
Improving reinforcement learning with interactive feedback and afforda...
-
3D content fingerprinting
-
Handling Real-World Context Awareness, Uncertainty and Vagueness in Re...
-
3D human action segmentation and recognition using pose kinetic energy
-
Evaluating spatiotemporal interest point features for depth-based acti...
-
A two-layered approach to recognize high-level human activities
-
Human activities segmentation and location of key frames based on 3D s...
-
Discriminative Hierarchical Modeling of Spatio-temporally Composable H...
-
Learning latent structure for activity recognition
-
Learning Actionlet Ensemble for 3D Human Action Recognition
-
“Important stuff, everywhere!” Activity recognition with...
-
Learning Actionlet Ensemble for 3D Human Action Recognition
-
Physically Grounded Spatio-temporal Object Affordances
-
Pipelining Localized Semantic Features for Fine-Grained Action Recogni...
-
Detecting Social Actions of Fruit Flies
-
Skeleton Tracking Based Complex Human Activity Recognition Using Kinec...
-
Modeling 4D Human-Object Interactions for Event and Object Recognition
-
Infinite Latent Conditional Random Fields
-
Combining color and depth data for edge detection
-
Tangled: Learning to untangle ropes with RGB-D perception
-
Anticipating human activities for reactive robotic response
-
Affordance graph: A framework to encode perspective taking and effort ...
-
[The document that should appear here is unavailable]
-
Multilevel Depth and Image Fusion for Human Activity Detection
-
Representing Videos Using Mid-level Discriminative Patches
-
Hallucinated Humans as the Hidden Context for Labeling 3D Scenes
-
3D-Based Reasoning with Blocks, Support, and Stability
-
Predicting Functional Regions on Objects
Figures and tables
Figures & Media
Tables
View Options
View options
PDF/ePub
View PDF/ePubGet access
Access options
If you have access to journal content via a personal subscription, university, library, employer or society, select from the options below:
loading institutional access options
IOM3 members can access this journal content using society membership credentials.
IOM3 members can access this journal content using society membership credentials.
Alternatively, view purchase options below:
Purchase 24 hour online access to view and download content.
Access journal content via a DeepDyve subscription or find out more about this option.
