Bridging Generative Diffusion and Reinforcement Learning for Dexterous Robot Manipulation: Developed a framework for learning contact-rich manipulation by retargeting human-object interactions from the ARCTIC dataset to a humanoid robot using RL environments. To enhance generalizability beyond the dataset, we are integrating a text-conditioned Diffusion Transformer with IK-guidance and Critic-informed sampling, ensuring the generative model accounts for robot kinematics and task-success metrics. Stay tuned!

Building a data engine for web-scale robot training. Stay tuned!

Developed SkillGraph, an ontology-driven framework that unifies semantic knowledge representation with automated task planning to enable flexible multi-robot assembly. By bridging high-level reasoning with low-level control, the system dynamically maps abstract task goals into executable skill primitives, ensuring robust manipulation in complex environments. The SkillGraph comprises of multiple user-interfaces to interact with the system, one of which is a human-video demonstration —where the system is able to learn the skill from the human-video demonstration and then use it to plan and execute the task.

Developed HAT, an end-to-end policy that models human demonstrators as a distinct humanoid embodiment to enable scalable imitation learning. The framework utilizes a unified kinematic representation to directly map egocentric human video to robot actions, bypassing the need for complex intermediate affordance representations. This approach allows for efficient learning from diverse human demonstrations —significantly improving policy robustness and out-of-distribution generalization for bimanual manipulation.

We developed an unsupervised approach to learn temporal abstractions of skills incorporating agent-environment interactions. We hope to learn representations of patterns of motion of objects in the environment, or patterns of change of state. Our approach is able to learn semantically meaningful skill segments across robot and human demonstrations, despite being completely unsupervised.

We developed an algorithmic framework to extract different intrinsic features from human demonstrations. We are studying various features, including interactions with objects along the trajectory, analyzing the environment for interactions with the background (e.g., wiping or writing), and classifying the type of motion within a trajectory segment (e.g., shaking, rotating, or transporting).

We presented a method to learn human motions using a Learning-From-Demonstration approach. Using Dynamic Motion Primitives, we were able to teleoperate a Franka Panda Arm using the learned trajectories.