At the IROS conference in Vancouver, Li Feifei, a computer vision expert and director of Stanford's AI Lab & Vision Lab, gave a speech on "A Quest for Visual Intelligence", which is the first time that Li Feifei participated in the IROS robot-themed conference. It is worth noting that a paper on the new robot learning framework, Neural Task Programming: Learning to Generalize Across Hierarchical Tasks, was recently discovered in Arxiv. The main author of the paper is from Stanford. The instructor is Li Feifei and his husband. Silvio Savarese. This also indicates that the two communities of artificial intelligence and robots are more and more coincident in terms of visual and perceptual, and the cooperation will become closer and closer.

We propose a new robot learning framework called Neurological Task Programming (NTP), which can be learned with less demonstration and neural programming. NTP can recursively input incoming normative tasks (such as video demonstrations of tasks) into more sub-task specifications that are passed to the hierarchical neural program, which interacts with the environment through the underlying sub-program that can be called. At the same time, we validated our method in three robot maneuvering tasks. In the experiment, NTP demonstrated the strong generalization ability of the sequential tasks in displaying the hierarchical structure and the combined structure. The experimental results show that NTP has a good effect in learning and expanding the learning process of variable length, variable topology and changing unknown tasks.

Today, with the increasing emphasis on “human-machine collaboration”, robots often interact with people for long periods of time in certain environments, such as object sorting, assembly, and cleaning. However, in the past, robots usually need to interact with people in a fixed scene. How to make robots adapt to complex operational tasks, new mission targets and surrounding environment is a major challenge in robotics.

Imagine an object classification task in a warehouse, which usually includes specific tasks such as classification, retrieval, and packaging. Each task can be decomposed into several actions, such as grabbing, moving, laying down, etc., which constitute such as grabbing and placing. Sub-tasks, if you consider the combination of different objects, sorts, sub-tasks, the changes will be very complicated, for example, the four categories of items into four containers, the combination of which will be 256 There are many. In this paper, we hope to address two main challenges of complex task planning: a) a learning strategy from a new conceptualization to a new task goal, and b) a combination of basic native code modules that interact with the long-term environment.

The key basic idea of ​​NTP is in cross-task learning and reusable representations in shared domains. NTP interprets a task specification that describes the task process and the final target time series and transforms its hierarchical policy instance into a neural program. NTP decodes the target object from the input specification and decomposes it into subtasks and interacts with the surrounding environment and feedback until it reaches the target. Each program calls the environment observation and task specification (input), and outputs the next subroutine and The corresponding subtask specification.

As shown in the figure above, given the input, task specification and current environment, the NTP model predicts which subroutine to run and as the input to the next subtask, and determines if the current program is finished, otherwise it will continue Cycle through tasks.

test

The research team evaluated NTP by simulating single-arm operation tasks in simulation and actual robot experiments. Tasks include: stacking blocks, target sorting, and cleaning up the table.

Two goals of the study: I. Learning multiple tasks in the same domain; and II. Forming generalization capabilities in testing through a single example.

As shown, the robot can learn from the simulated Demo environment (including image, video, VR input) and complete the stacking of blocks;

NTP programming: Output the policy induction program through the hierarchical task conditions and call the Robot API to operate.

The figure above shows a sample execution trace of NTP in a stacked block task. The task is to stack the alphabet blocks as specified, the top-level program participates as an input in the entire presentation, predicts the next running subroutine, and as part of the specified task for a particular input, the underlying API calls the robot to act accordingly (eg, Move_to(Blue), Grasp (Blue), etc.). When the program end character (EOP) is True, the current program stops and returns to its calling program.

If the environment changes (such as vandalism to complete a completed task), the robot will detect changes in the environment and repeat the task.

Changes to the task structure include: changing completion conditions (task semantics), variable subtask alignment (task topology), and larger task volumes (length of tasks). As the number of tasks increases, NTP can produce better results for new task teachers and goals.

WMS Light Guide Rack

Twinkle System Technology Co Ltd , https://www.pickingbylight.com