SDF: Nut and Bolt Extension

Learning Objectives

This tutorial shows how a Franka Robot can be programmed through state machines to tie nuts onto bolts (SDF meshes) using Isaac Sim Core. This tutorial is a starting point for further automating robot scenarios that deal with detailed assets that require SDF meshing.

10 Minute Tutorial

Getting Started

Prerequisites

• Review the Introductory Tutorial Series prior to beginning this tutorial.

The Environment

This environment compromises of the default ground plane, a table, a vibrating table (to feed the nuts one by one to the Franka Robot), a pipe with the bolts enfused on it, other minor assets, a number of nuts on the vibrating table and the Franka Robot of course.

SDF Meshes

Both the nut and bolt assets in the enviroment above are using a collision detection feature called Signed-Distance-Field (SDF). This feature allows the simulation of dynamic and kinematic rigid bodies that use high-detail triangle meshes as their collision shape. This example utilizes this feature to simulate such contact-rich interactions between dynamic objects. Learn more about SDF and how to apply it to an asset here.

Extension Overview

To run the demo, load the Franka Nut and Bolt scene by selecting the option Isaac Examples -> Manipulation -> Franka Nut and Bolt. After opening the extension, press on Load World button.

At any time you can press the Reset and all the assets will return to its default position. Pressing play in the viewport would restart the simulation.

Code Overview

We will be using the same extension base wrappers that are used in the Core Tutorials. This extension consists of 5 files:

• franka_nut_and_bolt_extension.py where the UI elements sits. This is enheriting from the base extension example used in the Core Tutorials.

• franka_nut_and_bolt.py where the whole logic sits. This also uses the same BaseSample used for developing extensions under the Core Tutorials.

  • The __init__ function has a set of constant variable that are going to be used throughout the example.

  • The setup_scene function is importing and referencing all the assets that are going to be used during the demo.

  • As for the setup_post_load function, this is where the assets are placed in their according positions while taking into account the offsets between the assets stacked on each other (e.x: pipe and bolts). This is also where we add a physics callback to generally keep controllers updated on the current state of the enviroment. The controller can then take any necessary actions based on the current state.

• nut_and_bolt_controller.py is a high-level simple state machine and controller that syncronizes the automation of other lower-level controllers underneath.

• screw_controller.py is a controller for the process of tieing a nut onto a bolt.

• nut_vibra_table_controller.py is a state machine for feeding the nuts one by one to the end of the vibrating table for the Franka robot to pick one.

All the controllers and state machines have the states documented in the code.

How to build on top of the Demo

Click on the Open Source Code symbol on the Franka Nut and Bolt Demo Extension. After going through the source code, there are a few things you can do to build on top of this demo:

• Build faster and more accurate custom controllers that may be based on other motion generator algorithms. The demo currently uses RMPFlow as the motion generator that indures certain limitations on the screwing angles. The controllers developed in this demo are relatively simple and meant to show the SDF interactions. Controllers are usually more complex and detailed.

• Change the number of nuts or bolts variables in the init function of franka_nut_and_bolt.py.

• Make the extension tensor-based to enable cloning of environments to run in parallel on a single workstation. You can use the Cloner-API to do so.

• Use Reinforcement Learning to train the Franka robot to tie the nut onto the bolt using Isaac Gym.

• Continue on with building and training further manufacturing robots that require the use of such high-detail collision interactions.

Summary

This tutorial covered the following topics:

1. SDF Meshes

2. The Nut and Bolt Extension overview

   To use Isaac Sim Core in programming state machines and robot controllers for scenes with contact-rich interactions between SDF-enabled objects.