Robotic Graphics for Astronaut Training

This type of system, in which a robot manipulator "stands in" for objects manipulated in a virtual environment has been termed "robotic graphics" by team member Dr. William McNeely.

The baseline system uses a seven degree-of-freedom (DOF) robot manipulator (K-1207) made by the Robotics Research Corporation as the force feedback device. This project will design and implement a computing environment and sensor array which will safely simulate the motion of th ORUs in response to forces applied by astronauts.

This document will describe the project goals and approach in several sections.

Safty

Purpose

Clearly safety must be paramount in the design of this system. Existing robot safety standards rely on a simple concept: erect a set of physical and or electronic barriers to guarantee that no humans will occupy the robot workvolume at any time that the robot is active or potentially active. Clearly this standard cannot be met in "robotic graphics" or other force feedback devices. Thus, development of an appropriate safety methodology is a key research component of this project. Another robotic application has this same problem: medical robotics. This project will make use of some of the pioneering work which has been done in medical robotics, and hopefully contribute new safety methods to that field as well as the field of robotic graphics.

Scope

This is a research project designed to designed to determine feasibility and performance limits of the baseline system and to explore emerging research issues in robotic graphics. Therefore, although safety will be paramout in our implementation as well as a key research issue in the project, it is not appropriate to conduct our project as a traditional excercise in safety engineering. To do this would severely impact our ability to innovate. Instead our product will be a prototype safety system, a performance evaluation, and a small body of operating experience. Test operators will be protected at all times by human observers with hardware stop buttons in addition to our experimental safety measures.

Risks

The basic threat to safety of the user of the baseline system is the risk of unwanted exchange of mechanical energy between the robot and human operator. There are several ways this unwanted energy transfer between manipulator and user could occur. These include,

A robot sensor or control circuit could fail, causing uncontrolled robot motion.
A user could do something which is not expected by the robot program.
Software could command robot motion that harms the user.
Another possible threat is electric shock from contact with the system. We will assume that adherence to standard electrical codes will adequately address this risk.

Approach

The fundamental approach to safety which will be employed is to add redundant sensing devices to the robot, monitor it's position, force, and velocity, and to remove the robot's ability to cause damage when robots motion exceeds a specified "safe" envelope, or when sensor readings disagree. We would like to use the idea that as more sensors are added, the safety of the system is increased. Unfortunately, it is not obvious that this is the case.

Sensor based robot control is still a rapidly developing area. So we need an architecture which permits us to easily add new safety sensors or to remove them. A potential problem is the growing number of interactions as sensor systems are added. A software system which could intelligently combine the readings of each sensor would thus introduce its own set of risks.

We assume here that the risk of software error is proportional to the number of interactions between distinct pieces of information. For example, if readings are obtained from N sensors, there are potentially N! interactions between them.

Software safety is a field which is receiving increasing attention. On both theoretical and practical levels, it is very difficult to insure that a piece of software is "correct" and thus that it is safe. In critical applications such as flight control systems, elaborate and expensive methodologies are now in use to reduce the chances of software safety problems.

A possible reason for the difficulty of checking software is that it is so easy to make software complex. We take it as axiomatic that as a system is made more complex, its safety is increasingly difficult to ascertain. Thus it is desireable to keep the safety systems as simple as possible.

The safety of hardware systems can also be difficult to ascertain. However, the physical nature of hardware makes the cost directly proportional to its physical complexity (the exception to this is probably VLSI hardware which will not be considered below). The rising cost with complexity creates an incentive to develop low complexity systems which in turn are easier to verify.

Thus, in evaluating potential designs for this project, we will prefer hardware safety measures over software safety measures, and simple systems over that more complex ones.

Robust Safety

A further issue is the "robustness" of the safety measures. Any safety sensor will have a finite noise probability that noise will result in a false risk indication. As more and more safety sensors are added, the probability of a false risk signal goes up. In most robot safety systems, there are only two possible conditions: "safe" and "unsafe". In the unsafe condition, a hardware circuit disengages all robot power. After the problem is cleared, the robot system must be re-booted, calibrated, etc. and this takes a considerable amount of time. An otherwise "safe" system which must constantly be re-started is of little use. Thus, the safety system must deal with this problem.

We will take the following approach to "robust safety". Multiple sensor systems will measure quantities related to the safety of the system such as manipulator end effector position, end effector force, etc. Two types of judgements will then be made of the system's safety. The first type of judgement will be based instantaneously on the sensor value itself.

For example, a system measuring the distance from the end effector to the robot base, will make a safety judgement based on the distance measurement at each instant. If the distance from base to end effector exceeds a threshold, the robot might be capable of hitting the operator's body or head and thus a potentially unsafe condition exists.

The second type of safety judgement will require comparison between sensor values to ensure consistency. It is assumed that if sensor values disagree, then an unsafe condition exists. The difficulty here is that different sensor systems will have different units and coordinate systems, and thus will require computation in order to be compared against each other. This means that the comparisons will have to be done in software and thus will be less reliable or verifiable.

A given sensor system may be able to contribute to both types of judgements of system safety. In the example above, the distance measurement system will make a local judgement of safety based on the measured radius alone. At the same time, it will communcate its value to a software process which will compare it's value to the computed value based on joint sensors and forward kinematics calculations. If the values differ by more than a specified amount, this process will signal an unsafe condition to the hardware safety circuit.

During system operation, noise or "non-serious" safety threats could cause one or more sensor systems to signal the "unsure" state. This will cause the system safety level to go below the maximum value ("safe"). To maintain "robust" safety at intermediate safety levels, the system will continue to run, but at degraded physical capability. As the safety level increases, the system will gracefully and gradually resume full physical capability without pausing or operator intervention. In the event that the system safety level reaches "unsafe", (even if no individual sensor reads "unsafe") the system will cause a hardware shutdown. The hardware shutdown will be interfaced to the traditional emergency stop button circuit and may require re-booting or initialization of software.

Guiding Principles

The Astronaut training force feedback system will have multiple safety mechanisms in both hardware and software. The safety measures will be as much as possible independent and decoupled from each other so that new safety sensors can be added incrementally with a net gain of system safety.
The guiding principles of the safety architecture will be:

Redundant sensors for measurement of critical safety quantities.
A notion of safety which contains more than two levels (i.e. has one or more intermediate states between safe and unsafe).
Hardware safety measures. The hardware safety measures must be simple, and be designed to create safe states in the event of their own hardware failure.
Software safety measures. These will resolve different measurement units and coordinate systems, and compare measurements from different sensors. Discrepancies will be reported to the hardware safety system.
A notion of "Robust Safety".

Biomechanics and Human Factors

Objective

The objective of the Biomechanics and Human Factors section is to establish a set of specific aims defining the limits of physical interaction between an astronaut and a robotic-computer graphics system. The purpose of the system is to provide a realistic training environment for crew participating in extravehicular activities (EVA). Unfortunately, the precise limits of human work performance in zero gravity is unknown. Many questions remain unanswered and much work is in progress.

Project Goals

The capabilities of the robotic interface are expansive. The list of potential simulations range from handling whole satellites to operating EVA hand tools for removing threaded fasteneners. For this phase of the project, the robotics system will be constrained to the handling of small and large modules under three operational senarios:

Stationary conditions. In this operational scenario, the orbiter is in "free drift" mode and the crew (either tethered or in foot restraints) is free to manually maneuver items about the payload bay.

Remote manipulator sytem (RMS) operations. In this scenario, the EVA crew member, holding the module, is located on the RMS end effector in foot restraints. The RMS is then used to translate both the module and the crew member about the payload bay.

Operational conditions involving the orbiter's vernier reaction control system (VRCS) firings. In this scenario, the crew member (in foot restraints) is holding the module during VRCS engine firings.

Other EVA operations such as those involving assembly, use of tools, and other specialized activities will be reserved for future development.

Approach

The approach for defining specific biomechanic and human factors aims will be to create an open-ended database of EVA crew capabilities and work performance. It is expected that as both in-flight experiences and earth-bound investigations continue, the specific aims defining physical interaction with the robotic system may require revision.

Background

Paramount to the development of a realistic training tool coupling robotics and computer graphics is identifying human abilities and limitations in zero gravity. In addition to working in an environment without gravity, EVA crewmembers must wear an extravehicular manuevering unit (EMU) or space suit which is pressurized to 4.3 psi. While absolutely necessary, the suit further limits EVA capabilities by restricting mobility. Furthermore, mission planners rarely know the physical capabilities of an individual crew member. Will the EVA crew member be big and strong, or small and dextrous? Each of these areas contribute to the challenge of specifying the human-machine interface requirements.

Study of previous shuttle experiences, identifying specific EVA tasks, and reviewing the current literature on crew capabilities provides essential background for the generation of accurate Biomechanic and Human Factors objectives.

Relevant EVA Experiences

During the shuttle era, there have been a number of flights which have significantly contributed to the "crew capability" body of knowledge. These previous flights, as well as current EVA training facilities, provide both rookie and veteran crew members with some idea of what to expect during the EVA. A 1-3 below] have attempted to classify generic missions requiring EVA support. These EVA activities include:
Alignment of transmitter/receiver elements
Deployment/retraction of solar arrays
Large space structure or truss assembly
Large module capture, manipulation, and launch
Small module manipulation, construction, assembly, and installation
Satellite consumables recharge
Orbiter supported activities (using the remote manipulator system)
On station test and evaluation
Specific examples of these activities being conducted in-flight can be found in the ref 4].

The Remote Manipulator System (RMS)

Another orbiter initiated action is operation of the shuttle RMS. With an EVA crew member on the end of the RMS, movement of modules across the payload bay can be accomplished with relative ease. For the STS-61 mission (Hubble Space Telescope Service Mission - 1), the nominal course rate for the RMS was set at 0.7 feet per second [ref 5].

Crew Capabilities

The robotic/computer graphic system can provide personal experience to the astronaut asked to perform certain tasks during an EVA. Using this system, a crew member will be able to experience the answer to questions like:
If the VRCS engines fire while I'm holding module X, what kind of forces/torques will I have to apply to keep from banging it against the payload bay?
What will it feel like holding onto module X while the RMS moves me across the payload bay?
If I exert a reasonable amount of torque, how long will it take to rotate the solar array into the installation position?
Clearly, data on module mass and inertia will be required information for accurate simulation. Yet from a design point of view, how much force or torque a crew member can generate is just as important. Unfortunately, as mentioned earlier, the precise limits of human work performance in zero gravity is unknown. However, some data is available on:

Grip Strength
Pinch Strength
Grasp Strength
Grip Endurance
Small Item Assembly Tasks
Torque Capacity
Isolated Joint Strength
Small Module Forces and Torques
Isometric EVA Measurements

The results of these investigations have been summarized and provide the initial database on EVA crew capabilities. As this phase of the project is constrained to the handling of small and large modules, the limited existing data on grasp strength, small module forces and torques, and isometric EVA measurements will drive the Biomechanic and Human Factors objectives.

Specific Aims

Biomechanics and Human Factors specific aims for the UW-JSC Robotic Graphics Project include:
Orbiter Systems
Specific aims developed from orbiter systems include the capability to simulate both VRCS and RMS accelerations:

VRCS Accelerations

The robotic system must be able allow for orbiter accelerations from 0.0 to 3.3 x 10exp(-4) g's in any direction.
RMS Accelerations
The robotic system must be able to allow for RMS course rates from 0.0 to 2.0 feet per second in any direction.

Crew Capabilities

Specific aims developed from the experimental, human work capacity database include the capability to simulate small to large module handling:
Small and Large Module Handling
Direction Minimum Nominal Maximum
Push/Pull 0 N 50 N 250 N
Left/Right 0 N 50 N 250 N
Up/Down 0 N 50 N 1200 N
Yaw 0 N*m 15 N*m 75 N*m
Pitch 0 N*m 15 N*m 75 N*m
Roll 0 N*m 15 N*m 75 N*m
(non-Netscape viewers please refer here for this information)

Tool Applications

The robotic system shall be able react forces applied by EVA tools of up to 1000 N in any direction.
The robotic system shall be able to react torques applied by EVA tools of up to 250 N*m along any axis.
Please note that this phase of the project does not include implementation of the specific aims using tools. They are defined here for future applications.

For an explanation of the source for these specific aims, see the crew capabilities specific aims rationale.

Crew Interface

Specific aims for the crew interface include:

A removeable ORU work site interface. It is expected that EVA astronauts will switch one ORU for another within a short period of time. To maintain continuity during EVA simulations, the work site attached to the robotic end effector shall be changeable within two minutes by one person using standard metric or SAE fasteners. No special or custom tools shall be used.

An ORU work site consisting of a 0.75 x 0.75 meter surface with two standard EVA handrails (oval cross-section) of 0.5 meters in length. The EVA handrails shall be attached in parallel at opposite edges of the square surface facing the astronaut. The intention of this requirement is to provide a similar interface to a generic ORU or satellite.

A "chair" designed to maximize the load path to the astronauts feet. During small and large module handling tasks, the EVA crew is often situated in portable foot restraints. The intention of this requirement is to provide a similar load path with considerable design leeway.

Biomechanics and Human Factors References

Advanced EVA System Design Requirements Study. McDonnell Douglas Astronautics Corporation. Report No. MDC W0072, January 1986.
Advanced EVA System Design Requirements Study. Boeing Corporation. Report D180-28806-1, March 1986.
Extravehicular Activities Limitations (EVA) Study. Grumman Corporation. Report No. AS-EVALS-FR-8701, 1988.
Memo to USML-1 EIEs from Brian Matisak, USML-1 Pointing/Stabilization Engineer. Teledyne Brown Engineering. September 19, 1990.
Rajulu, SL, Klute, GK, Fletcher, L: Evaluation of COSTAR Mass Handling Characteristics in an Environment - A Simulation of the Hubble Space Telescope Service Mission. NASA Technical Paper 3489, August 1994.

Tasks and Applications

The following are a rough list of requirements for an application of this training capability being considered by the Boeing Co. Boeing is specifically interested in using this technology for improving the maintainability aspects of their aircraft designs. A robot-driven VR facility would allow engineers to quickly mock-up complex maintenance tasks for a given design and evaluate whether maintenance personnel will be able to perform these tasks safely and quickly. Any problems that may be revealed in this simulation can be rectified in the actual design before actual manufacturing takes place.

Boeing-specific requirements

Safety for a simulated 1-G environment: If a simulated object is so heavy that the user releases it, then the RRC arm should not allow the simulated object to drop and potentially harm the user.
For example, use a "dead man's handle" that the user must squeeze as a precondition for robot motion.

Manipulator grip that can be mounted on RRC arm's end effector

A cluster of 3 pushbuttons, mounted on or near manipulator grip, which can easily be pushed without releasing grip, whose states can be read out by a computer. An example is the Cybernet manipulator grip, although it may have insufficient mechanical bandwidth. It is described as: "The Cybernet PER-Force uses an aircraft-type sidearm-grip control stick that incorporates 3 cueing buttons, an analog trigger, and a palm-actuated deadman safety switch."

SGI computer with color monitor that will provide maximum 5 msec latency in the computation and application of forces at the manipulator grip

Boeing provided path control software for SGI

Boeing provided visualizer software for SGI, allowing for collisions to be highlighted on the visual display in real time

Boeing provided data

Simulated Dynamic Models

Requirements

The first phase of this project will consist solely of simulating large objects existing and being manipulated in free space. In phase two surface contact and collision will accounted for, while the combination of phase one and two including simulation of non-rigid or deformable objects will not be accomplished until the completion of phase three.

Phase I

The following properties must be known about each object to be simulated by this system:

The object mass properties (CG, mass matrix, total mass).
We will be primarily simulating objects that are well over 50 lbs.
An initial state of the object (position, velocity) [Note that position/velocity are 6-tuples]

Phase II

The following additional properties must be known about each object to be simulated by this system:
An exterior surface model comprised of polygonal primitives (bounded sections of planes, cylinders, spheres, etc.)
We assume that all object are rigid objects (i.e., that the mass properties are constant)

Phase III

The following additional properties must be known about each object to be simulated by this system:

The object stiffness

Objects will be simulated using a Newtonian dynamic model. We must determine whether orbital dynamics, STS-RMS dynamics, and/or STS reaction thrusters are significant to the simulation process. If so, these must be included in the dynamic model.

Operator forces on objects (via touching the robot-mounted fixtures and sensed via the robot-mounted force sensor) will cause changes in the simulation.

Simulated Dynamic Models

Requirements

The first phase of this project will consist solely of simulating large objects existing and being manipulated in free space. In phase two surface contact and collision will accounted for, while the combination of phase one and two including simulation of non-rigid or deformable objects will not be accomplished until the completion of phase three.

Phase I

The following properties must be known about each object to be simulated by this system:

The object mass properties (CG, mass matrix, total mass).
We will be primarily simulating objects that are well over 50 lbs.
An initial state of the object (position, velocity) [Note that position/velocity are 6-tuples]

Phase II

The following additional properties must be known about each object to be simulated by this system:
An exterior surface model comprised of polygonal primitives (bounded sections of planes, cylinders, spheres, etc.)
We assume that all object are rigid objects (i.e., that the mass properties are constant)

Phase III

The following additional properties must be known about each object to be simulated by this system:

The object stiffness

Objects will be simulated using a Newtonian dynamic model. We must determine whether orbital dynamics, STS-RMS dynamics, and/or STS reaction thrusters are significant to the simulation process. If so, these must be included in the dynamic model.

Operator forces on objects (via touching the robot-mounted fixtures and sensed via the robot-mounted force sensor) will cause changes in the simulation.

Robot Motion Constraints and Sensing Requirements

Robot Motion

The robot motion dynamics is physically limited only by its internal components (actuators, sensors, power amps, etc.) and by physical boundaries (walls, floor, etc.). However, for reasons of safety, we need to impose reduced dynamic capabilities (maximum force output, maximum velocity, etc.). The requirements for these limitations are specified in the safety section.

Force Sensing

Some form of force sensing is needed to detect intended operator interactions with the simulated environment. The RR_Arm contains internal joint torque sensors. However, for reasons of redundancy and force resolution, we will need to have a force sensor at the end of the robot that is capable of resolving Cartesian forces and moments within the ranges given in the Biomechanics and Human Factors section of this document.

Position Sensing

The RR_Arm contains two types of internal position sensors mounted with one each per joint:

Optical rotary shaft encoders

Joint potentiometers or resolvers
(specific model data needed to determine this)

Regardless of the two types of sensors used, together they will provide sufficent position resolution and measurement redundancy.

Fixturing

The simulation will need to include a capability of mounting objects on the robot which simulate the shape of surfaces that are to be manipulated by the operator. For example, during a satellite capture sequence, the robot must hold a mock-up of the appropriate grappling fixture. These fixtures must be rigid structures since we are not planning on including internal dynamics of flexing fixtures.

Kinematics

Safety requirements dictate a minimum allowed distance between any part of the robot/fixture and the operators head and torso (distance to the operator's hands may be zero). To meet this requirement, the DH parameters of the RR_Arm must be known with sufficent accuracy to give 1-cm absolute cartesian positioning accuracy.

Control

We will implement a hybrid position/impedance controller to support the dynamic simulation. It is anticipated that the existing RR_Arm controller will be insufficient for this task and that an external controller will be needed to work in conjunction with the RR_Arm controller.

The RR_Arm controller includes an internal PID control system, inverse kinematics for cartesian control, and some fault checking. To achieve impedance control, at first we will try to measure the force/torque, and control the displacement using the RR_Arm PID controller. If this solution will not provide the desired bandwidth, a force control will have to be implemented by closing a force loop outside of this controller. In any case, parts of the existing controller will be used where possible to provide some additional error checking of position boundaries, etc.

The hybrid controller, running on a separate computation engine, will need to be interfaced to the existing RR_Arm controller. It must be able to read out current positions and current joint torques (from joint torque sensors) and send command force (and position) back to the RR_Arm controller. In addition, it must be able to read the force data from the external force sensor mounted at the end of the RR_Arm.

Emphasis will be given in designing a robust and stable controller. If necessary, different implementations will be tried (at least one to be passive), to guarantee the stability of the overall system, including the operator. The controller needs also to be robust to changes in the inertia-stiffness parameters of the virttual objects.

The control loop in this computation engine must run at a minimum of 400Hz to match the maximum update rate of the RR_Arm controller.

Investigation Phase

We will investigate the following properties of the robot and its controller:
We will determine the robot mechanical bandwidth so that we can tune the controller to avoid exciting mechanical modes of the structure.
We will determine the range object masses, object stiffnesses, object velocities, and object accelerations that can be simulated while maintaining robot control stability and subjective fidelity of the perceived object impedance. These numbers will be useful to make a comparison to the real objects that JSC wants to have simulated and determine the performance of the system.

Control

Robot Motion Constraints and Sensing Requirements

Robot Motion

The robot motion dynamics is physically limited only by its internal components (actuators, sensors, power amps, etc.) and by physical boundaries (walls, floor, etc.). However, for reasons of safety, we need to impose reduced dynamic capabilities (maximum force output, maximum velocity, etc.). The requirements for these limitations are specified in the safety section.

Force Sensing

Some form of force sensing is needed to detect intended operator interactions with the simulated environment. The RR_Arm contains internal joint torque sensors. However, for reasons of redundancy and force resolution, we will need to have a force sensor at the end of the robot that is capable of resolving Cartesian forces and moments within the ranges given in the Biomechanics and Human Factors section of this document.

Position Sensing

The RR_Arm contains two types of internal position sensors mounted with one each per joint:

Optical rotary shaft encoders

Joint potentiometers or resolvers
(specific model data needed to determine this)

Regardless of the two types of sensors used, together they will provide sufficent position resolution and measurement redundancy.

Fixturing

The simulation will need to include a capability of mounting objects on the robot which simulate the shape of surfaces that are to be manipulated by the operator. For example, during a satellite capture sequence, the robot must hold a mock-up of the appropriate grappling fixture. These fixtures must be rigid structures since we are not planning on including internal dynamics of flexing fixtures.

Kinematics

Safety requirements dictate a minimum allowed distance between any part of the robot/fixture and the operators head and torso (distance to the operator's hands may be zero). To meet this requirement, the DH parameters of the RR_Arm must be known with sufficent accuracy to give 1-cm absolute cartesian positioning accuracy.

Control

We will implement a hybrid position/impedance controller to support the dynamic simulation. It is anticipated that the existing RR_Arm controller will be insufficient for this task and that an external controller will be needed to work in conjunction with the RR_Arm controller.

The RR_Arm controller includes an internal PID control system, inverse kinematics for cartesian control, and some fault checking. To achieve impedance control, at first we will try to measure the force/torque, and control the displacement using the RR_Arm PID controller. If this solution will not provide the desired bandwidth, a force control will have to be implemented by closing a force loop outside of this controller. In any case, parts of the existing controller will be used where possible to provide some additional error checking of position boundaries, etc.

The hybrid controller, running on a separate computation engine, will need to be interfaced to the existing RR_Arm controller. It must be able to read out current positions and current joint torques (from joint torque sensors) and send command force (and position) back to the RR_Arm controller. In addition, it must be able to read the force data from the external force sensor mounted at the end of the RR_Arm.

Emphasis will be given in designing a robust and stable controller. If necessary, different implementations will be tried (at least one to be passive), to guarantee the stability of the overall system, including the operator. The controller needs also to be robust to changes in the inertia-stiffness parameters of the virttual objects.

The control loop in this computation engine must run at a minimum of 400Hz to match the maximum update rate of the RR_Arm controller.

Investigation Phase

We will investigate the following properties of the robot and its controller:
We will determine the robot mechanical bandwidth so that we can tune the controller to avoid exciting mechanical modes of the structure.
We will determine the range object masses, object stiffnesses, object velocities, and object accelerations that can be simulated while maintaining robot control stability and subjective fidelity of the perceived object impedance. These numbers will be useful to make a comparison to the real objects that JSC wants to have simulated and determine the performance of the system.

(*) Note: Most of the BRL publications are available on-line in a PDF format. You may used the publication's reference number as a link to the individual manuscript.