Information about Uncalibrated Image-Based Robotic Visual Servoing (knowdiff.net)

Visiting Lecturer Program (140)

Speaker: Azad Shademan

Ph.D. candidate

Department of Computing Sciences

University of Alberta, Canada

Title: Uncalibrated Image-Based Robotic Visual Servoing

Local Host: Ms. Nasim Pouraryan

Time: Wednesday, November 5, 2008, 12:30-2:00 pm

Location: Faculty of Electrical and Computer Engineering, University of Tehran, Tehran

Abstract:

Design of versatile vision-based robotic systems demands a solution with little or no dependence on system parameters. The problem of real-time vision-based control of robots has been long studied as robotic visual servoing. Most provably stable solutions to this problem require calibrated kinematic and camera models, because in a precisely calibrated system one can model the visual-motor function analytically. The uncalibrated approach has received limited attention mainly because the stability analysis is not as straightforward as that of calibrated image-based architecture. In an uncalibrated system the visual-motor function is not known, but partial derivative information (Jacobian) can be learned by tracking visual measurements during motion. In this talk, we study the uncalibrated image-based visual servoing and present different Jacobian learning methods.

Speaker: Azad Shademan

Ph.D. candidate

Department of Computing Sciences

University of Alberta, Canada

Title: Uncalibrated Image-Based Robotic Visual Servoing

Local Host: Ms. Nasim Pouraryan

Time: Wednesday, November 5, 2008, 12:30-2:00 pm

Location: Faculty of Electrical and Computer Engineering, University of Tehran, Tehran

Abstract:

Design of versatile vision-based robotic systems demands a solution with little or no dependence on system parameters. The problem of real-time vision-based control of robots has been long studied as robotic visual servoing. Most provably stable solutions to this problem require calibrated kinematic and camera models, because in a precisely calibrated system one can model the visual-motor function analytically. The uncalibrated approach has received limited attention mainly because the stability analysis is not as straightforward as that of calibrated image-based architecture. In an uncalibrated system the visual-motor function is not known, but partial derivative information (Jacobian) can be learned by tracking visual measurements during motion. In this talk, we study the uncalibrated image-based visual servoing and present different Jacobian learning methods.

Vision-Based Control A A A B B B current desired Left Image Right Image

Vision-Based Control Left Image Right Image B B B

Where is the camera located? Eye-to-Hand e.g.,hand/eye coordination Eye-in-Hand

Eye-to-Hand

e.g.,hand/eye coordination

Eye-in-Hand

Vision-Based Control Feedback from visual sensor (camera) to control a robot Also called visual servoing Visual servoing is the task of minimizing a visually specified objective by giving appropriate control commands to a robot Is it any difficult ? Images are 2D, the robot workspace is 3D 2D data 3D geometry

Feedback from visual sensor (camera) to control a robot

Also called visual servoing

Visual servoing is the task of minimizing a visually specified objective by giving appropriate control commands to a robot

Is it any difficult ?

Images are 2D, the robot workspace is 3D 2D data 3D geometry

Visual Servo Control law Position-Based: Robust and real-time pose estimation + robot’s world-space (Cartesian) controller Image-Based: Desired image features seen from camera Control law entirely based on image features Hybrid: Depth information is added to image data to increase stability

Position-Based:

Robust and real-time pose estimation + robot’s world-space (Cartesian) controller

Image-Based:

Desired image features seen from camera

Control law entirely based on image features

Hybrid:

Depth information is added to image data to increase stability

Position-Based Robust and real-time relative pose estimation E xtended K alman F ilter to solve the nonlinear relative pose equations. Cons: EKF is not the optimal estimator. Performance and the convergence of pose estimates are highly sensitive to EKF parameters.

Robust and real-time relative pose estimation

E xtended K alman F ilter to solve the nonlinear relative pose equations.

Cons:

EKF is not the optimal estimator.

Performance and the convergence of pose estimates are highly sensitive to EKF parameters.

Position-Based Desired pose Estimated pose

Position-Based Measurement noise State variable Process noise yaw pitch roll Measurement equation (projection) is nonlinear and must be linearized.

K x k-1,k-1 z k R k P k,k P k,k-1 C k Q k-1 x k,k x k,k-1 P k-1,k-1 Kalman Gain Measurement noise covariance A priori error cov. @ k-1 Process noise covariance Initial/previous state Linearization Measurement State update State prediction Error cov. prediction Error cov. update

Image-Based Desired Image feature Extracted image feature

Visual-motor Equation x 1 x 2 x 3 x 4 q=[q 1 … q 6 ] Visual-Motor Equation This Jacobian is important for motion control.

Visual-motor Jacobian Image space velocity Joint space velocity A A B B

Image-Based Control Law Measure the error in image space Calculate/Estimate the inverse Jacobian Update new joint values

Measure the error in image space

Calculate/Estimate the inverse Jacobian

Update new joint values

Image-Based Control Law Desired Image feature Extracted image feature

Jacobian calculation Analytic form available if model is known. Known model Calibrated Must be estimated if model is not known Unknown model Uncalibrated

Analytic form available if model is known. Known model Calibrated

Must be estimated if model is not known

Unknown model Uncalibrated

Calibrated: Interaction Matrix Analytic form depends on depth estimates. Camera/Robot transform required. No flexibility. Camera Velocity

Analytic form depends on depth estimates.

Camera/Robot transform required.

No flexibility.

Uncalibrated: Visual-Motor Jacobian A naïve method: Orthogonal projections

A naïve method:

Orthogonal projections

Uncalibrated: Visual-Motor Jacobian A naïve method: Orthogonal projections

A naïve method:

Orthogonal projections

Uncalibrated: Visual-Motor Jacobian A naïve method: Orthogonal projections …

A naïve method:

Orthogonal projections

Uncalibrated: Visual-Motor Jacobian A popular local estimator: Recursive secant method (Broyden update):

A popular local estimator:

Recursive secant method (Broyden update):

Relaxed model assumptions Traditionally: Local methods No global planning (-) Difficult to show asymptotic stability condition is ensured (-) Main problem of traditional methods is the locality. Calibrated vs. Uncalibrated Model derived analytically Global asymptotic stability (+) Optimal planning is possible (+) A lot of prior knowledge on the model (-) Global Model Estimation ( Research result ) Optimal trajectory planning (+) Global stability guarantee (+)

Relaxed model assumptions

Traditionally:

Local methods

No global planning (-)

Difficult to show asymptotic stability condition is ensured (-)

Main problem of traditional methods is the locality.

Model derived analytically

Global asymptotic stability (+)

Optimal planning is

possible (+)

A lot of prior knowledge on the model (-)

Global Model Estimation ( Research result )

Optimal trajectory planning (+)

Global stability guarantee (+)

Synopsis of Global Visual Servoing Model Estimation (Uncalibrated) Visual-Motor Kinematics Model Global Model Extending Linear Estimation (Visual-Motor Jacobian) to Nonlinear Estimation Our contributions: K-NN Regression-Based Estimation Locally Least Squares Estimation

Model Estimation (Uncalibrated)

Visual-Motor Kinematics Model

Global Model

Extending Linear Estimation (Visual-Motor Jacobian) to Nonlinear Estimation

Our contributions:

K-NN Regression-Based Estimation

Locally Least Squares Estimation

Local vs. Global Key idea: using only the previous estimation to estimate the Jacobian RLS with forgetting factor Hosoda and Asada ’94 1 st Rank Broyden update: Jägersand et al. ’97 Exploratory motion: Sutanto et al. ‘98 Quasi-Newton Jacobian estimation of moving object: Piepmeier et al. ‘04 Key idea: using all of the interaction history to estimate the Jacobian Globally-Stable controller design Optimal path planning Local methods don’t!

Key idea: using only the previous estimation to estimate the Jacobian

RLS with forgetting factor Hosoda and Asada ’94

1 st Rank Broyden update: Jägersand et al. ’97

Exploratory motion: Sutanto et al. ‘98

Quasi-Newton Jacobian estimation of moving object: Piepmeier et al. ‘04

Key idea: using all of the interaction history to estimate the Jacobian

Globally-Stable controller design

Optimal path planning

Local methods don’t!

K-NN Regression-based Method q 1 q 2 3 NN q 1 q 2 x 1 ?

(X,q) L ocally L east S quares Method q 1 q 2 x 1 ? K-neighbour(q)

Eye-to-hand Experiments Puma 560 Stereo vision Features: projection of the end-effector’s position on image planes (4-dim) 3 DOF for control

Puma 560

Stereo vision

Features: projection of the end-effector’s position on image planes (4-dim)

3 DOF for control

Measuring the Estimation Error

Global Estimation Error

Visual Task Specification Image Features: Geometric primitives (points, lines, etc.) Higher order image moments Shape parameters … Visual Tasks Point-to-point (point alignment) Point-to-line (colinearity) Point-to-plane (coplanarity) …

Image Features:

Geometric primitives (points, lines, etc.)

Higher order image moments

Shape parameters

…

Visual Tasks

Point-to-point (point alignment)

Point-to-line (colinearity)

Point-to-plane (coplanarity)

…

Eye-in-hand

Eye-in-Hand Experiments

Eye-in-Hand Experiments

Eye-in-Hand Experiments

Eye-in-Hand Experiments

Mean-Squared-Error Task 1 Task 2

Task Errors

Conclusions Reviewed position-based and image-based visual servoing schemes. Presented two global methods to learn the visual-motor function. KNN suffers from the bias in local estimations. LLS (global) works better than the KNN (global) and local updates. The Jacobian of more complex visual tasks can also be learned using LLS method. [email_address]

Reviewed position-based and image-based visual servoing schemes.

Presented two global methods to learn the visual-motor function.

KNN suffers from the bias in local estimations.

LLS (global) works better than the KNN (global) and local updates.

The Jacobian of more complex visual tasks can also be learned using LLS method.

Thank you!

Visual Ambiguity: Single Camera

Visual Ambiguity: Stereo

Uncalibrated 3D Stereo Image-based Dynamic Visual Servoing for Robot ... uncalibrated stereo camera systems ... In Image-Based Visual Servoing ...

Read more

6D Image-based Visual Servoing for Robot ... provide a solution to the open problem of uncalibrated 6D image-based visual servoing for robot ...

Read more

Uncalibrated Image-based Position-Force Adaptive Visual ... for visual servoing of constrained robots subject to dynamic friction is proposed.

Read more

... a new control scheme to control the robot using information of a uncalibrated stereo ... Uncalibrated Image-Based Visual Servoing ...

Read more

Uncalibrated Image-Based Visual Servoing of Rigid-Link Electrically Driven Robotic ... the uncalibrated fixed-camera visual servoing problem of ...

Read more

... IEEE International Conference on Robotics ... for uncalibrated image-based visual servoing. ... Uncalibrated Visual Servoing ...

Read more

Uncalibrated 3D Stereo Image-based Dynamic Visual Servoing for Robot ... Image-Based Visual Servoing ... image-based PID uncalibrated visual servoing ...

Read more

Read "Uncalibrated image-based visual servoing" on ... our new uncalibrated image Jacobian is ... R ELATED W ORK Vision-based robot control systems are ...

Read more

An image-based control is introduced to produce simultaneous ... a new adaptive scheme for visual servoing of constrained robots subject to dynamic ...

Read more

## Add a comment