Information about Knowdiff visiting lecturer 140 (Azad Shademan): Uncalibrated Image-Based...

knowdiff.net

Design of versatile vision-based robotic systems demands a solution with little or no dependence on system parameters. The problem of real-time vision-based control of robots has been long studied as robotic visual servoing. Most provably stable solutions to this problem require calibrated kinematic and camera models, because in a precisely calibrated system one can model the visual-motor function analytically. The uncalibrated approach has received limited attention mainly because the stability analysis is not as straightforward as that of calibrated image-based architecture. In an uncalibrated system the visual-motor function is not known, but partial derivative information (Jacobian) can be learned by tracking visual measurements during motion. In this talk, we study the uncalibrated image-based visual servoing and present different Jacobian learning methods.

Design of versatile vision-based robotic systems demands a solution with little or no dependence on system parameters. The problem of real-time vision-based control of robots has been long studied as robotic visual servoing. Most provably stable solutions to this problem require calibrated kinematic and camera models, because in a precisely calibrated system one can model the visual-motor function analytically. The uncalibrated approach has received limited attention mainly because the stability analysis is not as straightforward as that of calibrated image-based architecture. In an uncalibrated system the visual-motor function is not known, but partial derivative information (Jacobian) can be learned by tracking visual measurements during motion. In this talk, we study the uncalibrated image-based visual servoing and present different Jacobian learning methods.

Vision-Based Control A A A B B B current desired Left Image Right Image

Vision-Based Control Left Image Right Image B B B

Where is the camera located? Eye-to-Hand e.g.,hand/eye coordination Eye-in-Hand

Eye-to-Hand

e.g.,hand/eye coordination

Eye-in-Hand

Vision-Based Control Feedback from visual sensor (camera) to control a robot Also called visual servoing Visual servoing is the task of minimizing a visually specified objective by giving appropriate control commands to a robot Is it any difficult ? Images are 2D, the robot workspace is 3D 2D data 3D geometry

Feedback from visual sensor (camera) to control a robot

Also called visual servoing

Visual servoing is the task of minimizing a visually specified objective by giving appropriate control commands to a robot

Is it any difficult ?

Images are 2D, the robot workspace is 3D 2D data 3D geometry

Visual Servo Control law Position-Based: Robust and real-time pose estimation + robot’s world-space (Cartesian) controller Image-Based: Desired image features seen from camera Control law entirely based on image features Hybrid: Depth information is added to image data to increase stability

Position-Based:

Robust and real-time pose estimation + robot’s world-space (Cartesian) controller

Image-Based:

Desired image features seen from camera

Control law entirely based on image features

Hybrid:

Depth information is added to image data to increase stability

Position-Based Robust and real-time relative pose estimation E xtended K alman F ilter to solve the nonlinear relative pose equations. Cons: EKF is not the optimal estimator. Performance and the convergence of pose estimates are highly sensitive to EKF parameters.

Robust and real-time relative pose estimation

E xtended K alman F ilter to solve the nonlinear relative pose equations.

Cons:

EKF is not the optimal estimator.

Performance and the convergence of pose estimates are highly sensitive to EKF parameters.

Position-Based Desired pose Estimated pose

Position-Based Measurement noise State variable Process noise yaw pitch roll Measurement equation (projection) is nonlinear and must be linearized.

K x k-1,k-1 z k R k P k,k P k,k-1 C k Q k-1 x k,k x k,k-1 P k-1,k-1 Kalman Gain Measurement noise covariance A priori error cov. @ k-1 Process noise covariance Initial/previous state Linearization Measurement State update State prediction Error cov. prediction Error cov. update

Image-Based Desired Image feature Extracted image feature

Visual-motor Equation x 1 x 2 x 3 x 4 q=[q 1 … q 6 ] Visual-Motor Equation This Jacobian is important for motion control.

Visual-motor Jacobian Image space velocity Joint space velocity A A B B

Image-Based Control Law Measure the error in image space Calculate/Estimate the inverse Jacobian Update new joint values

Measure the error in image space

Calculate/Estimate the inverse Jacobian

Update new joint values

Image-Based Control Law Desired Image feature Extracted image feature

Jacobian calculation Analytic form available if model is known. Known model Calibrated Must be estimated if model is not known Unknown model Uncalibrated

Analytic form available if model is known. Known model Calibrated

Must be estimated if model is not known

Unknown model Uncalibrated

Calibrated: Interaction Matrix Analytic form depends on depth estimates. Camera/Robot transform required. No flexibility. Camera Velocity

Analytic form depends on depth estimates.

Camera/Robot transform required.

No flexibility.

Uncalibrated: Visual-Motor Jacobian A naïve method: Orthogonal projections

A naïve method:

Orthogonal projections

Uncalibrated: Visual-Motor Jacobian A naïve method: Orthogonal projections

A naïve method:

Orthogonal projections

Uncalibrated: Visual-Motor Jacobian A naïve method: Orthogonal projections …

A naïve method:

Orthogonal projections

Uncalibrated: Visual-Motor Jacobian A popular local estimator: Recursive secant method (Broyden update):

A popular local estimator:

Recursive secant method (Broyden update):

Relaxed model assumptions Traditionally: Local methods No global planning (-) Difficult to show asymptotic stability condition is ensured (-) Main problem of traditional methods is the locality. Calibrated vs. Uncalibrated Model derived analytically Global asymptotic stability (+) Optimal planning is possible (+) A lot of prior knowledge on the model (-) Global Model Estimation ( Research result ) Optimal trajectory planning (+) Global stability guarantee (+)

Relaxed model assumptions

Traditionally:

Local methods

No global planning (-)

Difficult to show asymptotic stability condition is ensured (-)

Main problem of traditional methods is the locality.

Model derived analytically

Global asymptotic stability (+)

Optimal planning is

possible (+)

A lot of prior knowledge on the model (-)

Global Model Estimation ( Research result )

Optimal trajectory planning (+)

Global stability guarantee (+)

Synopsis of Global Visual Servoing Model Estimation (Uncalibrated) Visual-Motor Kinematics Model Global Model Extending Linear Estimation (Visual-Motor Jacobian) to Nonlinear Estimation Our contributions: K-NN Regression-Based Estimation Locally Least Squares Estimation

Model Estimation (Uncalibrated)

Visual-Motor Kinematics Model

Global Model

Extending Linear Estimation (Visual-Motor Jacobian) to Nonlinear Estimation

Our contributions:

K-NN Regression-Based Estimation

Locally Least Squares Estimation

Local vs. Global Key idea: using only the previous estimation to estimate the Jacobian RLS with forgetting factor Hosoda and Asada ’94 1 st Rank Broyden update: Jägersand et al. ’97 Exploratory motion: Sutanto et al. ‘98 Quasi-Newton Jacobian estimation of moving object: Piepmeier et al. ‘04 Key idea: using all of the interaction history to estimate the Jacobian Globally-Stable controller design Optimal path planning Local methods don’t!

Key idea: using only the previous estimation to estimate the Jacobian

RLS with forgetting factor Hosoda and Asada ’94

1 st Rank Broyden update: Jägersand et al. ’97

Exploratory motion: Sutanto et al. ‘98

Quasi-Newton Jacobian estimation of moving object: Piepmeier et al. ‘04

Key idea: using all of the interaction history to estimate the Jacobian

Globally-Stable controller design

Optimal path planning

Local methods don’t!

K-NN Regression-based Method q 1 q 2 3 NN q 1 q 2 x 1 ?

(X,q) L ocally L east S quares Method q 1 q 2 x 1 ? K-neighbour(q)

Eye-to-hand Experiments Puma 560 Stereo vision Features: projection of the end-effector’s position on image planes (4-dim) 3 DOF for control

Puma 560

Stereo vision

Features: projection of the end-effector’s position on image planes (4-dim)

3 DOF for control

Measuring the Estimation Error

Global Estimation Error

Visual Task Specification Image Features: Geometric primitives (points, lines, etc.) Higher order image moments Shape parameters … Visual Tasks Point-to-point (point alignment) Point-to-line (colinearity) Point-to-plane (coplanarity) …

Image Features:

Geometric primitives (points, lines, etc.)

Higher order image moments

Shape parameters

…

Visual Tasks

Point-to-point (point alignment)

Point-to-line (colinearity)

Point-to-plane (coplanarity)

…

Eye-in-hand

Eye-in-Hand Experiments

Eye-in-Hand Experiments

Eye-in-Hand Experiments

Eye-in-Hand Experiments

Mean-Squared-Error Task 1 Task 2

Task Errors

Conclusions Reviewed position-based and image-based visual servoing schemes. Presented two global methods to learn the visual-motor function. KNN suffers from the bias in local estimations. LLS (global) works better than the KNN (global) and local updates. The Jacobian of more complex visual tasks can also be learned using LLS method. [email_address]

Reviewed position-based and image-based visual servoing schemes.

Presented two global methods to learn the visual-motor function.

KNN suffers from the bias in local estimations.

LLS (global) works better than the KNN (global) and local updates.

The Jacobian of more complex visual tasks can also be learned using LLS method.

Thank you!

Visual Ambiguity: Single Camera

Visual Ambiguity: Stereo

Speaker: Azad Shademan Ph.D. candidate Department of Computing Sciences University of Alberta, Canada. Title: Uncalibrated Image-Based Robotic Visual Servoing

Read more

Robotics Chapter 6 â Visual Servoing Dr. Amit ... Servoing Architectures Image based Positions are not extracted from image Control directly ...

Read more

visual servoing for nao robot. ... Knowdiff visiting lecturer 140 (Azad Shademan): Uncalibrated Image-Based Robotic Visual Servoing.

Read more

Image-Based Visual Hulls. Wojciech Matusik Chris Buehler Ramesh Raskar Steven Gortler Leonard McMillan. Presentation by: Kenton McHenry. Visual Hull. ...

Read more

Speaker: Azad Shademan Ph.D. candidate Department of Computing Sciences University of Alberta, Canada. Title: Uncalibrated Image-Based Robotic Visual Servoing

Read more

Visual servoing using 2-dof helicopter. Contents:. Quick Review Proportional Controller Results Proportional + Derivative Controller Conclusion Future ...

Read more

View 1950 Azad It posts, presentations, experts, and more. Get the professional knowledge you need on LinkedIn. LinkedIn Home What is LinkedIn? Join Today

Read more

Model Independent Visual Servoing. CMPUT 610 Literature Reading Presentation Zhen Deng. Introduction. Summaries and Comparisons of Traditional Visual ...

Read more

visual servoing for nao robot. visual servoing for nao robot. Docslide.us. Upload Login / Signup. Leadership; Technology; Education; Marketing; Design ...

Read more

## Add a comment