Information about Galilean Differential Geometry Of Moving Images

Published on May 18, 2008

Author: danielfagerstrom

Source: slideshare.net

Presented at ECCV2004 (as a poster) and at Institut Mittag-Leffler 2003.

Differential Structure of Movies How can we describe the local structure of an image sequence? We will assume that a movie is a smooth function of 2+1 dimensional space-time Looking for generic properties

How can we describe the local structure of an image sequence?

We will assume that a movie is a smooth function of 2+1 dimensional space-time

Looking for generic properties

Approaches for Motion Analysis Optical flow Spatio-temporal texture Spatio-temporal differential invariants

Optical flow

Spatio-temporal texture

Spatio-temporal differential invariants

Optical Flow Geometry of the projected motion of particles in the observers field of view Binding hypothesis needed to use it on image sequences “ Top down” Local formulation, but non-local, due to binding hypothesis Undefined when particles appears and disappears, e.g. motion boundaries Does not use image structure

Geometry of the projected motion of particles in the observers field of view

Binding hypothesis needed to use it on image sequences

“ Top down”

Local formulation, but non-local, due to binding hypothesis

Undefined when particles appears and disappears, e.g. motion boundaries

Does not use image structure

Spatio-Temporal Differential Invariants Local geometry of spatio-temporal images ” Bottom Up”, low level No binding hypotheses, connection to the environment considered as a higher level problem Well defined everywhere Does use image structure, extension of low level vision for still images

Local geometry of spatio-temporal images

” Bottom Up”, low level

No binding hypotheses, connection to the environment considered as a higher level problem

Well defined everywhere

Does use image structure, extension of low level vision for still images

Overview Galilean geometry Moving frames Image geometry 1+1 dimensional Galilean differential invariants 2+1 dimensional Galilean differential invariants What is required for a more realistic movie model

Galilean geometry

Moving frames

Image geometry

1+1 dimensional Galilean differential invariants

2+1 dimensional Galilean differential invariants

What is required for a more realistic movie model

Galilean Geometry Spatial and temporal translation a , spatial rotation R and spatio-temporal shear v Shear x t x t

Spatial and temporal translation a , spatial rotation R and spatio-temporal shear v

Galilean Geometry Insensitive to constant relative motion for parallel projection, approximately otherwise Simplest meaning full model Assumed implicitly when one talk about optical flow invariants: div, rot, dev, i.e. first order flow Shape properties from the environment can be derived from relative motion (Newton physics describe Galilean invariants)

Insensitive to constant relative motion for parallel projection, approximately otherwise

Simplest meaning full model

Assumed implicitly when one talk about optical flow invariants: div, rot, dev, i.e. first order flow

Shape properties from the environment can be derived from relative motion

(Newton physics describe Galilean invariants)

Galilean Invariants Planes of simultaneity (constant t ) are invariant and has Euclidean geometry: distances and angles are invariants i.e. an image sequence The temporal distance between planes of simultaneity is an invariant

Planes of simultaneity (constant t ) are invariant and has Euclidean geometry: distances and angles are invariants

i.e. an image sequence

The temporal distance between planes of simultaneity is an invariant

Galilean ON-System An n+1 dimensional Galilean ON-system (e 1 ,e 2 , ,e 0 ) is s.t. (e 1 ,e 2 , ,e n ) is an Euclidean ON-system and ||e 0 || T =1

An n+1 dimensional Galilean ON-system (e 1 ,e 2 , ,e 0 ) is s.t. (e 1 ,e 2 , ,e n ) is an Euclidean ON-system and ||e 0 || T =1

Moving Frames Galilean geometry has no metric We will use Cartan's method of moving frames, that does not require a metric Moving frame: e:M ! G ½ GL(n) Attach a frame that is adapted to the local structure in each point Differential geometry: the local change of the frame: de

Galilean geometry has no metric

We will use Cartan's method of moving frames, that does not require a metric

Moving frame: e:M ! G ½ GL(n)

Attach a frame that is adapted to the local structure in each point

Differential geometry: the local change of the frame: de

Moving Frames C(A) contains the differential geometric invariants expressed in the global frame i

Image Geometry Image space: E 2 I - trivial fiber bundle with Euclidian base space and log intensity as fiber (Koenderink 02) z=f(x,y) f is smooth Image geometry Global gray level transformations Lightness gradients

Image space: E 2 I - trivial fiber bundle with Euclidian base space and log intensity as fiber (Koenderink 02) z=f(x,y)

f is smooth

Image geometry

Global gray level transformations

Lightness gradients

Gradient Gauge For points where r f 0 we can choose an adapted ON-frame { u , v } s.t. f u =0 All functions over i u j v f, i+j ¸ 1 becomes invariants w.r.t. rotation in space and translation in intensity

For points where r f 0 we can choose an adapted ON-frame { u , v } s.t. f u =0

All functions over i u j v f, i+j ¸ 1 becomes invariants w.r.t. rotation in space and translation in intensity

Gray Level Invariants

Hessian Gauge For points where r f 0 we can choose an ON-frame { p , q } s.t. f pq =0 and |f pp |>|f qq | All functions over i p j q f, i+j ¸ 2 becomes invariants w.r.t. rotation in space, translation in intensity and addition of a linear light gradient (Koenderink 02)

For points where r f 0 we can choose an ON-frame { p , q } s.t. f pq =0 and |f pp |>|f qq |

All functions over i p j q f, i+j ¸ 2 becomes invariants w.r.t. rotation in space, translation in intensity and addition of a linear light gradient (Koenderink 02)

Galilean 1+1 D Two cases: Isophotes cut the spatial line, motion according to the constant brightness assumption Isophotes are tangent to the spatial line (along curves), creation, annihilation

Two cases:

Isophotes cut the spatial line, motion according to the constant brightness assumption

Isophotes are tangent to the spatial line (along curves), creation, annihilation

Tangent Gauge Let { t , x } be a global Galilean ON-frame, for points where f x 0 we can define an adapted Galilean ON-frame { s , x } s.t. f s =0.

Let { t , x } be a global Galilean ON-frame, for points where f x 0 we can define an adapted Galilean ON-frame { s , x } s.t. f s =0.

Isophote Invariants

Hessian Gauge Let { t , x } be a global Galilean ON-frame, for points where f xx 0 we can define an adapted Galilean ON-frame { r , x } s.t. f rx =0 .

Let { t , x } be a global Galilean ON-frame, for points where f xx 0 we can define an adapted Galilean ON-frame { r , x } s.t. f rx =0 .

Hessian Invariants

Galilean 2+1 D General case Also here are two different main cases Isophote surfaces transversal to the spatial plane. Motion of isophote curves in the image Isophote surfaces tangent to the plane. Creation, annihilation and saddle points

General case

Also here are two different main cases

Isophote surfaces transversal to the spatial plane. Motion of isophote curves in the image

Isophote surfaces tangent to the plane. Creation, annihilation and saddle points

Invariants in the General Case a u , a v - acceleration u , v - divergence u , v - skew of the ”flow field” - rotation of the plane in the temporal direction u , v - flow line curvature in the plane

a u , a v - acceleration

u , v - divergence

u , v - skew of the ”flow field”

- rotation of the plane in the temporal direction

u , v - flow line curvature in the plane

More Descriptive Invariants D - rate of strain tensor for the spatio-temporal part of the frame field a, curl D, div D, def D - are flow field invariants a , , , u , v - are not flow field invariants

D - rate of strain tensor for the spatio-temporal part of the frame field

a, curl D, div D, def D - are flow field invariants

a , , , u , v - are not flow field invariants

Tangent Gauge Let { t , x , y } be a global Galilean ON-frame, for points where ||{ f x ,f y }|| 0 we can define an adapted Galilean ON-frame { s , u , v } s.t. f s =f u =f su =0 Principal acceleration extrema Direction of u constant along s – used in Guichard (98)

Let { t , x , y } be a global Galilean ON-frame, for points where ||{ f x ,f y }|| 0 we can define an adapted Galilean ON-frame { s , u , v } s.t. f s =f u =f su =0

Principal acceleration extrema

Direction of u constant along s – used in Guichard (98)

Tangent Gauge

Hessian Gauge Let { t , x , y } be a global Galilean ON-frame, we define an adapted Galilean ON-frame { r , p , q } s.t. f pq = f rp = f rq =0 . Also defined when the spatial tangent disappears, e.g. for creation and disappearance of structure r is the same vector field as when the optical flow constraint equation is solved for the spatial image gradient

Let { t , x , y } be a global Galilean ON-frame, we define an adapted Galilean ON-frame { r , p , q } s.t. f pq = f rp = f rq =0 .

Also defined when the spatial tangent disappears, e.g. for creation and disappearance of structure

r is the same vector field as when the optical flow constraint equation is solved for the spatial image gradient

Hessian Gauge

Real Image Sequences Localized filters are not invariant w.r.t. Galilean shear, velocity adapted ( s – directed) filters are needed More generic singular cases for imprecise measurements

Localized filters are not invariant w.r.t. Galilean shear, velocity adapted ( s – directed) filters are needed

More generic singular cases for imprecise measurements

Conclusion Theory about differential invariants for smooth Galilean spatio-temporal image sequences Local operators “ Bottom up” Contains more information about the image sequence than optical flow Extension of methods for still images

Theory about differential invariants for smooth Galilean spatio-temporal image sequences

Local operators

“ Bottom up”

Contains more information about the image sequence than optical flow

Extension of methods for still images

Abstract. In this paper we develop a systematic theory about local structure of moving images in terms of Galilean differential invariants. We argue that ...

Read more

In this paper we develop a systematic theory about local structure of moving images in terms of Galilean differential invariants. We argue that Galilean ...

Read more

Galilean Differential Geometry of Moving Images on ResearchGate, the professional network for scientists.

Read more

The set of Galilean invariants for moving images also contains the Euclidean invariants for ... Galilean Differential Geometry of Moving Images. November 2016.

Read more

Galilean Differential Geometry of Moving Images. ... theory about local structure of moving images in terms of Galilean differential ...

Read more

... about local structure of moving images in terms of Galilean differential ... Galilean differential geometry of moving ... Fagerström, Daniel

Read more

A morphological, affine, and galilean ... Galilean Differential Geometry of Moving ... structure of moving images in terms of Galilean ...

Read more

... Prague, Czech Republic, May 11-14, 2004 ... Morphological Operations on Matrix-Valued Images. ... Galilean Differential Geometry of Moving ...

Read more

In this paper we develop a systematic theory about local structure of moving images in terms of Galilean differential invariants. We argue that Galilean ...

Read more

## Add a comment