Model-driven engineering of user interfaces

50 %
50 %
Information about Model-driven engineering of user interfaces

Published on May 20, 2007

Author: jeanvdd

Source: slideshare.net

Description

This presentation contains the slides of the Doctoral Course given at University of Valencia (Spain) regarding model-driven engineering of user interfaces based on UsiXML (User Interface eXtensible Markup-Language, www.usixml.org), November 2006.

Model-Driven Engineering of User Interfaces Jean Vanderdonckt Université catholique de Louvain (UCL) Louvain School of Management (LSM) Information Systems Unit (ISYS) Belgian Laboratory of Computer-Human Interaction (BCHI) http://www.isys.ucl.ac.be/bchi

Course outline Day 1: Why do we need to model the user interface? H1: Motivations and introduction to usability evaluation of user interfaces Case studies and evaluation H2: Concepts used for usability evaluation for every UI Usability guideline, ergonomic criteria, IFIP properties Day 2: What do we need to model to cover UI aspects? (Part 1) H1: The Cameleon reference framework for multi-target UIs Underlying metamodel Terminology and revisitation of IEEE Taxonomy of RE Task modelling, domain modelling in IdealXML H2: Abstract UI Mapping model Model-to-model transformation in IdealXML

Day 1: Why do we need to model the user interface?

H1: Motivations and introduction to usability evaluation of user interfaces

Case studies and evaluation

H2: Concepts used for usability evaluation for every UI

Usability guideline, ergonomic criteria, IFIP properties

Day 2: What do we need to model to cover UI aspects? (Part 1)

H1: The Cameleon reference framework for multi-target UIs

Underlying metamodel

Terminology and revisitation of IEEE Taxonomy of RE

Task modelling, domain modelling in IdealXML

H2: Abstract UI

Mapping model

Model-to-model transformation in IdealXML

Course outline Day 3: What do we need to model to cover UI aspects? (Part 2) H1: Concrete UI Model-to-model transformation in TransformiXML Model-to-code generation in GrafiXML Graph grammars and other techniques H2: Final UI UI rendering: by interpretation, by compilation UI rendering in multiple computing platforms Day 4: When do we need to model what to cover UI aspects? H1: Multi-path development of UIs Forward, reverse, lateral engineering UI adaptation: graceful degradation H2: Multiple targets Multiple users, platforms, environments

Day 3: What do we need to model to cover UI aspects? (Part 2)

H1: Concrete UI

Model-to-model transformation in TransformiXML

Model-to-code generation in GrafiXML

Graph grammars and other techniques

H2: Final UI

UI rendering: by interpretation, by compilation

UI rendering in multiple computing platforms

Day 4: When do we need to model what to cover UI aspects?

H1: Multi-path development of UIs

Forward, reverse, lateral engineering

UI adaptation: graceful degradation

H2: Multiple targets

Multiple users, platforms, environments

Course outline Day 5: How do we assess the UI modelled? H1: Automated evaluation of guidelines Evolution of code static analysis Evaluation for web and non-web applications H2: Towards genuine model-checking of UIs Usability advisor: natural language evaluation Evaluation of usability guidelines, design rules, properties of interest Final conclusion: evolution of MDA for UIs Low threshold High ceiling Wide walls

Day 5: How do we assess the UI modelled?

H1: Automated evaluation of guidelines

Evolution of code static analysis

Evaluation for web and non-web applications

H2: Towards genuine model-checking of UIs

Usability advisor: natural language evaluation

Evaluation of usability guidelines, design rules, properties of interest

Final conclusion: evolution of MDA for UIs

Low threshold

High ceiling

Wide walls

What is the situation today? Technological aspects of user interfaces progress significantly faster than Software engineering aspects It takes time to develop a user interface with a new device, a new interaction technique It takes more time to develop a toolkit It takes even more time to rely on a model-driven approach Usability engineering aspects New user interfaces are shipped with usability problems because Little or no experience No past, no empirical evidence Empirical experiments require a lot of resource if done carefully Day 1: Why? [Dragicevic et al.,2004]

Technological aspects of user interfaces progress significantly faster than

Software engineering aspects

It takes time to develop a user interface with a new device, a new interaction technique

It takes more time to develop a toolkit

It takes even more time to rely on a model-driven approach

Usability engineering aspects

New user interfaces are shipped with usability problems because

Little or no experience

No past, no empirical evidence

Empirical experiments require a lot of resource if done carefully

Adapted from [Palanque,2002] Target Applications, Domains Notations and tools User Interface Interaction Techniques 2006 TODAY No Interaction Technique Automated, batch systems Nothing Character UIs Business applications Screen definitions Graphical User Interfaces Information systems Entity-relationship Attribute model State-transition diagrams

What’s the situation of today? Interactive Software evolution: context of use =(U,P,E) time Day 1: Why? Platform User Environment Language

Interactive Software evolution: context of use =(U,P,E)

Software evolution Day 1: Why?

Diversity of users Traditional users People with disabilities Visual, motor, cognitive Day 1: Why?

Traditional users

People with disabilities

Visual, motor, cognitive

Variety of tasks Users want to determine their path on each platform Day 1: Why? [Forrester Research,2003]

Users want to determine their path on each platform

Heterogeneousness of platforms Day 1: Why?

Multiplicity of contexts of use Day 1: Why? TV is multi-media family device #1 Family Device Booking notification Everywhere connectivity for simple data exchange Travelling Travel booking site Powerful interface for complex operations Working Multimedia Travel programme Sporting Experience Role Location

Consequence Proliferation of developments Day 1: Why? [Abrams et al.,2001] UI #12 UI #11 UI #10 UI #9 Application 3 UI #8 UI #7 UI #6 UI #5 Application 2 UI #4 UI #3 UI #2 UI #1 Application 1 Platform #4 Platform #3 Platform #2 Platform #1 Application 1 Application 2 Application 3 UI model #1 UI model #2 UI model #3 Platform #1 Platform #2 Platform #3 Platform #4 Platform model #1 Platform model #2 Platform model #3 Platform model #4

Proliferation of developments

Is this web site usable? Day 1: Why?

Ergonomic criteria Criteria for assessing the usability of any UI A priori and/or a posteriori At design and/or evaluation time Structured into 8 first-level criteria Compatibility Consistency Work load Adaptation Dialog control Guidance Error management Implicit order: sorted by importance Day 1: Why? [Scapin & Bastien,1997] [Vanderdonckt,1995]

Criteria for assessing the usability of any UI

A priori and/or a posteriori

At design and/or evaluation time

Structured into 8 first-level criteria

Compatibility

Consistency

Work load

Adaptation

Dialog control

Guidance

Error management

Implicit order: sorted by importance

Navigation Use a reactive image as a navigation map Day 1: Why? No map without any relation Metaphoric map

Use a reactive image as a navigation map

Navigation Include navigation clues Day 1: Why? Navigation bar Landmark Global vs local diagram

Include navigation clues

Navigation Navigation clues should be consistent Day 1: Why? Inconsistencies between the menu and navigation bar Use the same clues at the same location for the same purpose

Navigation clues should be consistent

Navigation No more "Clic here" Day 1: Why? Clic here to go to the Publications page Clic here to go to the LSM page Clic here to go to the UCL page Go to : UCL / LSM / Publications

No more "Clic here"

Navigation Do not use "Return to..."  links Day 1: Why? Aller à :... Button Avoid : come back, return, redo, cancel, undo, redirect Home page Page A Page B Page C Page D Page E First site Page 1 Page 2 Page 3 Page 4 Page 5 Second site

Do not use "Return to..."  links

Presentation Information wich are semantically related should be presented together Day 1: Why? Related label Related label

Information wich are semantically related should be presented together

Presentation Example of a web page format (before) Day 1: Why? Display zone of browser navigation buttons Browser status bar Area for accessing to main commands, langages, map, about, contact User category picture Site menus zone Informational contents Display zone for external links and external applications Organisation logo area Area for accessing to local commands Structure indicator

Example of a web page format (before)

Presentation How to format? Position of home page link, internal links Day 1: Why? [Shaikh & Lenz,2006]

How to format? Position of home page link, internal links

Presentation How to format? Position of search engine, advertisements, about us Day 1: Why? [Shaikh & Lenz,2006]

How to format? Position of search engine, advertisements, about us

Presentation Example of a web page format (after) Day 1: Why? Logo to home page Site menus zone Informational contents External links You are here: structure indictor NL -FR-DL Title, sub-title Contact - Mentions légales – Vie privée – Médiateur - Accessibilité Log in – Site map - Help Search Update – Printer-friendly v.

Example of a web page format (after)

Simple choice Mixed domain: Know domain: Day 1: Why? [Vanderdonckt & Faulkner,2002] Unknown domain : Amount of possible values  [2,3] Amount of possible values  [8,50] Amount of possible values  [4,7] Amount of possible values  ]50,+  [ Amount of possible values  [2,3] Amount of possible values  [4,7] Amount of possible values  [8,50] Amount of possible values  ]50,+  [

Choix multiple Mixed domain: Known domain: Day 1: Why? [Vanderdonckt & Faulkner,2002] Domaine inconnu : Amount of possible values  [4,7] Amount of possible values  [8,50] Amount of possible values  ]50,+  [ Amount of possible values  [2,3] Amount of possible values  [4,7] Amount of possible values  [8,50] Amount of possible values  ]50,+  [ Amount of possible values  [2,3]

Graphics Use graphics for headers Day 1: Why? Use moderately artistic fonts for headers Keep hear in a separate layer Use anti-aliasing

Use graphics for headers

Multimedia elements The background should be appropriate for the task Use various horizon lines depending on the context Day 1: Why? Horizon line Example Meaning Low Brings attention to the sky, the abstract, high values High Stresses immediate results, the concrete Median Suggests balance, equilibrium, peace Diagonal Creates some instability, a feeling of moving Sharp Suggests dynamic aspects, changing, aggressivity

The background should be appropriate for the task

Multimedia elements Low horizon line Day 1: Why?

Low horizon line

Multimedia elements High horizon line Day 1: Why?

High horizon line

Multimedia elements Median horizon line Day 1: Why?

Median horizon line

Multimedia elements Diagonal horizon line Day 1: Why?

Diagonal horizon line

Multimedia elements Sharp horizon line Day 1: Why?

Sharp horizon line

Multimedia elements Do prefer low-intensity, light backgrounds Day 1: Why? No white on black

Do prefer low-intensity, light backgrounds

Multimedia elements Do not use backgrounds with textures Day 1: Why? Pale backgrounds, submit and test Pages with automatically built background

Do not use backgrounds with textures

Keep the good color combinations Multimedia elements Day 1: Why? [Murch,1987] Blue (94%) Black (63%) Red (25%) White (75%) Yellow (63%) Yellow (75%) White (56%) Black (44%) Black (100%) Blue (56%) Red (25%) White (75%) Yellow (63%) Cyan (25%) Blue (69%) Black (56%) Red (37%) Black (63%) White (56%) Blue (44%) Background Thin lines and text White Black Red Green Blue Cyan Magenta Yellow Background Thin lines and text White Black Red Green Blue Cyan Magenta Yellow Red (63%) Blue (63%) Black (56%)

Keep the good color combinations

Keep the good color combinations Multimedia elements Day 1: Why? [Murch,1987] Black (69%) Blue (63%) Red (31%) Yellow (69%) White (50%) Green (25%) Black (50%) Yellow (44%) White (44%) Black (69%) Red (63%) Blue (31%) Yellow (38%) Magenta (31%) Black (31%) Red (56%) Blue (50%) Black (44%) Blue (50%) Black (44%) Yellow (25%) Red (75%) Blue (63%) Black (50%) Background Bold lines and panels White Black Red Green Blue Cyan Magenta Yellow Black (69%) Blue (63%) Red (31%) Yellow (69%) White (50%) Green (25%) Black (50%) Yellow (44%) White (44%) Black (69%) Red (63%) Blue (31%) Yellow (38%) Magenta (31%) Black (31%) Red (56%) Blue (50%) Black (44%) Blue (50%) Black (44%) Yellow (25%) Red (75%) Blue (63%) Black (50%) Background Bold lines and panels White Black Red Green Blue Cyan Magenta Yellow Background Bold lines and panels White Black Red Green Blue Cyan Magenta Yellow

Keep the good color combinations

Avoid the bad color combinations Multimedia elements Day 1: Why? [Murch,1987] Yellow (100%) Cyan (94%) Blue (87%) Red (37%) Magenta (25%) Magenta (81%) Blue (44%) Green (25%) Cyan (81%) Magenta (50%) Yellow (37%) Green (62%) Red (37%) Black (37%) Green (81%) Yellow (75%) White (31%) Green (75%) Red (56%) Cyan (44%) White (81%) Cyan (81%) Background Thin lines and text White Black Red Green Blue Cyan Magenta Yellow Yellow (100%) Cyan (94%) Blue (87%) Red (37%) Magenta (25%) Magenta (81%) Blue (44%) Green (25%) Cyan (81%) Magenta (50%) Yellow (37%) Green (62%) Red (37%) Black (37%) Green (81%) Yellow (75%) White (31%) Green (75%) Red (56%) Cyan (44%) White (81%) Cyan (81%) Yellow (100%) Cyan (94%) Blue (87%) Red (37%) Magenta (25%) Magenta (81%) Blue (44%) Green (25%) Cyan (81%) Magenta (50%) Yellow (37%) Green (62%) Red (37%) Black (37%) Green (81%) Yellow (75%) White (31%) Green (75%) Red (56%) Cyan (44%) White (81%) Cyan (81%) Background Thin lines and text White Black Red Green Blue Cyan Magenta Yellow Background Thin lines and text White Black Red Green Blue Cyan Magenta Yellow

Avoid the bad color combinations

Avoid the bad color combinations Multimedia elements Day 1: Why? [Murch,1987] Yellow (95%) Cyan (75%) Blue (81%) Magenta (31%) Magenta (69%) Blue (50%) Green (37%) Cyan (81%) Magenta (44%) Yellow (44%) Green (44%) Red (31%) Black (31%) Yellow (69%) Green (62%) White (56%) Cyan (81%) Green (69%) Red (44%) White (81%) Cyan (56%) Green (25%) Background Bold lines and panels White Black Red Green Blue Cyan Magenta Yellow Yellow (95%) Cyan (75%) Blue (81%) Magenta (31%) Magenta (69%) Blue (50%) Green (37%) Cyan (81%) Magenta (44%) Yellow (44%) Green (44%) Red (31%) Black (31%) Yellow (69%) Green (62%) White (56%) Cyan (81%) Green (69%) Red (44%) White (81%) Cyan (56%) Green (25%) Background Bold lines and panels White Black Red Green Blue Cyan Magenta Yellow Background Bold lines and panels White Black Red Green Blue Cyan Magenta Yellow

Avoid the bad color combinations

Multimedia elements Consider font variations induced by different platforms Day 1: Why? Fixed size font Label on top of field Same fonts Two browsers

Consider font variations induced by different platforms

IFIP Quality Properties for UIs Ten properties in 3 categories Category 1: information presentation Multiplicity of devices: keyboard, mouse, tablet,… Multiplicity of representations: alternate representations both in input and output, honesty Input/output reusability (use output produced by one action as input for another) Category 2: ordering of task planning Multiplicity of user roles Multiplicity of execution paths: users should be able to be engaged in different tasks simultaneously Non-preemptiveness: degree of freedom for the user to decide what’s next Reachability: possibility to navigate in the system (undo, redo) Observability vs Browsability Category 3: dialog adaptation Reconfigurability: system ability to support user personalization Adaptivity: system ability to support automated adaptation Migrability: system ability to transfer responsibility from one user to another, among users, among users and systems Plasticity: system ability to adapt to the context of use while preserving predefined usability properties Dix’s Principle of Effort Maximility A complex action should be easy to do, but complex to undo Day 1: Why? [Gram & Cockton,1986]

Ten properties in 3 categories

Category 1: information presentation

Multiplicity of devices: keyboard, mouse, tablet,…

Multiplicity of representations: alternate representations both in input and output, honesty

Input/output reusability (use output produced by one action as input for another)

Category 2: ordering of task planning

Multiplicity of user roles

Multiplicity of execution paths: users should be able to be engaged in different tasks simultaneously

Non-preemptiveness: degree of freedom for the user to decide what’s next

Reachability: possibility to navigate in the system (undo, redo)

Observability vs Browsability

Category 3: dialog adaptation

Reconfigurability: system ability to support user personalization

Adaptivity: system ability to support automated adaptation

Migrability: system ability to transfer responsibility from one user to another, among users, among users and systems

Plasticity: system ability to adapt to the context of use while preserving predefined usability properties

Dix’s Principle of Effort Maximility

A complex action should be easy to do, but complex to undo

IFIP Quality Properties for UIs Multiplicity of devices and representations Day 1: Why? [Stephanidis,2001]

Multiplicity of devices and representations

Derivation panel with preview CTTE Operator Operator parameters Design comment resulting from Applying guidelines Design Preview that dynamically changes according to parameters Legends Day 1: Why? [Vanderdonckt et al.,2003]

Plastic UI Example: the Virtual keyboard Day 1: Why? [Grolaux et al.,2001]

Example: the Virtual keyboard

Plastic UI Plastic UI: adaptable bounded value Day 1: Why? [Vanderdonckt et al.,2005]

Plastic UI: adaptable bounded value

Plastic User interface Property of plasticity = adaptation to the context of use while satisfying predefined usability properties of interest Day 1: Why? [Grolaux et al.,2002]

Property of plasticity = adaptation to the context of use while satisfying predefined usability properties of interest

How to address this issue? Capture essence of concepts through models Separation of concerns, Correlability Parsability, editability If possible, human readability Typical models Day 2: What? [Calvary et al.,2003]

Capture essence of concepts through models

Separation of concerns, Correlability

Parsability, editability

If possible, human readability

Typical models

Typical models

What do we want to get? Final user Interface T Concrete user Interface T Task and Domain T Abstract user Interface T T=Target context of use Concrete user Interface S Final user Interface S Task and Domain S Abstract user Interface S S=Source context of use http://www.plasticity.org UsiXML unsupported model UsiXML supported model User S User T Day 2: What? [Calvary et al.,2003] Environment T Reification Abstraction Reflexion Translation Platform S Environment S Platform T

Software engineering interpretation Different types of engineering Forward engineering Reverse engineering Lateral engineering Diagonal engineering (with or without shortcuts) Different approaches Top-down Bottom-up Middle-out Different development paths Example: Round-trip engineering = composition of Reification CUI -> FUI Reflexion FUI -> FUI Abstraction FUI -> CUI Reflexion CUI -> CUI

Different types of engineering

Forward engineering

Reverse engineering

Lateral engineering

Diagonal engineering (with or without shortcuts)

Different approaches

Top-down

Bottom-up

Middle-out

Different development paths

Example: Round-trip engineering = composition of

Reification CUI -> FUI

Reflexion FUI -> FUI

Abstraction FUI -> CUI

Reflexion CUI -> CUI

IEEE Reverse Engineering Taxonomy Reference: Elliot J. Chikofsky , James H. Cross II, Reverse Engineering and Design Recovery: A Taxonomy, IEEE Software, v.7 n.1, p.13-17, January 1990: http://labs.cs.utt.ro/labs/acs/html/resources/ReengineeringTaxonomy.pdf Reengineering "the examination of a subject system to reconstitute it in a new form and the subsequent implementation of the new form.“ Restructuring "a transformation from one form of representation to another at the same relative level of abstraction." The new representation is meant to preserve the semantics and external behavior of the original. Redocumentation "a form of restructuring where the resulting semantically-equivalent representation is an alternate view intended for a human audience.“ Design recovery "a subset of reverse engineering in which domain knowledge, external information, and deduction or fuzzy reasoning are added to the observations of the subject system." The objective of design recovery is to identify meaningful higher-level abstractions beyond those obtained directly by examining the system itself [Chikofsky & Cross,1990]

Reengineering

"the examination of a subject system to reconstitute it in a new form and the subsequent implementation of the new form.“

Restructuring

"a transformation from one form of representation to another at the same relative level of abstraction." The new representation is meant to preserve the semantics and external behavior of the original.

Redocumentation

"a form of restructuring where the resulting semantically-equivalent representation is an alternate view intended for a human audience.“

Design recovery

"a subset of reverse engineering in which domain knowledge, external information, and deduction or fuzzy reasoning are added to the observations of the subject system." The objective of design recovery is to identify meaningful higher-level abstractions beyond those obtained directly by examining the system itself

Revisitation [Bouillon,2006]

Multi-Path Development of UI Multi-path development qualifies a methodology that support the realization of multiples types of development paths withing a single framework Forward engineering Reverse Engineering Adaptation Any Relevant Composition [Limbourg,2004] Development step Development step Development path Development path * 1 Development Scenario Development Scenario * * Development step Development step Development path Development path * 1 Development Scenario Development Scenario * *

Multi-path development qualifies a methodology that support the realization of multiples types of development paths withing a single framework

Our goals Objectives Description of interactive systems Using pre existing domain specific meta-models Used both at design and run time Capitalization Properties Transformations Interactive system Model of the IS Model of the IS ‘ Interactive system ‘ Checks of properties Transformation Models, instances of Meta-Models described in MOF (even for properties and transformations…)

Objectives

Description of interactive systems

Using pre existing domain specific meta-models

Used both at design and run time

Capitalization

Properties

Transformations

UsiXML UsiXML = USer Interface exTensible Markup Language XML-compliant specification language for user interfaces suitable for any interface Web Java Windows-based applications Multimodal applications, 3D applications Virtual, mixed reality applications http://www.usixml.org Join the UsiXML Consortium by registering on line Download the CD image from http://www.usixml.org/index.php?download= UsiXML RelOne.iso

UsiXML =

USer Interface exTensible Markup Language

XML-compliant specification language for user interfaces suitable for any interface

Web

Java

Windows-based applications

Multimodal applications, 3D applications

Virtual, mixed reality applications

http://www.usixml.org

Join the UsiXML Consortium by registering on line

Download the CD image from http://www.usixml.org/index.php?download= UsiXML RelOne.iso

UsiXML = User Interface eXtensible Markup Language What is UsiXML? It is a XML-compliant User Interface Description Language Publicly available from http://www.usixml.org Free to use, open for access, easy to expand Definition of the language UML Class Diagrams UsiXML Reference manual XSD XML Schema Descriptions UsiXML Models

What is UsiXML?

It is a XML-compliant User Interface Description Language

Publicly available from http://www.usixml.org

Free to use, open for access, easy to expand

Definition of the language

Adapted from [Palanque,2002] Target Applications, Domains Notations and tools User Interface Interaction Technique 2006 TODAY 2007 No Interaction Technique Automated, batch systems Nothing Character UIs Business applications Screen definitions Graphical User Interfaces Information systems Entity-relationship Attribute model State-transition diagrams Multi-platform User Interfaces Web, Java applications Task model, context model, UML ,…

The problem To develop user interfaces (UIs) simultaneously for multiple contexts of use A context of use = triple User Computing platform Surrounding environment Organisation Socio-psychological factors [Calvary et al.,2003]

To develop user interfaces (UIs) simultaneously for multiple contexts of use

A context of use = triple

User

Computing platform

Surrounding environment

Organisation

Socio-psychological factors

The problem Why is it difficult? For the designer: Multiplicity of contexts of use No systematic design method For the user: Poor usability of resulting UI UI not adapted to the genuine context of use For the developer: Increase of development and maintenance costs Increasing development complexity Little of no factoring out of common elements

Why is it difficult?

For the designer:

Multiplicity of contexts of use

No systematic design method

For the user:

Poor usability of resulting UI

UI not adapted to the genuine context of use

For the developer:

Increase of development and maintenance costs

Increasing development complexity

Little of no factoring out of common elements

The problem Why should it be systematic? Some consider it noble to have a method Other consider it noble not to have a method Not to have a method is bad But to stop entirely at any method is worse still One should at first observe rules severely, then change them in an intelligent way The aim of possessing method is to seem finally as if one had no method [Lao Tch’ai Tche, many many years ago]

Why should it be systematic?

Some consider it noble to have a method

Other consider it noble not to have a method

Not to have a method is bad

But to stop entirely at any method is worse still

One should at first observe rules severely, then change them in an intelligent way

The aim of possessing method is to seem finally as if one had no method

Mono-platform UI Traditional approach Visual development

Traditional approach

Visual development

Mono-platform UI Advantages of traditional approach UI is graphical by nature A UI prototype can be Rapidly produced Easily modified Visually demonstrated

Advantages of traditional approach

UI is graphical by nature

A UI prototype can be

Rapidly produced

Easily modified

Visually demonstrated

Mono-platform UI Shortcomings of traditional approach No structured method for developing a UI All the knowledge remain implicit Modification can lead to unstructured design Little or no tool support for iterative design Selection of widgets can be inappropriate UI Layout can be tedious, repetitive Problem of the spaghetti of callbacks Early prototyping is hard to achieve and time consuming Limited reusability (throw away)

Shortcomings of traditional approach

No structured method for developing a UI

All the knowledge remain implicit

Modification can lead to unstructured design

Little or no tool support for iterative design

Selection of widgets can be inappropriate

UI Layout can be tedious, repetitive

Problem of the spaghetti of callbacks

Early prototyping is hard to achieve and time consuming

Limited reusability (throw away)

Mono-platform UI Shortcomings of traditional approach Limited use of UI builder [Szekely,1996] Table with dynamic data Gantt chart Direct manipulation Desired interface Obtained interface Menu bar and pull-down menus Toolbar Scrollbars

Shortcomings of traditional approach

Limited use of UI builder

Table with dynamic data

Gantt chart

Direct manipulation

Menu bar and pull-down menus

Toolbar

Scrollbars

Mono-platform UI Interface builders cannot build their own UI [Szekely,1996]

Interface builders cannot build their own UI

Mono-platform UI Any development method (or methodology) is decomposed into 3 axes: Models : explicitly capture knowledge about UI and Interactive Applications with appropriate abstractions Method : structures the definition and use of underlying models in a stage-wise approach Supporting tools : support the use of the method by providing tools for models and their related operations. Ideally, one model should be supported by at least one tool [Bodart,1989]

Any development method (or methodology) is decomposed into 3 axes:

Models : explicitly capture knowledge about UI and Interactive Applications with appropriate abstractions

Method : structures the definition and use of underlying models in a stage-wise approach

Supporting tools : support the use of the method by providing tools for models and their related operations. Ideally, one model should be supported by at least one tool

Mono-platform UI Goal: to integrate all three facets Interface 1 Method Model 1 Model … Model n Models Model Model Model Tools

Goal: to integrate all three facets

MDE based on UsiXML Model to Model Platform Independent Model (PIM) Platform Specific Model (PSM) Model to Code Source code MDA Components Techniques proposed based on UsiXML Computing Independent Model (CIM) Model to Model UsiXML model: Abstract user interface UsiXML model: Concrete user interface Rendering Final user interface UsiXML models: task, domain Graph transformations Graph transformations

CIM Step 1: Task model

New Abstraction: the user’s task Task = set of actions carried out by a user in a given context to reach a goal Logical decomposition of task into sub-tasks Temporal ordering: LOTOS operators (in CTTE) T1 >> T2 Enabling T1[ ]>>T2 Enabling + information passing T1 |> T2 Suspend/resume T1 [ ] T2 Non-deterministic choice T1  T2 Deterministic choice T1 [> T2 Disabling (e.g. Form submit) T1 |=| T2 Independence (any order, but finished) T1* Iteration T1{n} Finite iteration T1 ||| T2 Concurrency T1 |[x]| T2 Concurrency + information passing [T] Optional T Recursion [Markopoulos,1992]

Task = set of actions carried out by a user in a given context to reach a goal

Logical decomposition of task into sub-tasks

Temporal ordering: LOTOS operators (in CTTE)

T1 >> T2 Enabling

T1[ ]>>T2 Enabling + information passing

T1 |> T2 Suspend/resume

T1 [ ] T2 Non-deterministic choice

T1  T2 Deterministic choice

T1 [> T2 Disabling (e.g. Form submit)

T1 |=| T2 Independence (any order, but finished)

T1* Iteration

T1{n} Finite iteration

T1 ||| T2 Concurrency

T1 |[x]| T2 Concurrency + information passing

[T] Optional

T Recursion

New Abstraction: the user’s task Task definition = action + object Action types CRUD pattern: create, read, update, delete Select, control,… Acquire, render, modify, publish, compute, derive,… Object types: Element, list, table, collection, compound,…

Task definition = action + object

Action types

CRUD pattern: create, read, update, delete

Select, control,…

Acquire, render, modify, publish, compute, derive,…

Object types:

Element, list, table, collection, compound,…

New Abstraction: the task meta-model [Limbourg,2004]

CIM Step 2: domain model [Limbourg,2004]

CIM Step 3: Task-domain mappings IdealXML [Limbourg,2004]

New Abstraction: the abstract UI Different CIOs can be used for the same purpose, but with different interaction modalities Definition Abstract Container = set of Abstract Individual Component AIC = abstraction of CIOs of the same type, but independently of any interaction modality Abstract User Interface (AUI) = decomposition into AC+AIC [Vanderdonckt & Bodart,1993]

Different CIOs can be used for the same purpose, but with different interaction modalities

Definition

Abstract Container = set of Abstract Individual Component

AIC = abstraction of CIOs of the same type, but independently of any interaction modality

Abstract User Interface (AUI) = decomposition into AC+AIC

Abstraction: the abstract UI

Abstraction: the abstract UI Notation: based on L. Constantine’s notation for canonical abstract prototypes IdealXML [Montero et al.,2005] [Constantine,2003] Abs.container Abs. component Input Output Navigation Control Select

Notation: based on L. Constantine’s notation for canonical abstract prototypes

IdealXML [Montero,2005]

Example of AUI produced

Mapping the models [Montero,2005] triggers (tg): { , } x updates (up): x observes (ob): x isExecutedIn (ex): x manipulates (ma): { , } x These mappings can be established:

Mapping the models Mapping the models with a mapping model (!!)

Mapping the models with a mapping model (!!)

Your turn now! Virtual Polling System Characterization of a participant Age Gender Region Country Polling Series of questions & answers Each answer may have Simple choice Multiple choices More capabilities…

Virtual Polling System

Characterization of a participant

Age

Gender

Region

Country

Polling

Series of questions & answers

Each answer may have

Simple choice

Multiple choices

More capabilities…

Example solution (first variant)

Example solution (first variant)

Task and domain models [Montero,2005]

Typed Model-to-Model Transformation Uses language Meta-Meta-Model Graph Structure is instance of Meta-Model Our Meta-Model Meta-Model Subset 1 e.g., Task+Domain Model is instance of Meta-Model Subset 2 e.g., Concrete UI Model is instance of Initial UI Model e.g., MyTaskAndDomainModel Resultant UI Model e.g., MyConcreteUIModel Transformation Rule Our transformation catalog [Limbourg,2004]

Expression of models as graphs All transformations are in UsiXML Each model = instance of meta-model Each model = graph as instance of graph type Each model transformation = graph transformation Set of productions

All transformations are in UsiXML

Each model = instance of meta-model

Each model = graph as instance of graph type

Each model transformation =

graph transformation

Set of productions

Definition of a production Find an occurrence of LHS in G (this occurrence is called a match). If several occurrences exist, choose one non-deterministically. Check preconditions of both type PAC and NAC. If not verified, then skip. Remove the part of G which corresponds to (LHS – K), where K is the morphism specified between LHS and RHS. Add RHS – K into G – (LHS – K) as it is given by the corresponding relation between RHS – K and K Check postconditions of both type PAC (and notably that the resulting graph is properly typed) and NAC. If not verified, then undo the transformation rule. LHS G RHS G’ r m m* r* LHS G RHS G’ r m m* r*

Find an occurrence of LHS in G (this occurrence is called a match). If several occurrences exist, choose one non-deterministically.

Check preconditions of both type PAC and NAC. If not verified, then skip.

Remove the part of G which corresponds to (LHS – K), where K is the morphism specified between LHS and RHS.

Add RHS – K into G – (LHS – K) as it is given by the corresponding relation between RHS – K and K

Check postconditions of both type PAC (and notably that the resulting graph is properly typed) and NAC. If not verified, then undo the transformation rule.

Transformation system [Limbourg,2004] G Host USIXM specification G’ Resultant USIXM specification LHS RHS Matches - Co-Matches Is Transformed Into Is Transformed Into Transformation Rule 1 Transformation Rule 2 … Transformation Rule N Transformation System NAC Not Matches +

PIM step: task+domain to AUI Abstract UI (AUI) = UI independent of any interaction modality Definition of AUI structure in terms of Abtract Containers (AC) Which tasks should be logically grouped? Definition of Abstract Individual Components (AIC) types Which « functionnality » should assume AICs and what data do they manipulate ? Definition of spatio-temporal arrangement How should AIC be arranged in space and time ? Definition of dialog control What is the valid flow of action on AICs ? UsiXML model: Abstract user interface UsiXML models: task, domain Graph transformations

Abstract UI (AUI) = UI independent of any interaction modality

Definition of AUI structure in terms of Abtract Containers (AC)

Which tasks should be logically grouped?

Definition of Abstract Individual Components (AIC) types

Which « functionnality » should assume AICs and what data do they manipulate ?

Definition of spatio-temporal arrangement

How should AIC be arranged in space and time ?

Definition of dialog control

What is the valid flow of action on AICs ?

PIM step: task+domain to AUI Identification of AUI structure Spatio-Temporal Arrangement of AIOs Selection of AIC Definition of Abstract Dialog Control STEP : From Task & Domain to AUI SUB-STEPS Derivation of AUI to Domain Relationships

How to read a graph transformation? Node type Node (Attribute,value) Edge type Edge (Attribute,value) Node Edge

Rule 1 LHS RHS ::= Ø

Rule 2 LHS RHS ::=

Rule 3 LHS RHS ::=

Rule 4 LHS RHS ::= NAC

Rule 5 LHS RHS ::= NAC1 NAC2

Rule 6 LHS RHS ::=

Rule 7 LHS RHS ::= PAC “ X < 3000”

Rule 8 LHS RHS ::= PAC “ X > 3000” NAC

Rule 9 LHS RHS ::=

Rule 10 PAC LHS RHS ::= “ X > Y”

PIM sub-step 1: Definition of AUI structure (AC)

PIM sub-step 1: Definition of AUI structure (AC)

Facet type

 

AC Participate to OpinionPoll AC Answer Poll AC Answer Questionnaire AC Answer Question AIC (Input/Single Selection) Select Proposition AIC (Output) See Statistics AIC (Input/Single Selection ) Chose Poll AIC (Control) Validate Question AIC (Output) Provide Personal Data AIC (Output) Read Question AIC (Input/Single Selection) Chose Questionnaire

PIM sub-step 2: definition of AIC types AC = Abstract Container AIC = Abstract Individual Component CIC = Concrete Interaction Component

PIM sub-step 3: Definition of spatio-temporal arrangement

PSM Step: AUI to CUI Concrete UI (CUI) = UI independent of toolkit Definition of CUI structure Which AIC is a window? Definition of Concrete Interaction Component (CIC) type Which « widget » should represent which AIC ? Definition of placement What layout can be specified between CICs,… Definition of navigation Which container can be started or closed from which container? Definition of dialog control What is the valid flow of action on AIOs UsiXML model: Abstract user interface UsiXML model: Concrete user interface UsiXML models: task, domain Graph transformations Graph transformations

Concrete UI (CUI) = UI independent of toolkit

Definition of CUI structure

Which AIC is a window?

Definition of Concrete Interaction Component (CIC) type

Which « widget » should represent which AIC ?

Definition of placement

What layout can be specified between CICs,…

Definition of navigation

Which container can be started or closed from which container?

Definition of dialog control

What is the valid flow of action on AIOs

PSM Step: AUI to CUI Reification of AC into CC Arrangement of CICs Selection of CIC Concrete Dialog Control Definition STEP : From AUI to CUI SUB-STEPS Definition of Navigation Derivation of CUI to Domain Relationships

PSM sub-step 3: definition of navigation An example of a complex rule

PSM: Concrete User Interface

Example: Platform adaptation widget substitution (1) <?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot; standalone=&quot;yes&quot;?> < cuiModel name =&quot; MyModel &quot;> < version modifDate =&quot; 2004-03-24T17:09:17.402+01:00 &quot; xmlns =&quot;&quot;> 7 </ version > < authorName xmlns =&quot;&quot;> Youri </ authorName > < window height =&quot; 500 &quot; width =&quot; 600 &quot; name =&quot; Formulaire (2/5) &quot; id =&quot; window_1 &quot;> < box relativeHeight =&quot; 100 &quot; name =&quot; box1_0 &quot; id =&quot; box1_0 &quot;> < box type =&quot; vert &quot; name =&quot; boxTodo &quot; id =&quot; boxTodo &quot;> ... < box type =&quot; horiz &quot; name =&quot; box_2_2_2_1 &quot; id =&quot; box_2_2_2_1 &quot;> < textComponent defaultContent =&quot; Sexe &quot; isBold =&quot; true &quot; id =&quot; label_2 &quot;/> < radioButton groupName =“ grupo01 &quot; defaultContent =&quot; Femme &quot; defaultState =&quot; false &quot; id =&quot; radiobutton_0 &quot;/> < radioButton groupName =&quot; grupo01 &quot; defaultContent =&quot; Homme &quot; defaultState =&quot; true &quot; id =&quot; radiobutton_1 &quot;/> </ box > ... </ box > </ box > </ window > </ cuiModel > Excerpt for a UsiXML CUI specification

Example: widget substitution (2)

Example: widget substitution (3) The UsiXML graph before applying any rule

Example: widget substitution (4) LHS RHS NAC Rule 1: Create a new comboBox with the same id and name as the name of the group of radioButtons.

Example: widget substitution (5) Rule 1: Create a new comboBox with the same id and name as the name of the group of radioButtons. The UsiXML graph after applying the first rule

Example: widget substitution (6) LHS RHS ::= Rule 2: Convert every radio button within the group “x” into an item for the comboBox “x” that we have just created thanks to Rule 1

Example: widget substitution (7) Rule 2: Convert every radioButton within the group “x” into an item for the comboBox “x”, we have just created. The UsiXML graph after applying the second rule: radio buttons disappeared

Example: widget substitution (8) <?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot; standalone=&quot;yes&quot;?> < cuiModel name =&quot; MyModel &quot;> < version modifDate =&quot; 2004-03-24T17:09:17.402+01:00 &quot; xmlns =&quot;&quot;> 7 </ version > < authorName xmlns =&quot;&quot;> Youri </ authorName > < window height =&quot; 500 &quot; width =&quot; 600 &quot; name =&quot; Formulaire (2/5) &quot; id =&quot; window_1 &quot;> < box relativeHeight =&quot; 100 &quot; name =&quot; box1_0 &quot; id =&quot; box1_0 &quot;> < box type =&quot; vert &quot; name =&quot; boxTodo &quot; id =&quot; boxTodo &quot;> ... < box type =&quot; horiz &quot; name =&quot; box_2_2_2_1 &quot; id =&quot; box_2_2_2_1 &quot;> < textComponent defaultContent =&quot; Sexe &quot; isBold =&quot; true &quot; id =&quot; label_2 &quot;/> < comboBox id =&quot; comboBox001 &quot; name =&quot; label_3 &quot; isDropDown =&quot; true &quot;> < item id =&quot; radiobutton_0 &quot; name =&quot; radiobutton_0 &quot; defaultContent =&quot; Femme &quot;/> < item id =&quot; radiobutton_1 &quot; name =&quot; radiobutton_1 &quot; defaultContent =&quot; Homme &quot;/> </ comboBox > ... </ box > </ box > </ window > </ cuiModel > Excerpt from the final transformated UsiXML specification

Example: widget substitution (9)

What do we have so far? Concrete user Interface S Final user Interface S Task and Domain S Abstract user Interface S S=Source context of use UsiXML unsupported model UsiXML supported model User S Reification Platform S Environment S

Multiple development paths

Mapping the Methodological World onto the Transformation World A development library stores (in usiXML textual format) paths, steps and sub-steps definition and their associated transformation systems and transformation rules Development step Development step Development sub - step Development sub - step Development path Development path Transformation System Transformation System Transformation Rule Transformation Rule isComposedOf isRealizedBy isComposedOf isComposedOf * 1 * 1 1 * 1 0..1 Development step Development step Development sub - step Development sub - step Development path Development path Transformation System Transformation System Transformation Rule Transformation Rule isComposedOf isRealizedBy isComposedOf isComposedOf * 1 * 1 1 * 1 0..1 Methodological World Development step Development step Development sub - step Development sub - step Development path Development path Transformation System Transformation System Transformation Rule Transformation Rule isComposedOf isRealizedBy isComposedOf isComposedOf * 1 * 1 1 * 1 0..1 Transformation World [Limbourg,2004]

Multiple development paths Rule n Transformation System 1 Rule 1 Rule 2 … Rule n Transformation System 2 Rule 1 Rule 2 … Rule n Transformation System ... Rule 1 Rule 2 … Rule n Transformation System n Rule 1 Rule 2 … : when source terminates apply target : execute development step Development Step α

TransformiXML API/GUI API = set of transformations

API = set of transformations

From T&D to AUI TransformiXML TransformiXML [Bouillon et al.,2005]

TransformiXML

Final user interface Two forms of UI rendering Interpretation By run-time static analysis and direct rendering (InterpiXML & FormiXML) Code generation By program synthesis (GrafiXML) By generative programming (Angie) Feature model Components assembling UsiXML model: Abstract user interface UsiXML model: Concrete user interface Rendering Final user interface UsiXML models: task, domain Graph transformations Graph transformations Generative programming

Two forms of UI rendering

Interpretation

By run-time static analysis and direct rendering (InterpiXML & FormiXML)

Code generation

By program synthesis (GrafiXML)

By generative programming (Angie)

Feature model

Components assembling

What is a Feature Model? C F1 C F1 F2 C F1 F2 Dependencies 1. 2. Optional Exclusive Alternate [Schlee & Vanderdonckt,2004]

Example of a Feature Model Alternate [Schlee & Vanderdonckt,2004]

Interpretation of a feature model C F1 F2 F3 F2a F2b F3a F3b C F1 F2 F3 F2a F2b F3a F3b 0 1 1 1 1 0 0 UsiXML Specifications <C> <F1>0</F1> <F2>1 <F2a>1</F2a> <F2b>0</F2b> </F2> <F3>1 <F3a>0</F3a> <F3b>1</F3b> </F3> </C> ABA ANGIE-Part [Schlee & Vanderdonckt,2004]

Generative Programming void channelFftSpectrum(T*, T*, T*, Pl* ppl = NULL); void channelRange (T*, Pl* ppl = NULL ); void nlinFilterCommon (const int& nID, Mask*, Pl*); int pot (const unsigned int& ) const; ImgProcessT<T>& channelXor (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelMix (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSub (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelAdd (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSubBalanced(T*, T*, Pl* ppl = NULL); ImgProcessT<T>& chann elMix ImgProcessT<T> & channelSub (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelAdd (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSubBalanced(T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelMix (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSub (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelAdd (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSubBalanced(T*, T*, Pl* ppl = NULL); V oid channelFftSpectrum(T*, T*, T*, Pl* ppl = NULL); void channelRange (T*, Pl* ppl = NULL ); void nlinFilterCommon (const int& nID, Mask*, Pl*); int pot (const unsigned int& ) const; ImgProcessT<T>& channelXor (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelMix (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSub (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelAdd (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSubBalanced(T*, T*, Pl* ppl = NULL); public: ImgProcessT(const char* const psz = &quot;&quot;, Pl* ppl = NULL); ImgProcessT(const int&, const int& nWidth = 20, const int& nHeight = 20, const int& nFillColor = 0, ProgressLine* ppl = NULL ); ImgProcessT(const ImgProcessT<T>&); virtual ~ImgProcessT(); virtual char* getClassName() const; virtual char getClassID () const; ImgProcessT<T>& operator+=(const ImgProcessT<T>&); ImgProcessT<T>& operator-=(const ImgProcessT<T>&); ImgProcessT<T>& operator^=(const ImgProcessT<T>&); ImgProcessT<T> operator+ (const ImgProcessT<T>&); ImgProcessT<T> operator- (const ImgProcessT<T>&); ImgProcessT<T> operator^ (const ImgProcessT<T>&); ImgProcessT<T>& channelNLinFilter(const int&, Mask*, T*, Pl* ppl=NULL); mgProcessT<T>& operator+=(const ImgProcessT<T>&); ImgProcessT<T>& operator-=(const ImgProcessT<T>&); ImgProcessT<T>& operator^=(const ImgProcessT<T>&); ImgProcessT<T> operator+ (const ImgProcessT<T>&); ImgProcessT<T> operator- (const ImgProcessT<T>&); ImgProcessT<T> operator^ (const ImgProcessT<T>&); ImgProcessT<T>& channelNLinFilter(const int&, Mask*, T*, Pl* ppl=NULL); void channelFftSpectrum(T*, T*, T*, Pl* ppl = NULL); void channelRange (T*, Pl* ppl = NULL ); void nlinFilterCommon (const int& nID, Mask*, Pl*); int pot (const unsigned int& ) const; ImgProcessT<T>& channelXor (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelMix (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSub (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelAdd (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSubBalanced(T*, T*, Pl* ppl = NULL); USIXML Specifications ImgProcessT<T>& channelMix (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSub (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelAdd (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSubBalanced(T*, T*, Pl* ppl = NULL) ImgProcessT<T>& channelMix (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSub (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelAdd (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSubBalanced(T*, T*, Pl* ppl = NULL) Components void areaOperation(const int&, const int&, const int&, const int&, ImgProcessT<T>& (ImgProcessT<T>::*pimgpr)(const int&, Pl*), const int& nAngle, Pl* ppl ); void areaOperation(const int&, const int&, const int&, const int&, ImgProcessT<T>& (ImgProcessT<T>::*pimgpr)(Pl*), Pl* ppl ); void areaOperation(const int&, const int&, const int&, const int&, ImgProcessT<T>& (ImgProcessT<T>::*pimgpr)(const int&, const int&, const int&, const int& ), const int&, const int&, const int&, const int& ); void channelShowHistogram( int*, const int&, const int&, const int&, const int&, const int&, const int&, const int& n = 0 ) const; void channelFft ( T*, T*, T*, const unsigned int&, const unsigned int&, Pl* ppl = NULL ); void channelIfft ( T*, T*, T*, const unsigned int&, const unsigned int&, Pl* ppl = NULL ); void channelFftSpectrum(T*, T*, T*, Pl* ppl = NULL); void channelRange (T*, Pl* ppl = NULL ); void nlinFilterCommon (const int& nID, Mask*, Pl*); int pot (const unsigned int& ) const; ImgProcessT<T>& channelXor (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelMix (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSub (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelAdd (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSubBalanced(T*, T*, Pl* ppl = NULL); public: ImgProcessT(const char* const psz = &quot;&quot;, Pl* ppl = NULL); ImgProcessT(const int&, const int& nWidth = 20, const int& nHeight = 20, const int& nFillColor = 0, ProgressLine* ppl = NULL ); ImgProcessT(const ImgProcessT<T>&); virtual ~ImgProcessT(); virtual char* getClassName() const; virtual char getClassID () const; ImgProcessT<T>& operator+=(const ImgProcessT<T>&); I Final code ImgProcessT<T>& channelAdd (T*, T*, Pl* ppl = NULL); ImgProcessT<T>& channelSubBalanced(T*, T*, Pl* ppl = NULL); public: ImgProcessT(const char* const psz = &quot;&quot;, Pl* ppl = NULL); ImgProcessT(const int&, const int& nWidth = 20, const int& nHeight = 20, const int& nFillColor = 0, ProgressLine* ppl = NULL ); void channelFftSpectrum(T*, T*, T*, Pl* ppl = NULL); void channelRange (T*, Pl* ppl = NULL ); void channelFftSpectrum(T*, T*, T*, Pl* ppl = NULL); void channelRange (T*, Pl* ppl = NULL ); void channelFftSpectrum(T*, T*, T*, Pl* ppl = NULL); void channelRange (T*, Pl* ppl = NULL ); void channelFftSpectrum(T*, T*, T*, Pl* ppl = NULL); void channelRange (T*, Pl* ppl = NULL ); [Schlee & Vanderdonckt,2004]

Developed by Benjamin Michotte GrafiXML consists of a user interface builder for designing a graphical user interface (GUI) independently of the platform and save it in UsiXML format language. Features Exports GUI in Java source (using Swing) XHTML 1.0 Strict XUL Works on Windows, Linux and MacOs X Available in English, French and Spanish Based on plugins Export Import Project management Graceful degradation of user interfaces Allows creating context models [Michotte,2004]

GrafiXML consists of a user interface builder for designing a graphical user interface (GUI) independently of the platform and save it in UsiXML format language.

Features

Exports GUI in

Java source (using Swing)

XHTML 1.0 Strict

XUL

Works on Windows, Linux and MacOs X

Available in English, French and Spanish

Based on plugins

Export

Import

Project management

Graceful degradation of user interfaces

Allows creating context models

Design Tab Design window Components toolbar Components options

Localisation of UIs GrafiXML allows the user to create multi-language GUI Support for mnemonics and shortcuts

GrafiXML allows the user to create multi-language GUI

Preview At any time, you can preview the UI in the language you want

At any time, you can preview the UI in the language you want

XML Editor GrafiXML contains a XML editor which shows the UsiXML specification of your work You can edit yourself some part of the XML Edit the UsiXML Show attributes A click on the tree highlights the corresponding UsiXML

GrafiXML contains a XML editor which shows the UsiXML specification of your work

You can edit yourself some part of the XML

Plugins GrafiXML works with plugins Install / remove using the plug-in manager Updated if needed using one click Click on « add » to open the downloader Choose the plugins you want install And install them Double-click on a plugin And look at the plugin informations

GrafiXML works with plugins

Install / remove using the plug-in manager

Updated if needed using one click

What do we have so far? Concrete user Interface S Final user Interface S Task and Domain S Abstract user Interface S S=Source context of use UsiXML unsupported model UsiXML supported model User S Reification Platform S Environment S

Reverse engineering From FUI to CUI From CUI to AUI, task & domain

From FUI to CUI

From CUI to AUI, task & domain

From FUI to CUI Do you have the source code? Markup languages (e.g., HTML): static code analysis (ReversiXML) Do you have the compiled code? Programming languages (e.g., compiled C): resource file extraction and static code analysis Do you have the running application? Video analysis: computer vision Do you have only the documentation? Image analysis: image processing [Bouillon & Vanderdonckt,2004]

Do you have the source code?

Markup languages (e.g., HTML): static code analysis (ReversiXML)

Do you have the compiled code?

Programming languages (e.g., compiled C): resource file extraction and static code analysis

Do you have the running application?

Video analysis: computer vision

Do you have only the documentation?

Image analysis: image processing

From compiled code [Marion,2005]

The calculator Begin VB.Form Calculator BorderStyle = 1 'Fixed Single Caption = &quot;Calculator&quot; ClientHeight = 2970 ClientLeft = 2580 ClientTop = 1485 ClientWidth = 3270 ClipControls = 0 'False BeginProperty Font name = &quot;System&quot; charset = 1 weight = 700 size = 9.75 underline = 0 'False italic = 0 'False strikethrough = 0 'False EndProperty Height = 3375 Icon = &quot;CALC.frx&quot;:0000 Left = 2520 LinkMode = 1 'Source LinkTopic = &quot;Form1&quot; MaxButton = 0 'False ScaleHeight = 2970 ScaleWidth = 3270 Top = 1140 Width = 3390 Begin VB.CommandButton Number Caption = &quot;7&quot; Height = 480 Index = 7 Left = 120 <Window id=“1” name=“1” isVisible=“true” isEnabled=“true” defaultBorderTitle=“Calculator” borderWidth=“1”height=“358” width=“309”> <Box id=“2” name=“2” type =“verticall” isVisible=“true” isEnabled=“true”> <button isEnabled=“true” … USIXML [Marion,2005]

Begin VB.Form Calculator BorderStyle = 1 'Fixed Single Caption = &quot;Calculator&quot; ClientHeight = 2970 ClientLeft = 2580 ClientTop = 1485 ClientWidth = 3270 ClipControls = 0 'False BeginProperty Font name = &quot;System&quot; charset = 1 weight = 700 size = 9.75 underline = 0 'False italic = 0 'False strikethrough = 0 'False EndProperty Height = 3375 Icon = &quot;CALC.frx&quot;:0000 Left = 2520 LinkMode = 1 'Source LinkTopic = &quot;Form1&quot; MaxButton = 0 'False ScaleHeight = 2970 ScaleWidth = 3270 Top = 1140 Width = 3390 Begin VB.CommandButton Number Caption = &quot;7&quot; Height = 480 Index = 7 Left = 120

From CUI to AUI, task & domain Re-engineering = reverse + forward Possible re-interpretation by QtkiXML

Re-engineering = reverse + forward

Possible re-interpretation by QtkiXML

Example of Derivation Rules Examples: For a textbox in HTML/WML  x  Ts : x = input ٨ (x.type=“text” ٧ x.type=“password” ٧ x.type=NULL)  Addnode (“textComponent”, idtext) where idtext= NodeAmount(Tt) For a table in HTML/WML  x  Ts : x = table ٨ x.border>0  Addnode (“table”, idtable) where idtable = NodeAmount(Tt)   x  Ts : x = table ٨ x.border=0  Addnode (“box”, idbox) where idbox = NodeAmount(Tt)  AddAttribute (idbox, “type”, “vertical”) Input Name=in1 Maxlength=15 Value=login Type=text Element B TextComponent Name=in1 Maxlength=15 defaultContent=login Id=in1 Element Y [Bouillon,2006]

Examples:

For a textbox in HTML/WML

 x  Ts : x = input ٨ (x.type=“text” ٧ x.type=“password” ٧ x.type=NULL)  Addnode (“textComponent”, idtext)

where idtext= NodeAmount(Tt)

For a table in HTML/WML

 x  Ts : x = table ٨ x.border>0

 Addnode (“table”, idtable) where idtable = NodeAmount(Tt)

  x  Ts : x = table ٨ x.border=0  Addnode (“box”, idbox) where idbox = NodeAmount(Tt)

 AddAttribute (idbox, “type”, “vertical”)

Derivation rules categories Rules can be classified into different categories, outlining the common issues in a RE process for various source UIs. Element recovery Attribute detection Layout / temporal ordering analysis Dialog recuperation Hierarchy recovery Multi-model transformations Retargeting operations  x  to Tt i ,y  to Tt 0 : x = window ٨ (y=box ٧ y=window) ٨ x.filename =y.targetfile  CloneNode(x.id, idnew, Tt0) where idnew =∑ node  Tt 0 ٨ RemoveNode(x, x.id) ٨ RemoveArc(ParentNode(x).id, x.id) ٨ z=root(Tt i ) ٨ Remove Node(z,z.id) ٨ AddArc(y.id, idnew)  x  to Tt i ,y  to Tt 0 : x = vocalGroup ٨ y=VocalGroup ٨ x.filename =y.insertFile  CloneNode(x.id, idnew, Tt0) where idnew =∑ node  Tt0 ٨ RemoveNode(x, x.id) ٨ RemoveArc(ParentNode(x).id, x.id) ٨ z=root(Tt i ) ٨ Remove Node(z,z.id) ٨ AddArc(y.id, idnew) Multi-model transformations Element 1 Element 2 Element 3 Horizontal box Horizontal box Vertical box Element 1 Element 2 Line Break Element 3  x  Tt: x=bix ٨ ((rigthSibling(x)!=table ٧ rigthSibling(x)!=bix ٧ rigthSibling(x)!=cell ٧ rigthSibling(x)!=box) ٨ rigthSibling(x)!=NULL)  CloneNode (rightSibling(x).id, idnew) ٨   RemoveNode (rightSibling(x), rightSibling(x).id) ٨ RemoveArc (ParentNode(rightSibling(x)), rightSibling(x)) ٨ AddArc(x.id, idnew) where idnew =∑ node  Tt Layout Analysis

Rules can be classified into different categories, outlining the common issues in a RE process for various source UIs.

Element recovery

Attribute detection

Layout / temporal ordering analysis

Dialog recuperation

Hierarchy recovery

Multi-model transformations

Retargeting operations

RE of Web Pages: ReversiXML Written in PHP5 On-line RE About 12,000 LOC Set of libraries

Written in PHP5

On-line RE

About 12,000 LOC

Set of libraries

RE of Phone Interfaces Major working hypotheses Static RE of WML 1.1 and voiceXML 2.0 up to the CUI in USIXML 1.4.6 Recovery of the layout and hierarchy/temporal ordering. Dialog-Navigation analyzed (but not scripts) No retargeting operations Similar method to HTML applied for WML & VoiceXML reverse engineering Description of languages meta-models and derivation rules.

Major working hypotheses

Static RE of WML 1.1 and voiceXML 2.0 up to the CUI in USIXML 1.4.6

Recovery of the layout and hierarchy/temporal ordering.

Dialog-Navigation analyzed (but not scripts)

No retargeting operations

Similar method to HTML applied for WML & VoiceXML reverse engineering

Description of languages meta-models and derivation rules.

RE of WML Phone Interfaces Example Prototype using XSLT + XPATH Local process allowing no design alternatives WML <p> Yahoo! ID: <input name=&quot;login&quot; value=&quot;&quot; format=&quot;*m&quot; />Password: <input type=&quot;password&quot; name= &quot;passwd&quot; value=&quot;&quot; format=&quot;*m&quot; /> <anchor title=&quot;OK&quot;>Submit <go method=&quot;post&quot; href=&quot;/raw?&quot;> USIXML <textComponent id= &quot;textComponent_1&quot; glueHorizontal =&quot;left“ isVisible=&quot;true&quot; isE

Add a comment

Related presentations

Related pages

Towards Model Driven Engineering of Plastic User Interfaces

2. MDE OF RIGID USER INTERFACES Before addressing the specific requirements for plasticity, let's see how Model Driven Engineering can be used for the ...
Read more

Model-Driven Engineering of User Interfaces: Promises ...

Model-Driven Engineering of User Interfaces: Promises, Successes, Failures, and Challenges Jean Vanderdonckt Belgian Laboratory of Computer-Human ...
Read more

Model-Driven Engineering of Multi-Target Plastic User ...

Model-Driven Engineering of Multi-Target Plastic User Interfaces Benoît Collignon1, Jean Vanderdonckt1, Gaëlle Calvary2 1Université catholique de ...
Read more

Model-Driven Engineering of Multi-target Plastic User ...

A Multi-target user interface is composed of a series of interconnected variations of the same user interfaces, but tailored for different targets or ...
Read more

Model-Driven Engineering of Workflow User Interfaces ...

User interface hand coded design. UI derivation from a workflow specification has been used on commercial tools , even though the UI is still manually ...
Read more

Model-Driven Engineering of Workflow User Interfaces ...

A model-driven engineering method is presented that provides designers with methodological guidance on how to systematically derive user interfaces of ...
Read more

Model-Driven Engineering of Workflow User Interfaces ...

Model-Driven Engineering of Workflow User Interfaces Josefina Guerrero García, Christophe Lemaigre, Jean Vanderdonckt, Juan Manuel González Calleros ...
Read more

The Beautification Process in Model-Driven Engineering of ...

The Beautification Process in Model-Driven Engineering of User Interfaces Inés Pederiva1, Jean Vanderdonckt1,2, Sergio España1, Ignacio Panach1
Read more

Model-driven Engineering of user interfaces based on UsiXML

Model-driven engineering (MDE) of user interfaces consists of describing a user interface and aspects involved in it (e.g., task, domain, context of use ...
Read more