50 %
50 %
Information about wobbrockDevices

Published on November 28, 2007

Author: Gabrielle

Source: authorstream.com

Lecture 17: Development of and for Devices:  Lecture 17: Development of and for Devices Jake Wobbrock 05-830 Advanced User Interface Software Not a Proper Toolkit Lecture:  Not a Proper Toolkit Lecture This lecture might have been about UI toolkits for devices There’s just one problem There really aren’t any At least, not like we think of toolkits for desktop software Instead we will talk about “things that help you develop for devices.” And other interesting device issues… Overview:  Overview Introduce work in multiple semi-related areas of software development of and for devices Palm OS Pocket PC / Windows CE .NET Compact Framework J2ME Waba BREW WAP and WML DirectX Overview cont.:  Overview cont. Highlight HCI research projects with and for devices Phidgets Rapid prototyping for physical devices (physical widgets) Pebbles Project Interoperation of handhelds and PCs Handheld Web browsing (various efforts) WebThumb, Power Browser, The Gateway, M-Links Motivation: Handhelds:  Motivation: Handhelds Cell phones and PDAs have seen huge growth in use over recent years. “Last year [2002] there was one cell phone sold for every 15 human beings on the planet … Worldwide mobile phone unit sales totaled 423.4M in 2002, a 6% increase from 2001.” “From 1999 to 2000 the number of PDAs sold nearly doubled, from some 3.6M units to 6.9M… By 2004, the number of units is expected to top 33.5M.” Source: Biz Journals Motivation cont.: Gaming:  Motivation cont.: Gaming Nintendo GameBoy NTT DoCoMo Inc. handhelds running Java Others by Qualcomm, L.M. Ericsson, Motorola, Siemens … Convergence: Games for phones and PDAs, not only custom handhelds Source: CNN The Punch Line:  The Punch Line PDAs, phones, other handhelds experiencing great growth in use Development community needs to have good tools and toolkits What is available today? Sadly, not much (at least compared to what’s available for the desktop) Palm OS:  Palm OS by PalmSource Acer AlphaSmart Fossil Garmin Handspring HandEra Kyocera Palm Samsung Sony Symbol Palm OS Visit http://www.palmsource.com/products/ Palm OS Facts:  Palm OS Facts As of January 2002: 16M Palm-powered PDAs shipped (to date) 7000 applications 150,000 registered developers Growing at 1000 per month Source: Palm OS Programming, O’Reilly Palm OS Development:  Palm OS Development Two main ways CodeWarrior for Palm OS PRC-Tools (GCC) Other ways Sun KVM and J2ME IBM VisualAge Micro Edition (Windows or Linux) Jump (use Java, compiles to Motorola 68K) Waba (subset of Java, stripped-down VM) PocketStudio (Pascal-based, aimed at Delphi users) Pocket C (develop on the handheld for the handheld) Assembler SDK (ASDK) – write directly for Motorola 68K Pocket Smalltalk (includes garbage-collecting VM) CodeWarrior for Palm OS:  CodeWarrior for Palm OS Special line of Metrowerks CodeWarrior releases for Palm OS development Three key components Editor Constructor Palm OS Emulator (POSE) Write code in C/C++ Downloaded to Palm via HotSync™ Editor:  Editor Constructor:  Constructor Emulator (POSE):  Emulator (POSE) Runs on the desktop Emulates from a “very low level” Designed to simulate speed despite running on a desktop Useful for debugging Not useful for interactions requiring pen-dexterity Technical Details:  Technical Details Palm OS apps center around event cascade Catch events by defining handlers and handling them But not nearly as much framework as Visual Basic or MFC… more like Win32 Often involves large switch statements Single-threaded, single-process The Event Cascade:  The Event Cascade CodeWarrior Summary:  CodeWarrior Summary Generally the best option Originally for Macintosh, then for Windows! BUT… Constructor is often not as WYSIWYG or Direct Manipulation as you’d like (e.g., text fields, scroll bars) It lacks options for all programmatically-settable options that affect visual appearance of controls (e.g., button borders) Emulator doesn’t work exactly like the device Sometimes this matters, often this doesn’t PRC-Tools (GCC):  PRC-Tools (GCC) Alternate means for Palm OS development Developed before CodeWarrior ran on Windows by the Free Software Foundation Collection of tools for Unix or Windows Gnu C Compiler (GCC) for Motorola 68K Build-PRC (combine binaries into *.prc file) Gnu Debugger (GDB) PilRC (compiles resource files) Falch IDE combines these with a UI Pocket PC? Windows CE?:  Pocket PC? Windows CE? Pocket PC Not considered (strictly-speaking) a PDA Or at least not marketed that way Meant to be a full-fledged PC but pocket-sized Refers to a device (platform) Also to a set of standard applications common on all Pocket PC devices, esp. Pocket PC Shell Windows CE (WinCE) Refers to just the OS that runs on Pocket PCs Has large number of capabilities (components) for the Pocket PC developer to choose from Current version is WinCE .NET Pocket PCs vs. Palm OS Devices:  Pocket PCs vs. Palm OS Devices “Basic PDAs allow you to store and retrieve addresses and phone numbers, maintain a calendar, and create to-do lists and notes. More sophisticated PDAs can run word processing, spreadsheet, money manager, games and electronic book reading programs and also provide email and Internet access.” Source: Microsoft Pocket PCs:  Pocket PCs Toshiba Compaq iPaq HP Jornada Audiovox T-Mobile Dell Axim ViewSonic Razor Zayo Cassio Cassiopeia NEC MobilePro Pocket PC Facts:  Pocket PC Facts Projected 2003 sales between 14M – 17M devices Compare to PC sales of 135M – 138M Pocket PC projected to catch up by 2008! Sales hit 10M after just 10 months on the market Introduced April 2000 In May 2001, held 26% of market. Rest held by Palm OS devices. Source: C|Net The Shrunken Desktop:  The Shrunken Desktop Pocket Word Pocket Excel Pocket Outlook Pocket IE Pocket Windows Media Player … ?? Pocket PC Development:  Pocket PC Development eMbedded Visual Tools 3.0 (VC++, VB) Largely just like Windows programming Can even use MFC for VC++ Compiles for many different processors Challenge:  Challenge Problem: Portability Not all Pocket PCs utilize the same WinCE components So not all Pocket PC apps will run on all Pocket PCs Despite same underlying OS! Example: The menu bar at the bottom is specific to Pocket PCs, not WinCE Created and passed to the Pocket PC shell Developers often have to change much UI code for different Pocket PC platforms Example: Handheld PC (H/PC) is wide and short, so tall dialogs have to be re-laid-out Resource Editor:  Resource Editor Pocket PC Summary:  Pocket PC Summary Powerful palmtop computers More complicated than Palm OS Devices (e.g., can be multi-threaded) Less integrated vertically than Palm OS devices Complicates development (e.g., UI components not often supported on all Pocket PC devices) With more power and capability comes more complexity and complication Affects the toolkits for development on such devices .NET Compact Framework:  .NET Compact Framework Subset of .NET framework for desktops Not PDA-specific but targeted to any “smart devices”: PDAs, mobile phones, set-top boxes, automobiles, etc. Delivered as extensions to Visual Studio .NET called “Smart Device Programmability” Tools and programming model are same as for desktop .NET platform Easy transition from .NET desktop development to .NET compact framework development .NET Compact Framework cont.:  .NET Compact Framework cont. Argument is that there are currently too many mobile CPUs and OSes, so development is difficult True – but we know the Microsoft answer .NET would allow integration of devices across multiple platforms, networks, and programming languages A vision behind .NET in general Compact Framework is a subset of all .NET capabilities Reduces footprint on resource-strained devices J2ME – Java 2 Micro Edition:  J2ME – Java 2 Micro Edition Uses subset of Java Virtual Machine for smaller footprint (i.e., KVM) Highly optimized runtime environment Not PDA-specific: smart cards, pagers, mobile phones, set-top boxes, vehicle telematics systems (e.g., OnStar system) Set of APIs defined by the Java Community Process Program™ Includes user interface, security, networking protocols, and more J2ME Architecture:  J2ME Architecture J2ME Architecture defines… Configurations: A virtual machine and a minimal set of class libraries For Palm OS, configuration is “Connected Limited Device Configuration” (CLDC) Profiles: Higher-level APIs that further define the application, user interface, and device-specific properties For Palm OS, profiles are PDA and “Mobile Information Device Profile” (MIDP) J2ME Architecture cont.:  J2ME Architecture cont. From http://java.sun.com/j2me/j2me-ds.pdf Waba:  Waba Subset of Java Stripped-down VM: small and fairly fast No longs, doubles, exceptions, or threads Open source: http://www.wabasoft.com/ Waba defines… A language (strict subset of Java) A virtual machine (Waba VM) A class file format (strict subset of Java bytecode) A set of foundation classes BREW:  BREW Qualcomm’s Binary Runtime Environment for Wireless (BREW) Like J2ME, provides application execution environment Goes farther! Provides business model for certifying, downloading, and charging premium content With J2ME, this is left entirely to the marketers and developers to figure out for themselves Based on C++ (so it is fast, and small binaries) WAP and WML:  WAP and WML WAP (Wireless Application Protocol) Provides “flat” Web content on handhelds Requires constant connectivity by the device Lacks a rich interactive experience for end-users WML (Wireless Markup Language) Specifies content and user interface for WAP-delivered information Based on XML: so describes data, not just presentation of data WMLScript WAP:WML:WMLScript :: HTTP:HTML:JavaScript Provides only minimal interactivity WAP and WML cont.:  WAP and WML cont. Four things specified by WML Text and image presentation and layout, including a variety of formatting commands Deck/card organizational metaphors for information (similar to HyperCard) Inter-card navigation and linking Card parameterization and state management The fate of WAP Hasn’t been the success its advocates hoped New push to make possible highly interactive content J2ME, BREW, .netCF, Waba, but these all send a VM and increase footprint Game Development: DirectX:  Game Development: DirectX DirectX: Advanced suite of multimedia APIs DirectDraw, Direct3D, DirectSound, DirectMusic, DirectInput, DirectPlay, DirectShow… Allows accessing of special hardware capabilities (e.g., graphics and sound cards) without requiring hardware-specific code DirectX exists for Pocket PC Handheld gaming projected to be a big new market opportunity Again, similar to programming for desktop Slide38:  Transition from (Mainly) Industrial Tools to Research Initiatives for and with Devices Phidgets (Greenberg et al.):  Phidgets (Greenberg et al.) Phidgets are “physical widgets” Play the same role in physical UIs as widgets do in graphical UIs Package IO Hide implementation details, provide abstraction Have additional characteristics Require a connection manager A link between the software and hardware parts A simulation mode in software only Phidget Architecture:  Phidget Architecture Physical Device Packaged physical unit to be used by the designer. Most phidgets are built around a circuit board. Wire Protocol Communication protocol between device and host computer. Not visible to end-programmers. Phidget Manager COM object that has event-based API for end-programmers for connection management. Phidget-specific COM objects Created by the Phidget Manager whenever a device is seen. They correspond 1:1 to physical devices. IGlabPhidget interface Generic interface which all phidgets have. End-programmers use it for basic identification and capabilities. Phidget-specific interface Capabilities specific to a particular phidget. Extends the basic IGlabPhidget interface. Phidget ActiveX controls Wraps up the phidget code and provides an on-screen user interface and the ability to simulate the phidget as an control. Pebbles (Myers et al.):  Pebbles (Myers et al.) Research project exploring the interoperation of handhelds and PCs Handhelds can be an aspect of the user, PCs can be a fixture of the environment Users may move from environment to environment, PCs may stay behind Handhelds can control other devices too Personal Universal Controller Handhelds can be used to control a PC by people with motor impairments A Current Research Issue:  A Current Research Issue Handheld User Interfaces for Web Browsing Challenges Constrained screen-space No mouse cursor (e.g., no link roll-overs) Pages designed on a desktop for display on a desktop Scrolling more difficult Graphic refresh can be slow, scrolling blurry Handheld Web Browsing:  Handheld Web Browsing The Gateway (MacKay) Pages displayed as thumbnails expand as certain parts of them are interacted with (CHI 2003). WebThumb (Wobbrock et al.) Exploration of interaction techniques for better handheld browsing (UIST 2002). M-Links (Trevor et al.) Uses modes to separate action from navigation on mobile phone Web pages (UIST 2001). Power Browser (Buyukkokten et al.) Generates summary views of handheld web content for display (CHI 2000). Conclusions:  Conclusions Toolkits for development for devices are far from Amulet-level sophistication Many processors and devices exist – complicates development and toolkit design Numerous standards and niche technologies Minimizing footprint on resource-constrained devices is a key constraint HCI research can tackle issues beyond the specifics of device hardware and software e.g., interaction techniques for PC/PDA interoperability e.g., interaction techniques for browsing

Add a comment

Related presentations

Related pages

Lecture 17: Development of and for Devices | Many PPT

Lecture 17: Development of and for Devices 1 . Lecture 17: Development of and for Devices . Jake Wobbrock 05-830 Advanced User Interface
Read more

pgtprod - Ace Recommendation Platform - 1

wobbrockDevices nword processing,spreadsheet,moneymanager, gamesandelectronicbookreading programsandalsoprovideemailand Internetaccess. Source:Microsoft ;
Read more

Human-Computer Interaction in eCommerce

Lecture 17: Development of and for Devices Jake Wobbrock 05-830 Advanced User Interface Software
Read more