Project 1: Designing Smart Phone Interaction Workarounds
Worth: 15%
Sketch, design, implement and document via a portfolio an animated touch-based and sensor based interface for a mobile phone that is an alternative to 'slide to unlock'.
Note: This exercise and the description below make extremely loose use of particular patents, their meaning, how patent disputes are fought and resolved, whether patents are valid or not, and what designs would actually infringe on a patent. It does not reflect an opinion, nor does it necessarily reflect any reality beyond the fact that these patents exist. It should not be used to suggest the validity or invalidity of particular patents, nor should should it be used as a summary or characterization of what these patents are about. The sole purpose of this description is to create an artificial scenario for a student exercise that promotes design and programming skills.
The recent patent lawsuits between Apple and various other smart phone producers include various disputes over particular interaction techniques. One well-publicized example is based on Apple's slide to unlock patents. The technique described in the patent purports to prevent accidental activation, where an inadvertent touch by a user may unintentionally activate some function on the phone. The basic strategy is that the phone is kept in a locked state, where the user has to perform a particular type of input to unlock it and access the phone functionality.
While there are various versions of this patent and its claims (e.g., the '721 in the US and '022 in Europe), they usually describe how a user makes contact with a graphical 'unlock image' on a hand-held touch-sensitive display, and then moves the unlock image from a pre-defined location either along a pre-defined displayed path or to a predefined unlock region. The first figure illustrates how one patent provides an example of this, while the second figure shows how the Apple iPhone actually realizes slide to unlock.
You will design, implement, document and demonstrate an interface that provides an alternative way to unlock a mobile phone equipped with a multi-touch display and with multiple sensors. In particular, you will
- develop two different techniques, with
- one based on gestural and/or multi-touch input,
- the other based on sensor input (e.g., for one-handed use).
- use animation to provide feedback that the system is recognizing the action you are taking, and how the system then unlocks the device after the action is completed.
- show how the technique avoids both accidental activation and how it also can be used as a (perhaps weak) password
It is up to you to be as creative as possible.
To prepare for this assignment, you will learn techniques on: how to brainstorm and refine ideas via sketching, how to program phones in WPF Silverlight (e.g., how to access sensor and touch input, and how to create rudimentary animations), and how to create a portfolio entry displaying your results.
Pedagogical Objectives
- Apply principles of sketching the lateral thinking -- particularly in terms of how constraints apply to your work here
- Acquire first-time experiences developing for and thinking about application design for hand-held phone devices
Assignment Deliverables and Details .
- 10 competing design sketches, 10 sketch details, and sketch refinements.
- An implemented WPF application. Implement a WPF application that demonstrates how your system works on the phone.
- Web portfolio. To your existing web porflio, include a brief introduction to yourself and (if you prefer) a photo. Make it engaging. Based on feedback from P0, evolve your web portfolio layout and content.
- Visual project summary. Based on feedback from P0, evolve your visual project summary pages. Remember to emphasize the visuals over text - let it tell its own story. Don't feel bound to a static page - you can have things unfold, have sets of pages, animations, overlays, etc. Have impact.
Materials
- an LG touch-sensitive phone running Microsoft Windows (provided)
- lab computers with Visual Studio running Silverlight and the Windows Phone emulator
- access to the above software where you can download it to your own computer
Evaluation
Your grade will be based on your sketches, your design creativity, your implementation, documentation and packaging, and your portfolio summaries. However, great successes or failures, or incompleteness in one exercise criteria will likely affect your total grade. Note that a successful implementation is required: if you cannot demonstrate your system, you will automatically receive a zero. Similarly, missing, late or incomplete work will result in a large grade penalty.
Lateral thinking
seeking to solve problems by unorthodox or apparently illogical methods. It is about moving sideways when working on a problem. It encourages people to apply different perceptions, different concepts and different points of entry to problem-solving.
Your first thoughts will likely be something that is fairly derivative of what you are already familiar with. To go beyond these, think about:
- the many ways you open, unlock, unlatch, and generally change the state of everyday things
- the many ways you signal some change of state via movements, gestures and touch
Look for inspiration from unusual sources. For example,
- As you move about your day to day world, consider what you see and the action you are performing, and ask yourself how it can relate and/or be applied to this exercise. For example, opening a book 'unlocks' it. Would this be a reasonable metaphor to use in a way that people can quickly understand it?
- Google images on gestures. For every one you see, sketch out a possible unlock method, no matter how unlikely. Don't worry about whether you can implement them or not, or if it is any good; you can make decisions on that later.
- Similarly, google images on anything that changes state: open, switch, latch, turn, etc. Let the images inspire you to think in different ways.
- List all the phone's sensor capabilities. Think about how you can use each one's input to unlock the phone, where you consider multiple variations for each one. Then combine sensor input to think about how the fusion of sensor data (perhaps combined with touch data) can unlock the phone.
Schedule
| 9/18 | * Introduction to P1 * Homework: sketch 10 different ideas |
| 9/23 | * Sketch critique * Tutorial: Phone sensors * Homework: Phone sensor exercise * Homework: Refined sketches |
| 9/25 | * Sketch critique * Tutorial: Animation redux * Homework: Scribble sketching * Homework: Camera sampling |
| 9/30 | * Homework: sketches as storyboards |
| 10/2 | * Mini demos and critiques |
| 10/7 | * Homework: sketch vocabulary |
| 10/9 | * Demos & portfolio critique |
The claims of slide to unlock
Claim 1. A computer-implemented method of controlling a portable electronic device comprising a touch-sensitive display (408, 1014), comprising:
- detecting contact with the touch-sensitive display while the device is in a user-interface lock state;
- transitioning the device to a user-interface unlock state if the detected contact corresponds to a predefined gesture;
- and maintaining the device in the user-interface lock state if the detected contact does not correspond to the predefined gesture;
characterized by
- moving an unlock image along a predefined displayed path on the touch-sensitive display in accordance with the contact,
- wherein the unlock image is a graphical, interactive user-interface object with which a user interacts in order to unlock the device.
Claim 3. ... wherein the predefined gesture corresponds to moving the unlock image along the predefined displayed path on the touch-sensitive display to a predefined location on the touch-sensitive display.