Gesture's Hard Coded

The gesture's have been hard coded in Finally.

Each gesture cariable (x, y, z, Roll, Pitch, Orientaiton) split in to 5 values. Each value is the average of a the rasw data. First 4 each the quartiles of the data values that are over/under the breakpoint. The 5th value isn't dependant on the breakpoint, but it's size is set at the same as the of quartiles. It's most useful for working out final orientation when the remote is held still (since the pitch and roll values aren't accurate while in motion. Turns out the easiest gestures to recognise are the mode gesture such as the picture and video mode since their orientation is really different from anything else.

The hard coded values are based on two things 1 experience (Doing the gesture over and over and over...) and looking for the recurrent relationships between the vaules. And 2 an excel spreadsheet with a whole bunch of pretty graphs. Excel used as a vidual aid for the gestures. Through not entirely useful because of the amount of data being compared. Data works out to be 5 values per 6 variables for 15 gestures. 450 values.


The gestures still need to classified further to determine the decision order and/or stringer or weaker restraints. So glad I added the buttons to it just in case I fall face first during the demo because a lot of the gesture rely on smooth fluid motion not "oh crap it's not working and I'm standing in front of a room full of people" type motion. Can already say that the system is going to break down because of the nature of recognition technology and that more feedback is required.

Planning to Start using around the house this weekend. Finally I'll have a remote back. All it's taken is 9 months, lots of stress and WAY to much redbull.

Fully functioning

It's early but all functionality is in Wiida. Well almost. I implemented the buttons for testing purposes and the program runs with full capabilities using them.

Control of Winamp, Window's Media Player, and Windows Picture and Fax viewer all run smoothly. Task list restricted to basically iPod controls (Play, Pause, ff, rewnd, skip, Vol, Zoom, Rnd, Repeat). I;'m pretty sure I'll implement tv control but probly not for the thesis since it doesn't add anything and theirs no way to demo it.

The Mode is controled 2 ways. A charades version to select a specific mode and a mode control switch to define either 1 mode or two. If running video the mode is considered to be singular.

Work that needs to be done:
1. Get the gesture breakpoints and settings
This will take a while but so thought should solve it. I think I'll try doing it the opposite way I've been doing it. Start with all gestures and remove gestures that don't work. While testing I've been playing around with the simple math (ie try using x, try using x *10, try using x * x) for the values. Show some interesting things about some of the gestures. For example Stop is quick and sharp whereas go is a lot more flowing.

2. Get some "randomly selected users" round for drinks and work out the best ways to implement the mode and discuss how it would be represented on the Wiimote. (got the lights to represent mode)


On Document Side of things:
Sat down for couple hour last night and went through acm and itee for papers again. Rippied a bunch but need to go through them. In Regard to Modality. The Wiidia is a "multidevice" system not a multimodal system.


Stephen pretty sure I'll forget but do I include the building experience into the Thesis? Certain programs where invaluable while making this. Not just media play application but programs like Spy++.

Abstract Version I

Getting really stressed about the actual build (the gesture recognizer is killing me) and limiting the scope. Not implementing the menu display unless I have time. Need to have a chat about this tomorrow.

Here a version of the abstract below. Blue bits just some guidelines for me.
Question relating to it:

1. Not sure about the tense needed. Written this if future because I assume it's based on the reader reading it BEFORE the document
2. Conclusion before the document is written. ?

------------------8<-----------------8<------------------

Wiidia – A Gestural Media Interface

Abstract:

Purpose:
To develop a 3 dimensional control interface for systems that are controlled either by 1 or 2 dimensional devices. The aim is control of multiple media system which can run concurrently using a ‘Nintendo Wii Remote Control’.

Scope
The control device is restricted to the device’s gestural control abilities. The single dimensional capacities of the remote are implemented for control device comparison.

The systems included are media player applications. Functional control for the interface is limited to tasks considered common and popular to the media systems.

The interface includes the ability to control multiple media systems concurrently. The interface itself is multimodal, whicl the systems it controls are to be considered as single mode. The interface provides a solution to the problems associated with mode error whi9ch arises from this.

Method

The control method is reached through analysis of current input devices related to the systems, the capabilities of the selected controller and the issues related to gestural interfaces.

The gestures are developed and explained through examination of gestural notations, movement analysis of proposed gestures and a gestural review with users.

Reviews of current and popular media player interfaces are performed and include examination of their menu structures and the functionality provided. Furthermore, important functionality related to media player is examined through user interviews. The results of these reviews and interviews are developed into the working functionality provided by the system.

Results

Recommendation

Conclusions

The interface provides a unique control method for media player control. The interface provides a fun and intuitive way of controlling media devices, expert users will find the interface only useful in specific contexts.

Quick Reivew

Been coding 24/7 on this lately and been getting nowhere fast. Been a streesed because of the lack iof progress .

Major updates are :
Changed the way the wiimote reads values, using the different between consecutive values now instead of difference from calibrated values. Finally worked oout how to get the class name for other windows. This is a follow on from desideing to use other programs and not the skin.

Now I have to use this window name and add the second/third/forth app to the program. Once I get the second app done the abstract will be up on the here. Withpout the second app the modal portion is missing.

-SRE

Quick Abstrc:

Using Wiimote to control media on a computer.
Not using the butons, only gestures.

The goal is to give most person the wiimote and they should be able to control a media plater without assistance for the most commonly used tasks.
The application helps the resolve modal error.

Furthermore other gestures should be easily understood and memorable.

Designed for multiple applications, though not implemented.
Design for mutplie control metods, through npot impemented (functionality for wiimote buttons is there, but not mapped to any thing)

Hit a wall

Not sure where to go now. The two possible ways are;

1. To write a skin and plug-in for Media portal.
Been skinning the last few days (learning xml and c#). Realized that media portal doesn't have a decent SDK to get to the players themselves.

To get to them I'm sure that I'm going to have to make use of the GPL and use their own code for the implementations for their media players. I'm don't think this is plagiarism due to the GPL . I know it technically doesn't show any academic merit but I'm not building a media player I'm building a way to control it.

2. Treat the remote as a keyboard with custom keys (I.E. volume button, play pause)
This allows a more global approach, for example right now I'm wanting to reach for the Wiimote to turn the sound up. Implications of this is letting windows do the work and giving the playlist creation controls to Windows Explorer.

These arbitrary gesture, i think, more reflect the spirit of this project of controlling the 2D space with 3D control and pretty much already exist from previous development cycles (there are arbitrary gestures in the skinned version)


To do this the design change needed is one gesture to represent enter the volume level selection (currently it clashed with Audio mode)

Well that didn't take long

Little less stessed. Turns out friends drinks to much and can't anymore. The project has almost caught up to the hackt version. Just a matter of determining the best Break levels. I.E. when it starts and when high. low and no values are recorded.
Now for the Gestures

Not at Ipswich

Stephen,

You won't have seen me at the lecture today. I've stayed in Bris. Left a msg on facebook for you with the reason.

In respect to the thesis,

Quiet;y starting to freak out myself. Bugs lately have been eating me alive. Their places in the code which are obfuscated to the nth degree, hence I just can't sit down and start. It's been taking about half hour to remember the place I was at. On top of that Internet is running at snails pace.

Things remaining:
Thesis
(Done by end of Holidays, 8 days remaining)
1. Parse the Input correctly (2 days)
Normalized values are driving me nuts. I had to convert the raw values into a float[] held by a generic gesture. Hence the normalized valued is an array of lengths like 150+ beiung spilt into 35ish average values. Long story short, slow to work out which values are actually being used. Not enough time now to convert the program back to the wii specific version.

2. Calculate the gesture vals based on the Parsed input (Dependant on 1.) (1 day)
Once the normalized values are working this is just going through the motions. Perform a gesture 20 times, see the identifying values. Hardcore the values. Perform each gesture look for common clashes. If clash occurs, restrict/increase the identifying values.
Ideally (I.e. not enough time left) get other people to perform the gestures and uysed their mannerisms as the identifiers.

3. Create the skin for media portal (2 days)
Exegesis in working through tutorials on the web. Languages is c#. This may cause problems, but the workaround if it does is explained below in 4.

4. Throw the gesture calls to the media portal skin. (1 days)
Ideally the calls are from c++ (Wiidia) to c#(Wiidia skin). Plan to get done through external calls to WindowCommand calls. Failing this Media Portal allows Keyboard mappings to be allocated, this is the work around by having media C++ throw event handlers for the logical keyboard mapping. I think this keyboard mapping lends itself to a nice comparison in the thesis.

Assessment
1. Poster (14 days remain)
Thinking background pictures of the program superimposed with explanations of the design reasoning and features will be a good layout. Invites trouble with logical flow to the poster through.

2. Abstract (14 days remain)
Going to get a version of this up here really soon. Begin with a version of the elevator pitch. This is dependent on the skin for the program. Ultimately the skin shows the features.

3. Thesis Document (32 days remain)
Drawn out from abstract. Really need to start writing.

On a personal note,
The bugs in the program are really starting to stress me out. Work is killing me, getting as many shifts as the full-timer, but I'm think I have a solution for that. Sooooo many group assignments for uni as well, I always stress when others are involved and 4 group assignments is just insane.
Hopefully, reason for starting in Bris today will be nothing, and I get the input to parse correctly before work tonight.

-SRE