Tracking Your Fingers with the Wiimote

Using an LED array and some reflective tape, you can use the infrared camera in the Wii remote to track objects, like your fingers, in 2D space. This lets you interact with your computer simply by waving your hands in the air similar to the interaction seen in the movie "Minority Report". The Wiimote can track upto 4 points simultaneously. The multipoint grid software is a custom C# DirectX program.

Software
To run the grid program you see in the video:
1. First, follow this walkthrough on using the wiimote with C#. You may need to download a copy of Visual C# Express to compile/run this sample if you don't have it yet.
2. Download a copy of the DirectX SDK. You may not need this to simply run the sample grid program, but you will need it if you want to make any changes to it.
3. Download the Wiimote Multipoint Grid sample program. Make sure your wiimote is connected via bluetooth, and then run the ".exe" shortcut in the main folder.

A visit to this project's FAQ and Advanced Discussion post may be very englightening. You may also find the official discussion forums for my wiimote projects helpful: WiimoteProject.com

Low-Cost Multi-point Interactive Whiteboards Using the Wiimote

Since the Wiimote can track sources of infrared (IR) light, you can track pens that have an IR led in the tip. By pointing a wiimote at a projection screen or LCD display, you can create very low-cost interactive whiteboards or tablet displays. Since the Wiimote can track upto 4 points, up to 4 pens can be used. It also works great with rear-projected displays.

Software
The calibration and mouse cursor emulation software is available for you to download and try yourself. Note: My mouse emulation code isn't perfect. If any of you are programmers and can get it working with Alias Sketchbook, drop me a line.
1. Connect your wiimote to your PC via Bluetooth. There are a number of tutorials online on how to do this, possibly even for you specific software/hardware configuration. The Wiimote works with many (but not all) Bluetooth drivers. You can report/read about compatibility issues at WiimoteProject.com
2. Download the Wiimote Whiteboard software to the right. Please read the "READ ME.txt" file first! Make sure your wiimote is connected via Bluetooth, and then run the ".exe" in the main folder. NOTE: Good placement of the wiimote is key to good tracking. View the README for more info.

Multitouch: The multitouch demos are custom C# DirectX programs. You may download the sample program to the right, but this is provided for developers without support or documentation. The code is built on top of this Wiimote library. Unfortunately, multi-touch capable applications are currently extremely rare. Hopefully, that will change as more developers explore its potential.

Building pens: Here is a simple schematic of the light pen. The LEDs that I use are Vishay TSAL6400s running at 100mA, but lots of other LEDs will work too. You also might be able to jump start your experimentation by retro-fitting a mini keychain light with an IR LED. I'm currently looking into manufacturing and selling IR pens, but this may take several months.

Mac/Linux Versions: Due to personal time contraints, I probably won't be able to make a port myself. But fairly mature versions are available online. However, I haven't tried them myself. I've also created a Source Forge Project - Wiimote Whiteboard, but it does not seem to be getting much love.

A visit to this project's FAQ and Advanced Discussion post may be very englightening. You may also find the official discussion forums for my wiimote projects helpful: WiimoteProject.com

Downloads:
Windows (32-bit)
Wiimote Whiteboard* (original 12/19/07) - includes source
Wiimote Whiteboard v0.2* (updated 3/27/08) - includes source
Wiimote Whiteboard v0.3* (updated 8/20/08) - includes source

* there appear to be some issues with 64-bit machines, and BlueSoleil. Alternative Bluetooth software may work. Check the forum for more info.

Mac (Java)
You can do a quick webserach or try the version created by uweschmidtwhich is a fairly mature Java-based version for Mac.

Multitouch:source code

Fun software (mostly free) to try with this:
Mapping software: Microsoft Virtual Earth, Google Earth
Physics simulation software: Phun
Virtual Tourism: Photosynth
Media Browsing: CoolIris
Painting progams: Alias Sketchbook Pro
Note taking:One Note
Handwritting recognition: Windows Tablet PC Edition - works great if you have it, not a stand alone program to install.

Head Tracking for Desktop VR Displays using the Wii Remote

Using the infrared camera in the Wii remote and a head mounted sensor bar (two IR LEDs), you can accurately track the location of your head and render view dependent images on the screen. This effectively transforms your display into a portal to a virtual environment. The display properly reacts to head and body movement as if it were a real window creating a realistic illusion of depth and space.

The program only needs to know your display size and the size of your sensor bar. The software is a custom C# DirectX program and is primarily provided as sample code for developers without support or additional documentation. You may need the most recent version of DirectX installed for this to work.

Software
To run the DesktopVR program you see in the video:
1. Connect your wiimote to your PC via Bluetooth. If you don't know how to do this, you can follow this tutorial. I've been told it works with other Bluetooth drivers, but I have not tested them myself.
2. Download the WiiDesktopVR (v02) sample program. Read the README file on program usage and configuration. Launch the "WiiDesktopVR.exe" in the main folder. A potentially more stable/Vista/64-bit compatible version has been created by Andrea Leganza. There also may be more variants on the web.

NOTE: If you are having trouble with running the program, you can check myproject blog post about it or check the forum for assistance. I am unable to replicate these problems, so it hard for me to debug them. But, other people have figured it out. Things that have been identified to help: delete the "config.dat" file and re-run the program, install a new version of Direct X, or istall .NET 2.0.

Developers Notes: The code is built on top of this Wiimote library. To compile the program, you will need a C# IDE and the DirectX SDK. More notes are in the README.

A visit to this project's FAQ and Advanced Discussion post may be very englightening. You may also find the official discussion forums for my wiimote projects helpful: WiimoteProject.com

Tracking Your Fingers with the Wiimote

Using an LED array and some reflective tape, you can use the infrared camera in the Wii remote to track objects, like your fingers, in 2D space. This lets you interact with your computer simply by waving your hands in the air similar to the interaction seen in the movie "Minority Report". The Wiimote can track upto 4 points simultaneously. The multipoint grid software is a custom C# DirectX program.

Software
To run the grid program you see in the video:
1. First, follow this walkthrough on using the wiimote with C#. You may need to download a copy of Visual C# Express to compile/run this sample if you don't have it yet.
2. Download a copy of the DirectX SDK. You may not need this to simply run the sample grid program, but you will need it if you want to make any changes to it.
3. Download the Wiimote Multipoint Grid sample program. Make sure your wiimote is connected via bluetooth, and then run the ".exe" shortcut in the main folder.

A visit to this project's FAQ and Advanced Discussion post may be very englightening. You may also find the official discussion forums for my wiimote projects helpful: WiimoteProject.com

Low-Cost Multi-point Interactive Whiteboards Using the Wiimote

Since the Wiimote can track sources of infrared (IR) light, you can track pens that have an IR led in the tip. By pointing a wiimote at a projection screen or LCD display, you can create very low-cost interactive whiteboards or tablet displays. Since the Wiimote can track upto 4 points, up to 4 pens can be used. It also works great with rear-projected displays.

Software
The calibration and mouse cursor emulation software is available for you to download and try yourself. Note: My mouse emulation code isn't perfect. If any of you are programmers and can get it working with Alias Sketchbook, drop me a line.
1. Connect your wiimote to your PC via Bluetooth. There are a number of tutorials online on how to do this, possibly even for you specific software/hardware configuration. The Wiimote works with many (but not all) Bluetooth drivers. You can report/read about compatibility issues at WiimoteProject.com
2. Download the Wiimote Whiteboard software to the right. Please read the "READ ME.txt" file first! Make sure your wiimote is connected via Bluetooth, and then run the ".exe" in the main folder. NOTE: Good placement of the wiimote is key to good tracking. View the README for more info.

Multitouch: The multitouch demos are custom C# DirectX programs. You may download the sample program to the right, but this is provided for developers without support or documentation. The code is built on top of this Wiimote library. Unfortunately, multi-touch capable applications are currently extremely rare. Hopefully, that will change as more developers explore its potential.

Building pens: Here is a simple schematic of the light pen. The LEDs that I use are Vishay TSAL6400s running at 100mA, but lots of other LEDs will work too. You also might be able to jump start your experimentation by retro-fitting a mini keychain light with an IR LED. I'm currently looking into manufacturing and selling IR pens, but this may take several months.

Mac/Linux Versions: Due to personal time contraints, I probably won't be able to make a port myself. But fairly mature versions are available online. However, I haven't tried them myself. I've also created a Source Forge Project - Wiimote Whiteboard, but it does not seem to be getting much love.

A visit to this project's FAQ and Advanced Discussion post may be very englightening. You may also find the official discussion forums for my wiimote projects helpful: WiimoteProject.com

Downloads:
Windows (32-bit)
Wiimote Whiteboard* (original 12/19/07) - includes source
Wiimote Whiteboard v0.2* (updated 3/27/08) - includes source
Wiimote Whiteboard v0.3* (updated 8/20/08) - includes source

* there appear to be some issues with 64-bit machines, and BlueSoleil. Alternative Bluetooth software may work. Check the forum for more info.

Mac (Java)
You can do a quick webserach or try the version created by uweschmidtwhich is a fairly mature Java-based version for Mac.

Multitouch:source code

Fun software (mostly free) to try with this:
Mapping software: Microsoft Virtual Earth, Google Earth
Physics simulation software: Phun
Virtual Tourism: Photosynth
Media Browsing: CoolIris
Painting progams: Alias Sketchbook Pro
Note taking:One Note
Handwritting recognition: Windows Tablet PC Edition - works great if you have it, not a stand alone program to install.

Head Tracking for Desktop VR Displays using the Wii Remote

Using the infrared camera in the Wii remote and a head mounted sensor bar (two IR LEDs), you can accurately track the location of your head and render view dependent images on the screen. This effectively transforms your display into a portal to a virtual environment. The display properly reacts to head and body movement as if it were a real window creating a realistic illusion of depth and space.

The program only needs to know your display size and the size of your sensor bar. The software is a custom C# DirectX program and is primarily provided as sample code for developers without support or additional documentation. You may need the most recent version of DirectX installed for this to work.

Software
To run the DesktopVR program you see in the video:
1. Connect your wiimote to your PC via Bluetooth. If you don't know how to do this, you can follow this tutorial. I've been told it works with other Bluetooth drivers, but I have not tested them myself.
2. Download the WiiDesktopVR (v02) sample program. Read the README file on program usage and configuration. Launch the "WiiDesktopVR.exe" in the main folder. A potentially more stable/Vista/64-bit compatible version has been created by Andrea Leganza. There also may be more variants on the web.

NOTE: If you are having trouble with running the program, you can check myproject blog post about it or check the forum for assistance. I am unable to replicate these problems, so it hard for me to debug them. But, other people have figured it out. Things that have been identified to help: delete the "config.dat" file and re-run the program, install a new version of Direct X, or istall .NET 2.0.

Developers Notes: The code is built on top of this Wiimote library. To compile the program, you will need a C# IDE and the DirectX SDK. More notes are in the README.

A visit to this project's FAQ and Advanced Discussion post may be very englightening. You may also find the official discussion forums for my wiimote projects helpful: WiimoteProject.com