FreeTrack Forum

I heard about FreeTrack on FrugalsWorld and just had to try it out ... it works really, really well. Top job guys.

In fact it works so well, I'd like to be able to make a 2dof glove to use with it as well.

I reckon by canabilising an old optical mouse and putting the left button in the glove fingertip and the led above, placing a cam behind pointing at the screen, you should be able to make a rudimentary touch screen.

It works in as much as FreeTrack can see the mouse LED in front of the screen (that's just way too cool), but here's the thing that's holding me up ...

I want to be able to use my 6dof headset AND my 2dof glove.

I have two cams one pointing at my head, the other at the screen. All I need to do is set another instance of FreeTrack running to follow my hand and I'll be in business.

So is it possible to get two instances of FreeTrack running on Windows XP at the same time?

Thanks!

dead_letter_recovery @ 19/10/2007 - 03h37 a dit:

the led above, placing a cam behind pointing at the screen, you should be able to make a rudimentary touch screen.

The camera image plane must be in the exact same plane as the screen, but even then, to get a 1:1 correspondence between screen cursor and glove would be tricky, not forgetting webcam lens distortion which FT doesn't account for. An interesting idea though.

I have two cams one pointing at my head, the other at the screen.

2 cams would use a fair bit of cpu.

So is it possible to get another instance of FreeTrack running on Windows XP?

Check your PM.

Thanks Kestrel -- that's just perfect.

It does take a fair bit of CPU -- about 20-25% to run both cams so It might be a bit intensive, but we'll see how it goes.

As for plane and accuracy ... well, initially I'm looking to see if it will work in a flight sim 3d cockpit. The buttons are fairly well spaced so I'm hoping the accuracy won't be too much of an issue -- I'm not looking at pixel perfect movement.

And for the plane issue, that is a problem. The cam I'm using would sit exactly where my nose is for a 1:1 screen shot and that's just not good.

Maybe I'll have to not use it as a touch screen per se, but place the button elsewhere in the glove so at least you can use the glove as a pointer.

Anyway, I'll have a play and let you know what I come up with.

Again, many thanks!!!

oo what a great idea!
so you want to use your head for camera control and your hand as an input device? preparing for black shark are we?  

maybe you should use 4dof for your glove, that way you could try to implement  some kinda triggering movement like activating a switch or something. so instead of only using the glove for moving the pointer, you could use it for clicking the pointer too.

it'd be sweet if it works, once i completed my final project i might try to make one for black shark. thanks for the idea, good luck!

Hi einsena

I've actually put the mouse left-button microswitch on the glove fingertip so you click by pressing on the screen ... I was thinking about butting the right button on the middle finger but I'm not too sure.

Well, the good news is it works, kinda ...

the 2dof setting is fine for tracking, but because of where the camera needs to be it's a pig to setup...

Essentially the monitor takes up only part of the webcams field of view ... which alleviates some of the lens distortion issues, but introduces perspective issues -- for a flight sim it's not too bad because the hotspots are fairly large and the distortion doesn't interfere too bad.

And it's really cool just putting your finger to the screen and pushing those buttons ...

BUT the tricky part is that you have to make a vignette for the camera so that it only tracks the LED whilst it's over the screen ... otherwise when you hand travels past the screen boundaries FreeTrack still registers movement but the cursor gets locked against the edge...

It's a bit tricky to explain, but it makes perfect sense once you see it... basically if you move your hand two inches past the right of the screen edge FreeTrack is still plotting the movement, but the cursor is no longer moving because its butted against the edge of the desktop.

When you move your hand back onto screen the cursor starts moving immediately, but is two inches to the left of your hand.

My only solution is to mask the camera lens so FreeTrack can't see the LED once it's off screen. That keeps things aligned pretty well.

And that's the bummer. The mask needs to be set up for the exact position of the camera, so if you move the camera (or monitor) you need to change the mask size. And getting the mask the right size is a real problem.

The solution is to make a more permanent setup (or something that's easily adjusted)

There are a few more issues but these are with my hardware ... my second webcam is really crappy and only gets about 8fps so I get loads of lag and jerks

The second problem is running two tracking cams AND a flightsim is pretty intense on my processor (although I'm still getting 30fps apparently ingame). The tracking however is pretty jerky which at the moment I'm putting down to crappy webcam AND not enough horsepower, but I may play with the settings a little more on FreeTrack.

The final problem is one I didn't expect, and this is a co-ordination thing ... in a 3d cockpit with turbulence buffering your headmovements and your own natural inclination to move your head toward what you're looking at, the cockpit buttons move all over the place -- it's not something you notice too much until you try to push one of those darned things. You end up chasing the button all across creation!

So I guess you need to train your movemnts to accomodate, or use the glove in a 2d environment...

AND have a good webcam

AND have a good processor

Thought I'd let ya all know, just in case someone else is thinking of doing the same thing...

Oh ... and the mouse LED is VERY bright so I'd really suggest using an IR LED to save your eyes.

BTW System SPECS AMD 64 3200+/ XP sp2/nVidea 5200fx/1GB crappy RAM, 1 crappy webcam @15fps, 1 crappier webcam @ 7fps, 19 Inch CRT monitor,

Tested with Falcon 4 RedViper.

Will let ya know if I find anything else.

dead_letter_recovery @ 21/10/2007 - 23h50 a dit:

BUT the tricky part is that you have to make a vignette for the camera so that it only tracks the LED whilst it's over the screen ... otherwise when you hand travels past the screen boundaries FreeTrack still registers movement but the cursor gets locked against the edge...

It's a bit tricky to explain, but it makes perfect sense once you see it... basically if you move your hand two inches past the right of the screen edge FreeTrack is still plotting the movement, but the cursor is no longer moving because its butted against the edge of the desktop.

When you move your hand back onto screen the cursor starts moving immediately, but is two inches to the left of your hand.

My only solution is to mask the camera lens so FreeTrack can't see the LED once it's off screen. That keeps things aligned pretty well.

And that's the bummer. The mask needs to be set up for the exact position of the camera, so if you move the camera (or monitor) you need to change the mask size. And getting the mask the right size is a real problem.

The solution is to make a more permanent setup (or something that's easily adjusted)

This is because, by default, mouse emulation uses relative position.  Ticking the checkbox labeled 'absolute' in the Freetrack -> Output -> Mouse section will enable absolute position and should solve the problem.

Note that this will 'steal' the mouse while tracking and sensitivity is affected by the screen resolution.

dead_letter_recovery @ 21/10/2007 - 23h50 a dit:

Hi einsena

I've actually put the mouse left-button microswitch on the glove fingertip so you click by pressing on the screen ... I was thinking about butting the right button on the middle finger but I'm not too sure.

Well, the good news is it works, kinda ...

the 2dof setting is fine for tracking, but because of where the camera needs to be it's a pig to setup...

Essentially the monitor takes up only part of the webcams field of view ... which alleviates some of the lens distortion issues, but introduces perspective issues -- for a flight sim it's not too bad because the hotspots are fairly large and the distortion doesn't interfere too bad.

And it's really cool just putting your finger to the screen and pushing those buttons ...

hmm, plz pardon my curiosity (and noobness)
but after reading your post about using your finger to track AND to press something on the screen by putting a mouse button sensor on your fingertip, i can only come up with something like this...



so are your build anything like my pictured design?
i assume that by pressing on the screen you are pressing on the clear glass screen, and not any kind of monitor or something that would obstruct the webcam FOV. of course i assume that the webcam would need to be behind some kind of screen in order to be able to track your fingertip constantly that is.

einsena @ 22/10/2007 - 15h43 a dit:

so are your build anything like my pictured design?

No, the second webcam is looking at the computer screen.

Kestrel @ 23/10/2007 - 03h27 a dit:

einsena @ 22/10/2007 - 15h43 a dit:

so are your build anything like my pictured design?

No, the second webcam is looking at the computer screen.

O i see, genius, i forgot that webcams have a mirror/reverse mode. it would seem that the webcam would need a lot of gain and exposure tweaking in order to eliminate noise lights omitted from the computer screen. time to look for a second webcam i guess  

einsena @ 23/10/2007 - 03h48 a dit:

webcam would need a lot of gain and exposure tweaking in order to eliminate noise lights omitted from the computer screen

Computor monitors emit little to no IR, so they appear black when using a visible-light blocking filter.

Kestrel, of course, is spot on.

The second webcam looks directly at the monitor -- and just by placing a piece of negative over my cam it knocks out all the visible light so only the IR from the LED is seen.

As far as the absolute tracking is concerned, that's what I set the profile to. The problem is that the computer monitor doesn't fill the entire field of view of the cam so once the cursor is moved to the edge of the desktop, if the glove moves further beyond the edge of the screen offsetting occurs.

Here's what I mean -- follow this example:

Move your cursor with the mouse to the very right edge of your desktop untill it will move no more. Take note of the position of your mouse on your table.

Now move the mouse further to the right. Obviously the cursor won't move because it's locked to the edge of the desktop -- BUT your mouse has moved.

Now slowly start moving the mouse back to the left to where you noted it reached the edge of the desktop. You'll see the cursor moves as soon as you move the mouse, and when you reach the point on the table where you noted the desktop cursor stopped, the cursor is now well within the desktop area.

This is the offset that I'm talking about.

Now the only way to stop this happening is to tell FreeTrack to stop tracking once the LED reaches one of the screen boundaries. Ideally some type of bounding box drawn over the webcam monitor overlay would be fantastic so we can tell freetrack to only track if it sees anything in these areas.

But I'm no programmer and have no idea what's involved there -- but the low tech version of this idea is to just make a vignette for in front of the cam to block the light from ever reaching the cam.

In my example above with the mouse, it'd be the equivalent of turning the mouse off once the cursor reaches the edge of the desktop.

Hope that helps explain things a little better.

dead_letter_recovery @ 24/10/2007 - 07h11 a dit:

As far as the absolute tracking is concerned, that's what I set the profile to. The problem is that the computer monitor doesn't fill the entire field of view of the cam so once the cursor is moved to the edge of the desktop, if the glove moves further beyond the edge of the screen offsetting occurs.

The computer screen doesn't need to fill the entire webcam view. Single point tracking can control the cursor across the entire screen using only a fraction of the webcam view. Sensitivity adjustment should allow you to match LED to cursor. Absolute mouse control doesn't produce offsetting, when the cursor hits the edge of the screen it will only move again if you bring the tracking point back to where it was just before it reached the edge of the screen (just like TrackIR).

Now slowly start moving the mouse back to the left to where you noted it reached the edge of the desktop. You'll see the cursor moves as soon as you move the mouse.

Absolute mouse emulation does not do this. Are you sure you've enabled the 'Absolute' checkbox? If your mouse can be used during tracking then it's in relative mode which produces the results you've described.

You could use a very tiny camera (or a completly striped down one) mounted between the TFT matrix glass and the "backlight chamber" of on old expendable LCD monitor.

Ofcourse you need some adaptation but it could work. If you build an intermediate chamber like 1-2 cm wide and you "hide" the camera in the middle of the "backlight chamber" I think the light will have the space to cover the "shadow" of the camera and you would not see any major modification of the screen image...

Something like a behind the mirror spy camera... like in the movies .

I think DI table setup or even better for this FTIR table setup (multi-touch community) would be great for that. You can rather replace the video projector by a LCD Screen

Check Out http://nuigroup.com (also the forum), http://mtg.upf.es/reactable/?software, http://www.multigesture.net/, then goolge

I'm gonna build a DI one for a personal project one of those days (nothing to do with FT or simulator though)

hi its a good idea the thing with a glove.. there is a program from Johnny Chung Lee that uses the wiimote to make a tochscreen that contols the mouse position and click with a infrared light (when the light apears it automatcly clicks where the light is so there is realy no need for a mouse just somthing that turns the light on when you hit the screen) i tryed it and works perfect this could be a good solution to what you are trying to do the bad thing is that you need a wiimote and build a little irlight that turns on with a pushbotton (the mouse button should work great)..

here is a video on how they use it
http://www.youtube.com/watch?v=5s5EvhHy7eQ&feature=related

this is the program
http://www.cs.cmu.edu/~johnny/projects/wii/WiimoteWhiteboard.zip

maybe somthing like this could be put in freetrack since i read that there might be wiimote soport and maybee with 2 wiimotes the other program might get confused on which wiimote to use, so maybe if its posible to conect 2 wiimote you could do the touch screen with it or just use this program seperatly with another wiimote...

by the way great program this is my first post and would like to thank you guys for this program it kicks @ss!!!! and love the freetrack though the LAN!!