Is this because my camera is slightly at wrong angle or where do i play with the settings in Freetrack to rectify this?
Freetrack can adjust cam-positions out of axis with this settings:
It's better to well install cam in front of you in a plane with your head-mount if possible.
But the issue you discribes are probably due to the dimension settings of head-mount.
See here:
http://forum.free-track.net/index.php?showtopic=3220&page=1#16965
A small dead-zone in curves can help to stay centred.
See:
http://forum.free-track.net/index.php?showtopic=3241&message=17062