Progress on the prototype

The parts finally arrived! We bought a keyboard from Lenovo that contains a TrackPoint and a joystick for arduino (as an alternative solution). We succeeded in disassembling the keyboard - that is, separating the upper plate from the lower one and pulling out all the circuits. We also found an Arduino Library for TrackPoint, which allow developers to get positional data from the pointer and send it to an Arduino module.

Meanwhile, we confronted some problems as well:
1) We could not fully detach the sensor part from the keyboard, because any of the screwdrivers we had didn’t fit to the joint screw. Therefore we could not connect it to Arduino right away.
2) Trackpoint is an isometric input device, and it uses pressure information only to move, not to select. To solve this, we ordered a circle-shaped pressure sensor about 4mm wide. We thought that if we get additional pressure input, we would be able to set a threshold to distinguish the idle state and the click state.

Setting aside the fact that we couldn’t use the TrackPoint on Arduino right away, we Prepare Hololens development environment → Heena is good at, who is in Canada this week

We also discussed how we should implement our ideas into codes. 1) In order to use the TrackPoint, we need to disable the head-gaze reticle, but still use the positional and the rotational information of the head, so that we know where the visible area locates. 2) We should then map the coordination values of the TrackPoint to the visible area, and Raycast (a concept from Unity developers use to check the user’s status - either standing still or moving) from the head. 3) This way, the TrackPoint will act like an eye-gaze system, which is different from a head-gaze one. This also might be helpful in that it can test how eye-gaze will work in the future.

As got clear, we came up with a brief experimental plan. Building Hololens app Or Using existing ‘adjust’ UI

1) To select a few objects in(A)/out(A’) of fov(field of view) of Hololens)
2) To drag an object in(B)/out(B’) of fov(field of view) of Hololens

Experiment sketch

Mentor meeting - with Rasel

Rasel helped us to completely detach the sensor from the keyboard. When we took the connector off, we realized that TrackPoint was connected on a ribbon circuit. He also suggested us to read a reference paper. This paper is about