We found out what 4 pinouts are (SP_X, SP_Y, SP_Z, SP_G)
SP_Y + SP_Z = ↑
SP_Y + SP_G = ↓
SP_X + SP_Z = ←
SP_X + SP_G = →
Prof.Ian guess 2 of 4 are related to power and ground(GND)
We concluded that Trackpoint can not be used without a existing IC chip.
Go with Plan B at the same time
We soldered trackball input.
We are going to connect
hacked keyboard - Arduino + A type USB-host shield
Arduino + Bluetooth module - Hololens
->Bluetooth module<-
Week 7 - Making prototypes
Progress on the prototype
The parts finally arrived!
We bought a keyboard from Lenovo that contains a TrackPoint and a joystick for arduino (as an alternative solution). We succeeded in disassembling the keyboard - that is, separating the upper plate from the lower one and pulling out all the circuits. We also found an Arduino Library for TrackPoint, which allow developers to get positional data from the pointer and send it to an Arduino module.
->keyboard disassembly success!<-
Meanwhile, we confronted some problems as well:
1) We could not fully detach the sensor part from the keyboard, because any of the screwdrivers we had didn’t fit to the joint screw. Therefore we could not connect it to Arduino right away.
2) Trackpoint is an isometric input device, and it uses pressure information only to move, not to select. To solve this, we ordered a circle-shaped pressure sensor about 4mm wide. We thought that if we get additional pressure input, we would be able to set a threshold to distinguish the idle state and the click state.
pressure sensor
Setting aside the fact that we couldn’t use the TrackPoint on Arduino right away, we
Prepare Hololens development environment
→ Heena is good at, who is in Canada this week
We also discussed how we should implement our ideas into codes.
1) In order to use the TrackPoint, we need to disable the head-gaze reticle, but still use the positional and the rotational information of the head, so that we know where the visible area locates.
2) We should then map the coordination values of the TrackPoint to the visible area, and Raycast (a concept from Unity developers use to check the user’s status - either standing still or moving) from the head.
3) This way, the TrackPoint will act like an eye-gaze system, which is different from a head-gaze one. This also might be helpful in that it can test how eye-gaze will work in the future.
As got clear, we came up with a brief experimental plan.
Building Hololens app
Or Using existing ‘adjust’ UI
1) To select a few objects in(A)/out(A’) of fov(field of view) of Hololens)
2) To drag an object in(B)/out(B’) of fov(field of view) of Hololens
Experiment sketch
Mentor meeting - with Rasel
Rasel helped us to completely detach the sensor from the keyboard. When we took the connector off, we realized that TrackPoint was connected on a ribbon circuit.
He also suggested us to read a reference paper. This paper is about
Week 6 - Spend more times with AR
Be more specific at problem
Prof.Ian: “I went to KAIST and saw some interesting research,
and they only had a specific problem (Re-sizing a window),
and if the problem is too big, you should make your problem specific.”
Prof.Ian: “To do that, I think you should use AR device a lot, sit down with Hololens(AR device), go around with it,
even you’d better go cafe with it. Spend more times with it finding a sub-situation of what does not work
which finger input device can solve it.”
Consider how you can implement your concept with existing head motion
We had to think about how we can use our concept with head motion.
If I turned my head to see the other side while I am moving my cursor with finger input,
and it would be weired if the cursor stays in the space before turning my head.
If we don’t think about the movement that users are going to expect, it would be failed.”
1st Prototyping plan
If we’re done with the circuit, we should consider prototyping material.
The prototype had better be 3D printed with elastic filaments like Ninjaflex or Semiflex.
Hard rings do not fit well.
Because of the nature of the filament, the layer is thicker.
Therefore, modeling it like 2.5-dimension will reduce the failure.”
Ninjaflex filament for 3D printing
Week 5 - Buy things and Make plan B
Buy components
We started to evaluate the novelty of joystick inputs in terms of research through papers(related works).
Trackpoint has drawback. It popped up in the ring.
Therefore, We considered flexible nail input as a plan B.
Plus, We considered pressure based input as a plan C.
Then plan D for trackball or joystick
Week 4 - Let's go for force based input
Candidates of our ring concepts
We started to evaluate the novelty of joystick inputs in terms of research through papers(related works).
There are isometric joysticks, isotonic joystick, and touchpads.
An isometric joystick (e.g., the IBM Trackpint) is a force-sensing joystick that returns to center when released.
Isotonic joysticks sense angle of deflection.
Touchpads are small, touch-sensitive tablets often found on laptop computers.
There are lots of movement based input device but force input device are usually used as supplementary device.
Force based input is smaller than movement based input so that force based input was more appropriate for small ring.
If you put the touchpad on the ring, it will be very small and you have to do repetitive and discontinuous tasks.
Therefore, we decided to find more works on force based input to evaluate novelty in terms of research.
“Okay, we are done with papers. The force based input is less-crowded area, there is little studies.” - Ian Oakley
Pros and Cons of force based input
Pros
You can move the cursor without moving your arms. Especially, if you have a sensor on your hand, it can be used without pushing back because it is used in hand. </cite>
Cons
It is difficult for beginners to learn first. Although it is easy to operate in one direction, it is difficult to move the cursor in a circular shape.
