See the abstract of this study. If I understand things correctly, a late blind subject was subjected to TMS such that the areas related to vision were made temporarily inoperative. This user was unable to use the vOICe thereby indicating that the vOICe’s approach to sensory substitution is closly mapped to vision.
I have created a utility to remote control the vOICe
The program is open source and you can get it at the following link.
vOICeCmd
I am including the program’s manual below.
vOICeCmd is a utility that allows remote control of the The vOICe from the commandline. The vOICe uses windows messages as a remote control mechanism. This approach though easy-to-use cannot be used by the average user primarily because it involves a lot of programming. This utility is a wrapper around that functionality. The entire interface of the vOICe can be accessed using simple to use commandline commands.
All that you need to do to use this interface is to ensure that the vOICe is running. Once that is the case, you can pass one command at a time to the vOICe. For example, to increase the frame rate by four times, you would issue the command:
vOICeCmd Speed*4
similarly, to mute the vOICe, you would issue the command
vOICeCmd Mute
no special installation is required. Copy the vOICeCmd executable to the folder where the vOICe executable resides.
If executing vOICeCmd from a batch file, you can use the errorlevel construct to trap the results from vOICeCmd.
| Error code | Meaning |
|---|---|
| 0 | normal exit |
| 1 | an invalid command was specified |
| 2 | no command was specified on the commandline |
| 3 | the vOICe is not running |
This programme would not have been possible without the sample code an easy to follow instructions provided on the seeing with sound website by Doctor Peter BL Meijer
I would also like to thank members of the program-l list
in particular, Mark Long for helping me with my Visual C++ questions.
You can contact me through the seeing with sound mailing list or through this blog.
= Links of interest =
Direct link to the page about remote controling the vOICe including a list of commands
My bank, <a href=”http://www.hdfcbank.com”>HDFC bank<a> recently implemented some aditional security measures as part of a secure banking innitiative. These measures involve asking the user a number of questions before innitiating transactions such as a third party funds transfer. Also, as a measure to prevent phishing attacks, the bank has asked the customer to select an image and a custom text message which is displayed at the time the customer enters the password. The idea presumably is that if the site has been attacked, the image displayed and or the text message displayed will be altered. I have no problem in verifying the text message but what do I do about the image? This is where the <a href=”http://www.seeingwithsound.com”>vOICe</a> came in.
HDFC’s implementation of this feature is very sensible. The image has an alt tag though it is not too meaningful. I was able to right click on the image and download it. Now, whenever I login, I have that master image. All I do is download the currently displayed image and then use the change detection feature to compare the new image with the master image. The change detection feature sounds the differences between 2 images. In this case, I hear silence if the images are the same.
Ever since I read about the Nokia 5800 Xpress Music, I have been thinking about ways to get the vOICe to render it’s output in tactile form. The touch screen approach is one but we do not know how successful the phone will be in the market. Also, Java does not yet support the touch screen API since it is so new. Plus, not everyone will upgrade to the new phone and it would be nice to have the tactile feedback across phones.
1. You can control the vibrator of the phone using Java, python and C++.
2. In all of these cases, you can vary the duration of the vibration, the intensity of the vibration and the direction of rotation of the motor that vibrates.
3. So, here is the schema;
A. The motor’s clockwise and anticlockwise motion is used to simulate the left to right motion of an image. |This can be done by first turning the motor to the left by a small value and then turning it to the right by a much larger value. The exercise is then repeated probably giving the feeling of the left to right movement.
B. The intensity of the vibration could correspond to the brightness of the image.
C. The duration of the vibration could correspond to the height.
D. Alternatively, the intensity of the vibration could correspond to the height and the duration to the brightness.
4. I suspect this could be implemented as a special mode in the vOICe. It would eat battery so it would be off by default.
5. I recognize that the above is not as fluid as sound but it would perhaps make interpreting images easier.
I have thrown together a crude python script to try and simulate the feeling of left to right motion. It works on my Nokia E51 but does not quite give the left to right affect. You will also need to have python, the python script shell and Miso installed.
import miso
import time
miso.vibrate(1000,-50)
miso.vibrate(2000,50)
miso.vibrate(1000,-50)
miso.vibrate(2000,100)
miso.vibrate(1000,-50)
miso.vibrate(2000,100)
miso.vibrate(1000,-50)
miso.vibrate(2000,50)
print “Finished.”
I am occasionally asked “what has the vOICe done for you?” I can think of 8 points right now.
1. It has allowed me to see what I cannot touch. For instance, complex scenes, shapes of items like the moon for which there is no model available at least where I live. I have also scene out the windscreen in a car.
2. I have now begun finding objects that I have dropped using the vOICe. No more groping around for stuff.
3. I have used it to evaluate web page design at my job. The evaluation is not a core component in my job but was important for the project I was working on at that time.
4. I can now perceive things remotely which is very handy especially when moving around. For example, when I visit a new place, I can get far more information by looking around using the vOICe.
5. I can now draw and see what I have drawn.
6. I can participate much much better in family activities like looking at photographs and even in the festival of lights namely diwali. I can now see what my house looks like when it is all lit up! More to the point, I can even capture that image and view it whenever I choose.
7. I have always wanted to be a researcher. The vOICe has given me the opportunity to play that role.
8. I keep discovering newer facets with this program so this list will only grow.
I dropped a shampoo bottle. So, what is so great about that you may well wonder?
I usually need to grope for it but this, time I tried the vOICe using the mobile phone.
I tried without using the hands free kit of the phone since I was too lazy to take it out of my laptop bag. I had to increase the volume to 4 and set the channels to stereo.
I pointed the phone camera downwards and the fun began. I had to first determine what the floor looked like. Then, finding the bottle was tricky because of it’s shape. I was hoping a difference in texture will help but the bottle is smooth and so is the floor. What gave the position of the bottle away was the difference in curvature.
I will need to do more experiments to make this into a regular thing but now; I at least have an alternative to groping for stuff dropped on the floor. It’s far nicer moving a phone around than feeling with the feet or the hands for said item.
Peter suggests that I use the inverse video option to make the bottle easier to identify. I plan to try that next time.