Skip links

techesoterica.com

A blog dealing with sensory substitution and other esoteric concepts and technologies like speech-recognition and chaos theory

Main navigation

Sensory substitution

Looking out the viewfinder of a camera using the vOICe

by ·

A photographer friend invited me to his house for dinner. During the course of the evening, I had a chance to play with his cannon slr camera. I used the vOICe to look out of the viewfinder of the camera. As I have observed previously, you see very little. However, at that time, the vOICe still had the bug where very little of the view was shown on third-generation mobile phones. I now tried looking out the viewfinder with the version of the vOICe that had this bug fixed. I saw more than what I earlier saw. My friend was standing against a white background namely, the door of the washroom. He had to use the flash but, the camera of my Nokia e51 managed without much difficulty. Initially, I saw a patch of bright light and then a lot of darkness. It turns out that the cap of the lens was in place. Once that had been removed, I tried to distinguish him from the door of the washroom. I found it rather hard to do but then, I was not wearing any earphones. As I moved closer, I eventually reached a stage where I could see the buttons of his shirt. I perceive them as a lot of bleeps. So, if you are photographing a person, be careful about how much detail you are getting in the photograph. If you are getting an excessive amount of detail, move the camera back a bit.

The inside of a desktop computer

by ·

insidepc

The inside of a personal computer
The inside of a personal computer
I needed to reseat the ram in my dad’s machine so had to open it up. Visually, the inside of the computer is not all that interesting. There is a lot of cable and it is difficult to make out much detail. The more interesting part was taking the image since I had to focus the lense myself. the cabinet was lying on it’s side between 2 tables. The table on the left has a reading lamp. This llamp was focused on the cabinet such that the inside of the cabinet was eluminated. the focusing involved me moving the phone to the correct height. The soundscape had to be crisp and I had to ensure that the maximum area inside the machine was being covered. My first image was very dark. I did not have the lamp switched on. It was ok as far as the soundscape went but sited people were unable to see much. I then switched on the lamp and got a better image.

Tagged soundscapes

by ·

tagged-squareThe image of the tagged square which is a rectangle technicallyAt long last, I was finally able to try tagging a soundscape. I took a basic rectangle and, using audio editing software, namely goldwave, I was able to mix my audio tags into the soundscape.

Since this is a proof of concept, I have only tagged the verticle sides of the rectangle. Tagging the horizontal sides is tricky since they are rendered as a single long beat. If you listen carefully, you will be able to make out that there are actually two notes representing the upper and lower sides. However, I have not found any easy way of tagging these. The idea behind the tagging is to add labels to soundscapes just like the way labels are added to images in print.

Finally, I have played the soundscapes at the “very slow motion” setting of the vOICe. This facilitates the tagging and also helps to render the image clearly at least for new users.

Note:
my thanks to my cousin Abhishek for helping with the mixing and for drawing the rectangle.

A what/where visual-to-auditory sensory substitution fMRI study: can blind and sighted hear shapes and locations in the visual cortex?

by ·

See this ECVP supplement for the abstract. Essentially, the study concludes that the visual cortex does activate when individuals hear soundscapes asociated with objects as opposed to listening to arbitrary sounds asociated with objects. My only question is how do we explain this to people and show them the difference? I have made several presentations on the vOICe and a lot of people equate the vOICe to sonar. This is not true and we need a convincing way to demonstrate this to the end user.

A tutorial on the vOICe

by ·

I have created a tutorial on the vOICe. You can see it at the following link. The tutorial is meant for a beginner. It walks the reader through simple images and familiarizes them with the vOICe and seeing with sound. The idea is to give the user a launch pad from where she can further explore the vOICe. Do study the other tutorials such as DR. Meijer’s plain text tutorial and the series of podcasts a number of users including me have recorded about the vOICe

Tutorial on the vOICe

An area wise map of sysmic activity in India

by ·

See this map of seismic activity in India. The map is divided into different bands. Red are the most active regions and white are the most stable regions. You will need to use the color filters of the vOICe to hear the different bands.

Yellow regions are the areas with a moderate amount of seismic activity. The color identifier of the vOICe is able to identify some of the colors in the image. I wish it would give me a complete color breakdown. It does render the entire image after all.
<