Skip links

techesoterica.com

A blog dealing with sensory substitution and other esoteric concepts and technologies like speech-recognition and chaos theory

Main navigation

Accessibility

Reading books on a raspberry pi 400 using text to speech

by ·

One of the challenges I faced in Linux was that there did not appear to be an accessible application to read epub files. Yes, I could open epub etc but that defeats the purpose of the format. I wanted a generic reading solution.

That solution is ebook-speaker
The program goes beyond your regular DAISY book reader solutions. It can read a bewildering number of file formats and can even talk to hardware scanners and OCR to ensure that image files are covered.

The best part is that it is a console application hence very light in terms of resource usage which makes a difference on resource constrained devices like the raspberry pi 400 which is where I am running it.

I launched the program from the mate terminal and put orca to sleep while I read the book. If you do not want to do that, you can specify the -S option so that the phrase numbers are not read.

One of the biggest challenges with the program is installing it. I wanted to use it on a raspberry pi and there was no package for raspberry pi os. I had to build the program from source.
This is not as hard as it sounds. It takes some time, say about an hour because you need to install many dependencies both for compile time and runtime.
The dependencies are mentioned in the documentation. However, I have created a package which will allow you to install the application with a

sudo dpkg -i ebook-speaker_6.1-1_armhf.deb

The direct link to the package is below.
Download the ebook-speaker package for arm32 namely the raspberry pi

So what can’t you do with the program besides make your morning coffee?
Below markup navigation remains a challenge. The best way so far is to save a portion of the content to disk.

Another point with ebook-speaker is the developer.
Mr. Lemmens is a responsive developer and is quick to implement constructive suggestions.
Ebook-speaker is a superb solution for reading epub books as well as other books and must be tried first before moving to any other solution.

Reading images in Linux via OCR desktop

by ·

Optical character recognition is one of those crucial things we cannot do without as a blind computer user. You don’t know when you will need it.
Most screen readers on windows have the ability to recognize text on controls or in an image. The orca screen reader on Linux does not have this capability. However, there is a script called ocr desktop that does this.

The script grabs am image of the current window, magnifies it and then runs optical character recognition on it. It then displays the results in a read only window which comes on the top of the existing window. You read the results with the arrow keys and standard reading commands. Once you have finished reading the results, you can dismiss them by pressing the escape key.

The script ocrdesktop is present in the arch linux repositories. On raspberry os, we need to install some dependencies before the script will work.
At a terminal, run the following commands to install the dependencies for ocrdesktop.

sudo apt install -y tesseract-ocr
pip3 install pillow
pip3 install tesserwrap
sudo apt install libtesseract-dev
sudo apt install -y libwnck-3-dev

Note

If you you do not have pip3 installed, then run

sudo apt install -y python3-pip

You can then download the ocrdesktop repository by cloning it from git.

git clone https://github.com/chrys87/ocrdesktop.git

Switch to the ocrdesktop folder and run

./ocrdesktop

If you run the script from a ssh prompt, you will get an error about unable to initialize the xinit server. That is good because it tells you that there are no unmet dependencies. This error occurs because your desktop is not active in the ssh session.

You can now navigate to the pi’s desktop and if you are using mate, use the keyboard button from the control center to assign a shortcut to the script. Other desktop environments have their own mechanisms to assign keyboard shortcuts to commands. In mate, which is what i am using, do the following.

  1. Navigate to the option to add keyboard shortcuts and activate it.
  2. You will be on a page wwhere there is a tree and a table of corresponding shortcuts.
  3. Tab to the button labeled “add” and activate it.
  4. You will be asked for a name and if you tab, the command you want to run.
  5. You then populate these values. The name can be anything you want. You will need to type the full path to the ocrdesktop script. On my raspberry pi, the path is as follows.

    /home/pi/ocrdesktop/ocrdesktop
  6. Once you confirm this dialogue box, you will be placed back in that table and the ocr desktop shortcut will be present in the table under the custom shortcut section.
  7. When you reach the entry of the shortcut, you will hear orca tell you that it is disabled.
  8. Hit the right arrow once to land on the column contained the term disabled and press the enter key.
  9. You may not hear any feedback from orca but this is where you press the shortcut you want to assign to ocrdesktop
  10. Once you have done so and assuming there are no errors, the script will be assigned that shortcut which you will be abkle to hear.

You can now grab any image and hit the assigned hotkey and have the text in that image recognized.
If starting from scratch, you may want to install a program to view images. There are plenty of them but a simple one I am using at the time of this writing is Shotwell

Future ready vision with the vOICe and the raspberry pi 4B

by ·

I recently acquired a raspberry pi 4B with 8GB of ram along with a pi camera and other sensors. My first project is to build a robust platform on which I can run the vOICe.
It is possible to run the vOICe on the phone and on smart glasses and I do both. However, I want an inexpensive solution whose components can be easily replaced. Moreover, I want something where the software is being upgraded quickly and that is mainstream.
The raspberry pi fits the bill. It is mainstream, is being upgraded and the raspberry OS is a derivative of debian which works.
Moreover, the solution is accessible.

Prerequisites

  • A raspberry pi 4B, get either the 4GB or 8GB variants.
  • A keyboard connected to the pi.
  • A pi camera that is connected to the pi. You can use a USB webcam.
  • A consistent power source
  • Headphones or powered speakers connected to the 3.5MM jack of the pi or you need audio over HDMI
  • A 32GB or greater sd card.
  • The pi must have an internet connection via wireless or a wired LAN
  • SSH must be enabled
  • Use a distribution with a desktop such as the raspberry OS full or desktop distribution

Testing if everything is working

  1. At the console type

    sudo raspi-config

    and check if audio out is set to your preferred device.
  2. Do enable the camera interface after which you may have to reboot the pi.

Testing if the camera is working

It is important to determine if the pi camera is connected correctly. The most accessible way to do this is as follows.
run the following command from within the console.

vcgencmd get_camera

On my raspberry pi, I get the following output.

pi@glasses:~ $ vcgencmd get_camera
supported=1 detected=1
pi@glasses:~ $

If you see that the detected camera is 0, then you need to check the wiring of the camera or something else. There is no point going beyond this step because you will not be able to get any output from the vOICe.

Getting the screen reader running

Raspberry OS has accessibility built-in. The problem is that the desktop needs a monitor connected before it launches. We are running the pi headless, therefore need to take additional steps to get the desktop running.

  1. Login into the pi and run the following commands.

    sudo apt update
    sudo apt upgrade -y
  2. Run the following command to determine if you can hear through your speakers and or headphones.

    aplay /usr/share/sounds/alsa /Noise.wav

    If you can here, then do not perform the next step. If you cannot, then continue with the subsequent step.
  3. Update the firmware of the pi to use a new alsa driver. Run the following program and follow the prompts.

    raspi-update
  4. Repeat the test for the sounds. If you can hear some noise, then continue. If not, troubleshoot
  5. You need to install a dummy display driver otherwise, the graphical desktop will not launch.
  6. To install the dummy display driver, run the following command.

    sudo apt install xserver-xorg-video-dummy -y
  7. You now need to add a configuration file so that the desktop uses the driver you have just installed.

    sudo nano /etc/X11/xorg.conf

    The file should contain the below entries. If the file is not there, then create a fresh file.

    Section "Monitor"
    Identifier "Monitor0"
    HorizSync 28.0-80.0
    VertRefresh 48.0-75.0
    # https://arachnoid.com/modelines/
    # 1920x1080 @ 60.00 Hz (GTF) hsync: 67.08 kHz; pclk: 172.80 MHz
    Modeline "1920x1080_60.00" 172.80 1920 2040 2248 2576 1080 1081 1084 1118 -HSync +Vsync
    EndSection
    Section "Device"
    Identifier "Card0"
    Driver "dummy"
    VideoRam 256000
    EndSection
    Section "Screen"
    DefaultDepth 24
    Identifier "Screen0"
    Device "Card0"
    Monitor "Monitor0"
    SubSection "Display"
    Depth 24
    Modes "1920x1080_60.00"
    EndSubSection
    EndSection
  8. Save and close the file. If using nano, hit ctrl+x and follow the prompts.
  9. It is important to test sound once again. If you do not hear any sound, check that audio is set to the 3.5MM jack via raspi-config
  10. Reboot the pi by issuing the following command.

    sudo reboot
  11. The graphical desktop should load and you will hear a prompt to press control plus alt plus space to install orca.
  12. Hit the ctrl+ + alt + space keystroke on the attached keyboard attached to your pi and follow the prompts. You will need to reboot once orca is installed.

At this point, you will have a pi which will speak to you. The orca screen reader will be active.

Running the vOICe

Launch chromium from the Internet menu and navigate to the following link.
The vOICe web app
Once the page loads, you will need to approve the use of the camera. Hit f6 until you reach the address bar.
Tab until you reach the button labeled “Allow” and activate it.
You should hear the vOICe sounding.

Making the solution future ready

In Linux, it is possible to share the camera. This allows me to add other solutions to the pi such as OCR and run them when the vOICe is running. I am using a virtual loop back device via the v2l4loopback kernel module. The first step is to install this module.
On raspberry OS, this involves building the module manually. This is because as of this writing, it is not present for the very latest build of the kernel.
You must first install one dependency.

sudo apt install ncurses-dev -y

Once you have carried out the above step, you are ready to get the kernel headers. Once again, there is an installation package available but it is usually not updated to the very latest kernels which is why we are following a manual approach.
Please Follow the instructions at the below page.
RPi-Distro
We are now ready to install v4l2loopback.
Please take the following steps.

  1. Ensure you are in your home directory by issuing the following command.

    cd ~
  2. We clone the repository to our pi.

    git clone https://github.com/umlaeute/v4l2loopback.git
  3. You have to build the module and add it to the kernal. Issue the following commands.

    cd v4l2loopback
    make && sudo make install
  4. If there were no errors, then run the following command.

    sudo depmod -a
  5. Test the installation by running the following command.

    modprobe v4l2loopback

    If you do not see any errors, then the module has been loaded successfully. If there are errors, then please review the installation instructions.

We now need to make the loop back devices permanent.
I have created two loop back devices but you can create as many or as few as you like.

  • We first load the module into memory.
    edit the following file by running the below command.

    sudo nano /etc/modules

    at the end of this file, add the following entry

    v4l2loopback
  • We need to specify options for v4l2 loopback. This is done By creating a file named v4l2loopback.conf in the following folder.

    /etc/modprobe.d
  • My v4l2loopback.conf file is below.

    options v4l2loopback video_nr=25,26 card_label="vOICe","ocr" exclusive_caps=1,1
  • Once you have saved this file, run the following command to commit the changes to the pi.

    sudo update-initramfs -c -k $(uname -r)

Reboot the pi and check if the right video entries have been created in the folder
Use the following command.

ls /dev/video*

On my raspberry pi, I see video25 and video26 which tells me that the loop back devices I want have been created.
You need to supply data to the loop back devices. I have done this via ffmpeg.
You first must determine which device is which. To do this, run the following command.

v4l2-ctl --list-device

Your hardware camera should be usually at /dev/video1 or at /dev/video0
Here is the output of this command when it is run on my raspberry pi.

pi@glasses:~/scripts $ v4l2-ctl --list-device
bcm2835-codec-decode (platform:bcm2835-codec):
/dev/video10
/dev/video11
/dev/video12

bcm2835-isp (platform:bcm2835-isp):
/dev/video13
/dev/video14
/dev/video15
/dev/video16

mmal service 16.1 (platform:bcm2835-v4l2-0):
/dev/video0

vOICe" (platform:v4l2loopback-025):
/dev/video25

"ocr (platform:v4l2loopback-026):
/dev/video26


This means that /dev/video0 is my camera while /dev/video25 and /dev/video26 are my loop back devices.
I need the copying process running whenever I start the pi so I am going to put it into a script. My script is below.


#!/bin/bash
ffmpeg -re -nostdin -i /dev/video0 -f v4l2 /dev/video25 -f v4l2 /dev/video26

I have called the script copy_camera_stream.sh
I have placed it in a sub directory called scripts under the pi user.
This becomes important because I will need to create a systemD unit file to run the script as a service.
Remember, once you have created the script, you must give it executable permission. From within the scripts folder, run the following command.

chmod +x ./copy_camera_stream.sh

Do test the file to ensure that there are no errors.
One way to test it is to use the screen program and run the file from within a screen session. You can then detach from the screen session, relaunch chromium and check the video source combo box of the vOICe web app to see if the video sources are working. Select the source you want to test and check if you hear the soundscapes.

It is time to create the unit file for systemD. This should be placed in the following folder.

/etc/systemd/system

The contents of my copycamerastream.service file are below.


[Unit]
Description=Run ffmpeg to copy the pi camera stream to virtual loop back devices
After=network.target

[Service]
ExecStart=/home/pi/scripts/copy_camera_stream.sh
WorkingDirectory=/home/pi/scripts
StandardOutput=inherit
StandardError=inherit
Restart=always
User=pi

[Install]
WantedBy=multi-user.target

You can launch the service and enable it

sudo service copycamerastream enable
sudo service copycamerastream start

Reboot the pi and check if the vOICe is working.
If yes, then congratulations!

The ActiveX (Australia) Ivy Plus, Digital Bathroom Scale For BMI Body Weight

by ·

I have recently bought the ActiveX (Australia) Savvy Bluetooth Body Fat Scale
I did have a talking weighing scale but that has stopped working which means that it is now giving a constant weight irrespective of who stands on it.
The savvy scale has a screen in the center. There are 4 pads on all 4 sides. You need to ensure that your feet are fully placed on those 4 pads. You can feel them quite easily.
I had no difficulty in finding the app. I am using the app on my Nokia 9 pure view phone running Android 10. If you need to search for the app, use the string “activex Australia”

  • The first few screens are accessible.
  • Be careful when typing your password. I suspect it does not take all special characters.
  • You will reach a point where you need to enter your date of birth. Tap that edit box and then swipe to set various parameters. You will want to tap the month first and then set it and the date and then the year.
  • The next problematic screen is the one where you need to enter your height. Get sighted assistance for this. You cannot enter the numbers directly. You have to tap the relevant number and explore by touch is the only way to get to these numbers.
  • After that you need to pair the scale with your phone. You need to stand on the scale and the pairing process starts. This is the most inaccessible screen of this app. You need to tap a graphical icon to complete the pairing. Keep that sighted assistant handy.

Using the scale however is accessible and you can do it independently. Launch the app, login if necessary and then stand on the scale. Your phone will sound a tone to indicate when measuring is complete. You can then use regular touch screen navigation to read your weight, body mass index etc.

In conclusion, the scale is hard to setup but once done, is easy to use and may be a good option if you do not find something that is more accessible.

EPIQ 2020: beyond my wildest dreams

by ·

Baby Yoda?? is probably the first thing that I think of as I look back at EPIQ 2020. It was a part of a message in Quorum Studio which is the integrated development environment for Quorum.
Quorum is an evidence based programming language. This means that user feedback is taken and incorporated before any feature addition. This is not a language that was crafted by someone in a lab. Yes, there is a lab involved but each feature has either a user request behind it and or has a user experience study behind it. This is perhaps why the language is so easy to use and what drew me to the language 4 years ago. At that time, I was building a chatbot and needed something accessible to create a user interface in. I wanted to use synthetic speech and quorum’s implementation was one of the easiest to code.
EPIQ or experience programming in quorum is the annual conference related to this language. This was an in person event and I used to routinely go green with envy as members of the quorum community would describe the fun they had. This time, the conference was conducted online.
It started with a fascinating key note by Dr. Jenna Gorlewicz about haptics and her work on touch interaction.
I did not realize how much you could do by varying the frequency of that single motor in my mobile phone. Quorum does have touch handling capabilities. I was also interested in learning about standards of haptic feedback and the extremely interesting work she was doing in using haptics to display graphs to the blind. This technology has huge potential because it is mainstream.
The session was interactive which means that I could ask whatever questions i wanted and did not fall a sleep.
Dr. Andreas Stefik next took the floor and outlined the development of quorum, the new scene editor, the licensing of quorum and a range of other things. By the end of his address, my fingers were itching to begin coding.

Every feature in quorum is designed with accessibility from the ground up. The scene editor is one of the best example’s of this. In other game engines, it is not easy to lay out characters if you are a screen reader user. Here, you have a nice tree view from where you select a character and drop it on to the grid. The coordinates of the grid are spoken as you navigate. If you are a programmer, this is similar to the visual basic tool box but even better because there, you had to check the properties of each control and do the math in your head.
Here, it is possible to get overlapping objects on your map and there is a mechanism in place to change the properties of each object.
We then broke into the advanced and learner sessions. There was no difference in content but a difference in speed.
In the first day’s session we delved further into the scene editor and made a map of our game’s world. There was a fascinating discussion we held on the different types of games such as role playing games, first person shooters etc.
The day ended all too soon.

Day 2 was as interesting. We dived into coding.
We had our map in place but now had to get our objects. We knew what the objective of our individual games was so had to decide what to do. We were working on a role playing game and had to setup our non-playing character who would tell us about our quest. Along the way, we continued learning about the physics system and movement. This is where I began to appreciate what I had learnt in high-school physics class. It was the practical application of the concept of velocity. Yes, in physics class, I was given examples and the teacher probably defined what it was but the application was clarified on day 2.
I had to use velocity to move the character in the game engine taking physics into account.

Day 3 was when we did some solid work around adding music and sound as well as completing our game’s objective. I think this is when we also were introduced to collision handling which is used everywhere in creating games. Mine was for my character to eat 21 mushrooms. Besides the coding, the facilitators encouraged us to learn the techniques relating to how something could be done. There was no one right way and many of us tried different approaches based on our various games.

Day 4 was the heaviest day in terms of brain cycles expended. We worked on functions such as finding the closest object to our player character. An overarching theme of our game design was accessibility. Just putting up our game universe, scenes and graphical players was insufficient. The idea was to make the game playable to as many people as was possible with varying abilities. Most of us incorporated screen reader support.
We also worked with the audio system and were able to add sounds to various game events like the collecting of weapons etc.

Day 5 was when I really felt the ease-of-use of quorum. I had completed my game and desired to add magnification to it. My wife is a user of magnification therefore this was a significant feature for me. In quorum, it took about 5 or 6 lines of code. In summary, I moved the camera with my player and used its setZoom function to allow for custom zoom values. Be warned, zoom values start from 1 which is the default. If you start with zero, you do not see anything on the screen.

On all 5 days, the sense of community was palpable and my compliments to the organizers for creating such a safe and vibrant group.
Day 5 was also when we were shown what quorum could do in a non-technical setting. The language is being used across schools in the USA and many of the teachers brought out how they had used LEGO to get children with multiple disabilities to emerge from their shells and begin interacting with their environment. Moreover, the language and the work also lead to a much better appreciation of blindness by sighted peers and FIRST® LEGO® League coaches. A really fascinating project was a LEGO robot that could throw an item across a table. The next one was the robot that swept a white cane and responded to obstacles in terms of finding its way around them.
The conference concluded with a stunning platform game demonstration by William Allee, one of the lead creator’s of quorum. He demonstrated a platform game with jumping, exceptionally creative use of the properties of the objects of the game engine and some terrific sound effects and parallax motion by using multiple scene files.
So, what else did I learn?

  • Pre-conference conversations are as much fun online as offline.
  • My parents were right to expose me to a wide variety of things and to give me the space to explore.
  • There are insects colloquially called murder hornets. I believe their sting is one of the most painful.
  • It is possible to not enable the physics engine but set objects to be collidible such that they respond to collision events.
  • Debugging errors is heaps of fun if you do it with the right set of people.
  • Accessibility works best when experienced so the lesson to all advocates of accessibility step away from presentations and use games.
  • I can sit without moving for over 4 hours given the right level of interest.
  • There are people in the world who ask good questions and from whom you learn
  • Math is much more fun when applied and this can start from early school.
  • My respect for researchers continues to deepen specially when I watch them collate multiple input and deal with conflicting priorities.

One of the questions we were asked frequently was “how EPIQ can be improved?”
It is a difficult one to answer because the online format was tried for the first time. I hope EPIQ 2121 adopts a hybrid model where those of us who cannot fly to the USA can also join. Hmm, how about a “destination EPIQ?”

converting JPEG files to PDF

by ·

I was recently in a situation where I had to convert a stack of JPEG files to PDF. This was a job need. The human resources department of my to be employer had asked me for a number of forms. These forms had to be filled and sent only as PDF. I did not think there would be a problem because I have a PDF printer installed. I was wrong; it appears that files have to be 2MB or less in size.
I am unaware of any equation that equates image size to file size. I searched and found a few online converters which would shrink the file to the size I wanted. However, I wanted to avoid online services because these were sensitive documents.
The answer was in a thread on the stack exchange forums.
Image To PDF or XPS
This is a tiny program written in visual basic .net. It is fully accessible and converted my files seamlessly. There was no complex dragging with the mouse and all the processing by the program was done on the local computer.
The program is supplied without charge but there is a donate button on the about page which you may want to use.
In addition, unlike the PDF printer approach where I would open each image, print, enter the file name and then move on to the next file, I can add the images I want to convert and the program will do the rest. The program can either combine all images into a single PDF file or it can save the individual files in a folder you specify.
It does not need any installation so there are no pesky licenses to manage.
Please note, there is no specific option that asks you about file size. The program does its conversion and by default, produces small PDF files.
Finally, the program can be run from the command line which allows you to use the program in any scripts you may have.

Playing armchair virologist with SARS-CoV-2

by ·

I wanted to take a look at SARS-CoV-2. Finding a downloadable image was tricky but I finally managed. Those of you with organic eyes can skip the rest of this post because you can see the image. However, those of us who have to use synthetic vision are not too badly off either.
See the below article that walks you through a few different images.
This Is What The COVID-19 Virus Looks Like Under The Microscope
You will need the vOICe Learning Addition for Windows to see these images. I am going to be as screen reader agnostic as possible.

  1. Navigate to the article at the above link.
  2. Read the article and then use the quick navigation feature to navigate to a graphic of choice.
  3. Launch The vOICe and activate its client area sonification feature.
  4. Swap back to the article and navigate to the graphic of interest.
  5. Pay attention to the soundscapes. You will be able to hear the spikes that give the virus its name.
  6. You can also use the color filter feature to focus on specific parts of the image as you read the article and follow along.

If you want a clean image of the virus to examine, then as of this writing, here are the steps. These may change because they involve accessing the image from the home page of a site.

  1. Navigate to the article titled New Images of Novel Coronavirus SARS-CoV-2 Now Available
  2. Once you are on the page, save the full HTML page including images and scripts. You can do this from your browser’s file menu or if using Google Chrome, from the chrome menu under the more tools option.
  3. Navigate to the folder where the page has been saved and then to the folder called NIH_ National Institute of Allergy and Infectious Diseases _ Leading research to understand, treat, and prevent infectious, immunologic, and allergic diseases_files
  4. In this folder, grab the file called Vero-Covid-19-24h-hi_i034-rk.jpg

You can load this image into the vOICe and run similar analysis and do whatever else you want to do. Try different color filters and play with the zoom and raster scan functions as a start.
If you want to see the image of our Sun’s corona for comparison, try the images at the following page.
What Is the Sun’s Corona?
You will have to use the techniques outlined to see the images. One thing that you will notice is that the spikes are significantly more dense.

Do let me know if you tried viewing any of the images and what happened.