What is a screen reader, and how does it work?

A screen reader is a software application which blind or visually-impaired people install on their devices, e.g. computers or smartphones, to help them experience what the screen shows.

What does a screen reader do?

Mainly, a screen reader reads the text on the screen out loud, much like a mother reads a book out loud to her child. It basically is a text-to-speech application. At least some screen readers can also send text to a Braille display, a hardware device that mechanically renders text as Braille script, one line at a time.

The user can navigate around the screen and choose which text shall be read. It is also possible to navigate within a text, e.g. skip back a few words to have a certain part of the text re-read, or to pause and resume the reading.

Some screen readers can identify the language of a text which they are about to read out, and then they read the text out loud in the appropriate language.

When a screen reader encounters an image, it may announce the image with, "Image," or, "Graphic," or something like that. NVDA says, "Graphic," at the beginning of an alt-text. If the image has an alt-text, then the screen reader reads out the alt-text immediately after announcing the image.

However, screen readers do not know multiple-paragraphs alt-texts. This is because long alt-texts with multiple paragraphs are exclusive to Mastodon. Nowhere else on the Web do alt-texts grow so long. Making alt-texts pleasant for sighted users to look at is a Mastodon-only thing, too; elsewhere, alt-texts are only for those cases in which the image can't be seen. No screen reader software is being designed specifically for Mastodon's culture. Thus, screen readers tend to interpret each paragraph as its own separate alt-text. NVDA starts reading out each paragraph of a multiple-paragraph alt-text with, "Graphic." And this is why you should never use line breaks in alt-text.

What limitations do screen readers have?

Some websites and blog posts claim that screen readers still have an upper character limit for alt-texts, i.e. they cannot read out alt-texts that are longer than a certain number of characters. This is no longer true.

If screen readers can identify a language and choose the right language in which to read a text out loud, they can only do so before reading out loud. They cannot necessarily identify and switch languages mid-reading and particularly in the middle of an on-going sentence. Most importantly, they cannot identify and switch languages in the middle of an alt-text. This is a problem when a post includes an image with text in a different language than the post itself.

While screen readers can navigate through text, they cannot navigate through alt-text. One they've started reading an alt-text, they can only either read through the whole alt-text or skip back to the beginning of the alt-text. But they cannot pause while reading an alt-text, and they cannot skip back to some point within the alt-text.

Imagine you've written an alt-text of 1,500 characters. Now, a screen reader user has their screen reader read your alt-text out loud. They're distracted by something at the 1,400-character mark. They need to have the last part of the alt-text re-read. But they can't just go back a few words or a sentence. They can only go back to the beginning of the alt-text and have the entire 1,500-character alt-text read out once again.

Also, while screen readers can pause while reading out normal text, they can't necessarily pause while reading out alt-text. However, the longer an alt-text is, the greater is the chance that someone may want the screen reader to pause while reading out the alt-text. This simply isn't possible. Screen reader users have to let the screen reader ramble down the entire alt-text all the way to its end, no matter how long it is. Alternatively, they can stop the screen reader entirely. But even then they can't resume in the middle of the alt-text.

This is one reason why alt-text should not be longer than absolutely necessary.

What else can't a screen reader do?

One thing that screen readers definitely cannot do is look at images and describe them. Particularly, describe them accurately, no matter how obscure the contents of the image are, depending on the context in which the images are posted and while following all the rules in this wiki. No AI out there is fully capable of this at all, especially not without any specific prompt.

Besides, an AI will take several seconds or even half a minute or more to put together an image description. But a screen reader cannot pause for that long and expect its user to wait for half a minute or so until the image description is calculated.

At most, some screen readers can use OCR to read text in an image if it's readable enough. But even that isn't 100% reliable.

The only really reliable source for accurate, context-dependent image descriptions and correct text transcripts are humans.

It's also important to know that screen readers don't process text exactly like a human brain. For example, humans tend to know certain abbreviations, and they tend to know whether it's an acronym that's supposed to be read out like a word, or whether it's supposed to be spelled letter by letter. Screen readers can't do that nearly as reliably.

Another example are certain special characters. If you add hashtags to an alt-text, a human reader will simply ignore the number sign (#). A screen reader will read the hashtag out as a hashtag, starting with, "Hashtag," even though hashtags don't work in alt-text anyway. Likewise, a screen reader will read out characters that it can't put into the context of the surrounding text, e.g. emoji characters or emoticons composed of multiple characters, by speaking out the official Unicode name of the character.

If you have emoji in your text, no matter if it's the post text or the alt-text, screen readers will read out the name of every single one of them, one by one. If you have a composed emoticon like "(╯°□°）╯︵ ┻━┻" in your text, no matter if it's the post text or the alt-text, a screen reader won't recognise it and read it out as "table flipping emoticon". Instead, it will read out the name of each character, "Left parenthesis box drawings light arc up and left degree sign white square quadrature, alchemical symbol for salt degree sign fullwidth right parenthesis box drawings light arc up and left presentation form for vertical left parenthesis space ess-pee box drawings heavy up and horizontal box drawings heavy horizontal box drawings heavy up and horizontal."

Examples of screen readers

Below are some notable screen readers. Warning: The links to Microsoft Narrator, VoiceOver and TalkBack lead to Microsoft, Apple and Google webpages respectively. Also, the "Open-source" links for NVDA and TalkBack lead to Microsoft's GitHub.

Name	Platforms	Price	Code	License
NVDA	Windows	Free	Open-source	GNU GPLv2
JAWS	Windows	$95/year for a non-professional home license	Closed-source	Proprietary
Microsoft Narrator	Windows	Free; built into the OS	Closed-source	Proprietary
VoiceOver	macOS, iOS, iPadOS	Free; built into the OS	Closed-source	Proprietary
TalkBack	Android	Free; built into the OS	Open-source	Apache 2.0
Orca	Linux, other *nix	Free	Open-source	GNU LGPLv2.1