In their paper A Practical Deep Learning-Based Acoustic Side Channel Attack on Keyboards (full PDF), UK researchers Joshua Harrison, Ehsan Toreini, and Marhyam Mehrnezhad claim that the trio of ubiquitous machine learning, microphones, and video calls “present a greater threat to keyboards than ever.”
Laptops, in particular, are more susceptible to having their keyboard recorded in quieter public areas, like coffee shops, libraries, or offices, the paper notes.
Combining the output of the keystroke interpretations with a “hidden Markov model” (HMM), which guesses at more-likely next-letter outcomes and could correct “hrllo” to “hello,” saw one prior side channel study’s accuracy jump from 72 to 95 percent—though that was an attack on dot-matrix printers.
The Cornell researchers believe their paper is the first to make use of the recent sea change in neural network technology, including self-attention layers, to propagate an audio side channel attack.
Because of this, the potential for a second machine-bolstered system to correct the false keys, given a large language corpus and the approximate location of a keystroke, seems strong.
The 2013 “Dropmire” scandal that saw the US spying on its European allies was highly likely to have involved some kind of side channel attack, whether through wires, radio frequencies, or sound.