Cornell Tech researchers have discovered a new type of online attack that can manipulate natural-language modeling systems and evade any known defense

Hackers can ‘poison’ open-source code on the internet | Cornell Chronicle

submited by

Style Pass

2021-08-14 10:30:02

Cornell Tech researchers have discovered a new type of online attack that can manipulate natural-language modeling systems and evade any known defense – with possible consequences ranging from modifying movie reviews to manipulating investment banks’ machine-learning models to ignore negative news coverage that would affect a specific company’s stock.

In a new paper, researchers found the implications of these types of hacks – which they call “code poisoning” – to be wide-reaching for everything from algorithmic trading to fake news and propaganda.

“With many companies and programmers using models and codes from open-source sites on the internet, this research shows how important it is to review and verify these materials before integrating them into your current system,” said Eugene Bagdasaryan, a doctoral candidate at Cornell Tech and lead author of “Blind Backdoors in Deep Learning Models,” which was presented Aug. 12 at the virtual USENIX Security ’21 conference. The co-author is Vitaly Shmatikov, professor of computer science at Cornell and Cornell Tech.

“If hackers are able to implement code poisoning,” Bagdasaryan said, “they could manipulate models that automate supply chains and propaganda, as well as resume-screening and toxic comment deletion.”