A lot has happened in the (as I first sat down to write this) five days since a viral tweet sparked The Black Spatula project. For those just joining

The Black Spatula Project: Day Five - by Steve Newman

submited by

Style Pass

2024-12-24 17:00:03

A lot has happened in the (as I first sat down to write this) five days since a viral tweet sparked The Black Spatula project. For those just joining in, this is a community project to leverage AI to help weed out errors in scientific papers. It’s named for the kerfuffle over kitchen implements made from black plastic, which were reported to contain dangerous levels of fire retardant chemicals, based on a publication that turned out to contain a simple math error. (I’m still not sure that black plastic is the best thing to be using around your food, but it’s 10x less bad than claimed by the paper.)

The community that came together spontaneously is doing all of the actual work. My role at this point is primarily cheerleader.

The progress in a few short days has been amazing! Our WhatsApp group is now a WhatsApp community with over 250 members; there are 242 people on the Discord. There’s a GitHub repository, a website, the beginnings of a database of known flawed papers, active collaboration on prompting techniques, and more.

We are already uncovering errors. For instance, a peer-reviewed paper on dengue fever was found to contain an error in the way cases of mild and severe dengue were grouped – we’ve reported this to the author. Many more apparent issues have been identified in other papers, and are still being verified.