Show HN: Tracking how much of the HN front page is AI-generated

7% of the Hacker News front page is AI-generated right now. Every top story scored with Pangram
4 points by salahadawi 109 days ago www.salahadawi.com 1 comments
Summary not yet generated for this article.

Original Text

For the past few months, I've been sending every story that reaches the HN front page through Pangram's AI detector and noting the scores. The reason I made it is that I kept noticing posts that sounded like AI-generated to me. Sometimes the comments pointed it out, sometimes a post was near the top all day and no one said a word. I wondered if the share was really increasing or if I was just primed to see slop everywhere.

How it operates: The site identifies the top 30, gets each article, and evaluates the text using Pangram. Besides the live front page view, there are daily and monthly trends, with leaderboards for domains and authors that keep scoring high. I personally find it quite helpful as a sanity check on my own intuition before declaring something AI in the comments.

6,500+ stories in, it turns out the front page is still very human after all, which is the part that shocked me the most. A fraction of 13% of stories were AI flagged over the last 30 days. That's slowly ramping up (roughly a point higher than the previous month) but still not close to the feeling of 'everything is slop now' I had. At the moment it's 1 flagged story out of 30.

The flagged ones still manage to get by quite well. One of them is at #6 today with 545 points and a 99% score, and the discussion section is divided on whether it's AI or not.

Standard disclaimers: a detector score is not a piece of evidence, false positives do occur, and a high score can also be the result of a human coming up with the ideas and then using a model for the prose. To me, it is an invitation to examine the matter more closely rather than a judgment.

Feel free to ask me anything about the setup. And yes, I ran this post through the detector before submitting it.

Comments (1 of ) ⌛

Loading more comments... ⌛
Scroll down to load more comments
No comments available.
Type to search HN and engineering blog articles
Press Escape to close  |  ↑↓ to navigate  |  Enter to open