Musk sets tweet-viewing limits to combat data scraping from large AI companies

In a bid to combat data scraping, Twitter is changing its policies, which will directly affect the number of tweets users will be able to read in one day.

For new unverified accounts, the number is 500. Meanwhile, accounts with the verified status are limited to 10,000 posts a day at present. This move was announced by tech billionaire Elon Musk, who initially set stricter limits, but changed them within hours of announcing the move.

Musk said the temporary limits were to address “extreme levels of data scraping and system manipulation”. He did not explain what was meant by system manipulation in this context.

“We were getting data pillaged so much that it was degrading service for normal users,” Musk explained on Friday, after users were presented with screens, asking them to log in to view Twitter content.

The move was described as a “temporary emergency measure”. Musk has not directly described what he means by preventing data craping. The move is could be to target AI enterprises, especially large AI companies that need huge amounts of data to train large language models, which power chatbots such as Open AI’s ChatGPT and Google’s Bard.

Data scraping

In simple terms, data scraping is the pulling of information from the Internet. Large language models need to learn from masses of real human conversations. But the quality is vital to the success of a chatbot. Reddit and Twitter’s huge trove of billions of posts are thought to be hugely important training data and used by AI companies.

Social-media companies such as Reddit and Twitter now want to be paid for this data. In April, Reddit’s chief executive Steve Huffman told the New York Times that he was unhappy with what AI companies were doing. ”The Reddit corpus of data is really valuable,” he said. “But we don’t need to give all of that value to some of the largest companies in the world for free.”

Ajimon Francis, Managing Director at Brand Finance India, said the move may not affect users and is aimed at enterprises. “Musk is seeking opportunities to monetise the platform better and turning off the tap on freebies. The tweet limits are too high to affect users.”