H
Did you know that 99% of email today is spam? Your inbox isn’t 99% spam because AI is used to filter it. The same 99% will happen here too, but if AI researchers continue to get perma-banned for making available the datasets needed to filter it, it’s going to make this platform unusable.
19,603,210 views
I
Replying to
@howard
This is a much more interesting argument. i think it would be more amenable if people were actually building those products right now, would be great if someone could demo some of this stuff! Based on some posts, a reply guy filter would be incredible
38,450 views
I
Replying to
@isaiahbishop
like if we want to build some community trust here, we should put up some use cases that show its worth something, and those uses are public and good faith
38,450 views
N
Replying to
@isaiahbishop
This feels like a classic chicken-and-egg situation. For example there is a Twitter dataset on the 2020 US election, which has been used in 100+ papers on topics like misinformation detection. But it probably also ended up being used for projects it wasn't meant for.
38,450 views
I
Replying to
@noone
Sure but you can build whatever else on other datasets and be able to say "i need to fine tune this" and will make it available to you, because public good for public good Randos putting stuff into HF is maybe less good than the twitter dataet right?
38,450 views
N
Replying to
@isaiahbishop
I’m not fully defending them, but I wouldn’t say I’m completely against creating datasets either. I think the way it was handled lacked reflection and could have been approached more thoughtfully.
38,450 views
I
Replying to
@noone
Yeah im also on the fence for it, but tbh im trying to steelman why this needs to exist on HF and i dont see why it would need to be. Like what purpose does this serve
38,450 views
H
Replying to
@isaiahbishop
It's a mild convenience for researchers. Rather than needing 4 lines of code to grab the firehose, it's two lines of code. And you don't need to leave it running for a few days to collect. Making it easier for researchers to build useful tools seems like a noble goal to me.
38,450 views
N
Replying to
@howard
Depending on the dataset, it is more than just a mild convenience. Extreme example is the 2016 US election dataset had 280 million tweets. With Twitter’s API limitations it would take 32 days to retrieve the full dataset. (IF all tweets would still be available)
38,450 views
N
Replying to
@noone
Other obvious downsides, beyond being time-intensive: It requires knowledge of the API or software (not obvious for e.g. social scientists) and data entropy as tweets get deleted over time, making comparisons across studies difficult.
38,450 views
Replies
N
Replying to
@noone
That being said, I nonetheless think the best approach is publishing IDs that point to the entries, not complete datasets. We cannot expect people wanting to delete their content to contact everyone who made these datasets public. The burden should be on those using the data, not the data subjects.
38,450 views