145+ Petabytes for a single copy of the archive and they currently have two copies of everything for a total of around 290 Petaybtes.
The largest hard drive I’m aware of is 32TB so you’d “only” need over 9,000 (lol) of the largest drives ever made. I can’t even tell you what that would cost since Seagate doesn’t have a publicly available price for the damn things!
And it had to be replicated, so 3 copies somewhere (granted proper backups are compressed).
Let’s say they have a proper backup compressed at (a random) 60%. That one backup is 87 petabytes. With daily incrementals, so another what, 14 PB to get through 2 weeks of incrementals? Something in the range of 600 PB total with replicas?
(I’m completely pulling numbers out of my ass, I’m not familiar with how such large datasets are managed from a DR perspective).
You’d need more than 9,000 of the largest hard drives made (32TB) to store the nearly 300 Petabytes of data they have. Still within the reach of an obscenely rich tech bro but not exactly cheap.
And, you’d want/need redundancy. One on-site back up for quick restoration and one off-site for surviving physical disaster. So, you’d need at least 3 times that. In HDD prices, that is roughly 2.5 million per set-up, or 7.5 million total for all three. And in SSD prices, well it’s about 3x that. 7.5 million per set up and 22.5million for all three.
An alternate option is a distributed back-up. They could have people volunteer to store and host like 10 gigs each, and just hand out each 10 gig chunk to 10 different people. That would take alot of work to set up, but it would be alot safer. And there are already programs/systems like that to model after. 10 gigs is just an example, might be more successful or even more possible in chunks of 1-2 terabytes. Basically one full hard drive per volunteer.
Lol, had to add that after doing the math for 10 gigs to ten people and realising that was 1000 people per terabyte, so would take 150 million volunteers. Even at 2 petabytes each, assuming we still wanted 10x redundancy in that model, it would be like 750 thousand volunteers or something like that. Maybe there is no sustainable volunteer driven model, lol.
That click bait title makes it seem like the Internet Archive itself will come crumbling down. Now that I skimmed read the article, it just seems like they are going to lose the court battle for this case. Maybe some other stuff, but I get to keep my MS-DOS abandonware downloads.
They’re facing hundreds of millions in punitive damages. Fragments of their servers will be in trophy cases in copyright enforcement agencies’ lobbies.
Sounds like this would at the very least remove $20-30m in revenue (according to the article) not counting if the copyright holders decide to go after them.
I think it puts them in a precarious situation overall.
Reading the article and I wonder what Internet Archive thought when they turned themselves into a Pirate Bay but for ebooks. They had this lawsuit coming and they have obviously no idea how to defend themselves in this case.
They should stick to archiving the web instead of shooting themselves in the foot.
I've been saying this for years, this was an incredibly boneheaded move by the Internet Archive and they just keep on doubling down on it. They shouldn't have done it in the first place. When they got sued, they should have immediately admitted they screwed up and settled - the publishers would probably have been fine with a token punishment and a promise to shut down their ebook library, it's not like IA cost them anything significant. But they just keep on fighting, and it's only making things worse.
This isn't even IA's purpose in the first place! They archive the Internet. They're like a guy who's caring for a precious baby who decides he should go poke a bear with a stick, and when the bear didn't respond at first he whacked it over the nose with the stick instead. Now the bear's got his leg and he's screaming "oh no, protect my baby!" And it's entirely his fault the baby's in danger.
They really fucked up and it’s so heartwrenching watching it all happen. I was following the story since it started and I just can’t believe they allowed anyone to download copyrighted books without a limit in 2020, without asking anyone for permission or whether it’s legally viable. Everyone knew they were losing this, and they gave publishers a convincing reason to sue them. by crossing the “legally grey” area to literal piracy.
FWIW, OpenLibrary is a good source of book metadata at least, even if it fails its goal of letting people read books on it.
Unlimited free distribution of copyrighted media is something I’m all for, but that’s a really tall order in terms of political capital for getting the law changed, a few protests aren’t going to do it, basically every elected official would strongly oppose it, you’d have to replace them all first.
Full access to currently published copyrighted books was way too much. Even Google just showed snippets or showed abandonware books. They really should have settled.
The Internet Archive created a program they called “Controlled Digital Lending” (CDL) – where a physical book is scanned, turned into a digital file, and that digital file is then “loaned” out to people on the Internet. In 2020, The Internet Archive removed what few restrictions existed with this Digital Lending program, allowing an unlimited number of people to download the digital copy of a book.
That was incredibly dumb. It’s just baffling that they didn’t run this by a lawyer or even a highschooler. They put all their good work in jeopardy for what exactly?
I still believe that serious hoarders who want to ensure other can access it should try to do so anonymously. IPFS, torrents, or something like eMule over I2P would be the best solution for this. Then copyright lawsuits like these would be very very very difficult to do without de-anonymizing people.
The books they loaned are loaded with DRM though, which make it unusable after the loan period expires, so it’s not like they’re handing out unlocked pdf en-masse like z-library. They probably thought this restriction was good enough and publishers have enough goodwill to let it slide during the height of the pandemic.
For me it’s not boot licking but recognizing that IA made a huge unforced error that may cost us all not just that digital lending program but stuff like the Wayback Machine and all the other good projects the IA runs.
Exactly. Less bootlickering and more “Leopards Ate My Face” material. This was heavily forseable. If they did this as a protest against copyright and announced it from the start, it would be one thing. But this was just incompetence and ignorance at a level that will likely ruin them.
How is licking the boots of the copyright office helping the people? I don’t give the first fuck about individuals trying to hoard up IP anymore; IP hoarding-- and the very concept of IP in the first place-- is fuckin bullshit, fuck 'em if they got a problem with that take. We already have to pay through the nose for too much else in society, so why are you licking copyright-uplifting boots?
Authors have the right to be compensated for their work. There are many things that are wrong with current copyright, but denying artists compensation for their work is not the way.
Wouldn’t you also be mad if your salary was denied?
I live every day of my life under a regime that steals my wages, calls it ‘profit’, and then in my face, gets away with pocketing it, with STEALING IT; that shit is ALREADY happening. At this point, it’d just even the playing field. If you want me to have solidarity with my fellow worker, you cannot have it both ways where you expect me to uplift the tools of capitalist theft too. Fuck out of my face if all you have is platitudes to capital.
And your attempted redress is bootlicking the mechanics by which capitalists steal from all of us. Sinclair was right; it really is impossible to get you people to understand shit when your material conditions are dependent on your not. And then you just smug around like you said some prosaic shit I am so sick of y’all.
Whilst I’ve heard lots of talk that lunduke is getting increasingly politica, and I disagree quite strongly with his politics, I’ll have to agree with him here. IA did something unnecessarily risky (redistributing unauthorised copies of print books), which has more jeopardised their mission of archiving the internet.
I also agree with everyone here saying that current copyright laws are ridiculous (and not just because they are “outdated”, the Victorians had better copyright laws than we do). However, I think only the most radical overhaul of copyright law would condone what IA did, and that isn’t coming any time soon (If ever).
I’ll give the parent poster the benefit of the doubt and say that it’s a satirical user account. It’s called spez_, named after the reddit CEO, I’m guessing.
The problem is an obsolet copyright law, made by big companies and elderly politicians confusing a remote control with a smartphone. Money is killing the culture and the knowledge.
Does this eBook downloading thing affect the rest of the Archive? Like, will the entire archive be affected or just the OpenLibrary part of the Archive?
lunduke.locals.com
Oldest