There have been multiple accounts created with the sole purpose of posting advertisement posts or replies containing unsolicited advertising.

Accounts which solely post advertisements, or persistently post them may be terminated.

What is the largest file transfer you have ever done?

I’m writing a program that wraps around dd to try and warn you if you are doing anything stupid. I have thus been giving the man page a good read. While doing this, I noticed that dd supported all the way up to Quettabytes, a unit orders of magnitude larger than all the data on the entire internet.

This has caused me to wonder what the largest storage operation you guys have done. I’ve taken a couple images of hard drives that were a single terabyte large, but I was wondering if the sysadmins among you have had to do something with e.g a giant RAID 10 array.

bulwark ,

I mean dd claims they can handle a quettabyte but how can we but sure.

data1701d OP ,
@data1701d@startrek.website avatar

dd can’t really handle quettabytes! GNU has taken us all for fools! Alert the masses! Wake up sheeple!

https://startrek.website/pictrs/image/3fc3b3cd-7ad4-4ccb-bfad-90cc2b109c88.png

Neuromancer49 ,

In grad school I worked with MRI data (hence the username). I had to upload ~500GB to our supercomputing cluster. Somewhere around 100,000 MRI images, and wrote 20 or so different machine learning algorithms to process them. All said and done, I ended up with about 2.5TB on the supercomputer. About 500MB ended up being useful and made it into my thesis.

Don’t stay in school, kids.

mEEGal ,

You should have said no to math, it’s a helluva drug

data1701d OP ,
@data1701d@startrek.website avatar

m.youtube.com/watch?v=6xi2Bl50XPc

southsamurai ,
@southsamurai@sh.itjust.works avatar

I think 16 terabytes? Might have been twelve. I was consolidating a bunch of old drives and data into a nas for a friend. He just didn’t have the time, between working and school and brought me all the hardware and said “go” lol.

ramble81 ,

I’ve done a 1PB sync between a pair of 8-node SAN clusters as one was being physically moved since it’d be faster to seed the data and start a delta sync rather than try to do it all over a 10Gb pipe. M

hperrin ,

I transferred my entire NAS storage, which includes all of my backups, cloud files, my family’s backups, and my… Linux ISOs. That was about 12TB.

Nomecks ,

I did 100TB, 100 streams of 1TB, all simultaneous with rsync

pixeltree ,
@pixeltree@lemmy.blahaj.zone avatar

I once deleted an 800 gb log file, does that count

Urist ,
@Urist@lemmy.ml avatar

I obviously downloaded a car after seeing that obnoxious anti-piracy ad.

averyminya ,

I once robocopied 16tb of media

MajorHavoc ,

I’ll let you know… If it finishes.

avidamoeba ,
@avidamoeba@lemmy.ca avatar

~15TB over the internet via 30Mbps uplink without any special considerations. Syncthing handled any and all network and power interruptions. I did a few power cable pulls myself.

pete_the_cat ,

How long did that take? A month or two? I’ve backfilled my NAS with about 40 TB before over a 1 gig fiber pipe in about a week or so of 24/7 downloading.

avidamoeba ,
@avidamoeba@lemmy.ca avatar

Yeah, something like that. I verified it it with rsync after that, no errors.

d00phy ,

I’ve migrated petabytes from one GPFS file system to another. More than once, in fact. I’ve also migrated about 600TB of data from D3 tape format to 9940.

Laborer3652 ,

I have to copy ~8 TiB of backup files twice a year.

Krafting ,
@Krafting@lemmy.world avatar

Rsynced 4.2TB of data from one server to another but with multiple files

Davel23 ,

Not that big by today's standards, but I once downloaded the Windows 98 beta CD from a friend over dialup, 33.6k at best. Took about a week as I recall.

absGeekNZ ,
@absGeekNZ@lemmy.nz avatar

Yep, downloaded XP over 33.6k modem, but I’m in NZ so 33.6 was more advertising than reality, it took weeks.

pete_the_cat ,

I remember downloading the scene on American Pie where Shannon Elizabeth strips naked over our 33.6 link and it took like an hour, at an amazing resolution of like 240p for a two minute clip 😂

  • All
  • Subscribed
  • Moderated
  • Favorites
  • •
  • random
  • [email protected]
  • lifeLocal
  • goranko
  • All magazines
    •