Backing up 1TB of data (1 Viewer)

the_net_2.0 · Nov 27, 2010

would anyone here have some advice on how I could back up about 1TB worth of media files to an external hardrive? I'm going to buy an additional drive on ebay, if I can find one, but I'm not the best way to go about doing this.

a simple copy and paste? I have a program on the machine called TERACOPY, which supposedly speeds up the copy and cut processes. It looks like it does, but with 1TB of data over 1500 files, it won't matter too much I'm sure.

Anyone have any advice on how to do this? In spurts? 50GB at a time? through DOS? I like copying and such through DOS because it's generally faster, but with this amount of data, I have no idea. Suggestions certainly welcome!

DCrake · Nov 27, 2010

I use teracopy all the time and have never come across any issues. What I like about it is you have a more informative interface showing you what is going on instead of a simple progress bar and those magical Windows Minutes remaining status....

the_net_2.0 · Nov 27, 2010

Yes, but copying THIS much data? Logically, if no other processes were running, it would take well over 5 or 6 hours, I would assume. NO?

I wonder what the chances of corruption of the copies or the source is too?

DCrake · Nov 27, 2010

What type of connection have you got between your External device and the PC? wireless, Lan, USB Rsr232 com to com?

the_net_2.0 · Nov 27, 2010

Ideally David, it would hard wired. I haven't bought it yet. I'm going to try and find one on Ebay. And yes, if it was wirelessly connected, there's no way I would attempt to copy these files. too much risk in airwaves.

DCrake · Nov 27, 2010

Hardwired is the best, and like I say the TeraCopy does things 1 at a time, not like Windows that does it in chunks.

I would be tempted to copy folder by folder. Then a least be doing one folder you can guesstimate the overall copy time, and it gives your prosessor time to clear its cache after each cycle.

the_net_2.0 · Nov 27, 2010

the entire 1tb of data is in one folder David.

how long would you guess this would take with teracopy?

DCrake · Nov 27, 2010

Find a file that is of measurable size and do a copy then repeat the exercise for consistancy then you can calulate the overall copy time. Otherwise set the hole lot going overnight.

however this is a good opportunity you subdivide your folder in to A-Z media names.

the_net_2.0 · Nov 27, 2010

well the subdivisions will be a pain david, so I don't think I'll do that, but I can move them into subdirs, maybe 100 at a time or something, with VBA. I know how to do that. then move back when the copying is done.

Galaxiom · Nov 28, 2010

I would be looking toward eSATA. This has a transfer rate of 3 Gigabits per second.
Otherwise USB3 which is in a similar league.

However despite their speed adding this facility to most computers not specifically designed to include it won't achieve this performance due to deeper bottlenecks.

Rather than copying the files to folders with VBA it would be more sensible to drive teracopy from VBA if that is possible. I have not seen teracopy so I don't know if it has a commandline interface.

Another alternative worth a look would be Microsoft's Robocopy. It is commandline driven so would be easy to use with VBA. It recovers from errors and has full logging available.

Also has some good switches. I like the one that logs what would be copied without actually doing the job.

The is also a GUI for it. All free.

the_net_2.0 · Nov 28, 2010

GalaxiomAtHome said:
Another alternative worth a look would be Microsoft's Robocopy. It is commandline driven so would be easy to use with VBA. It recovers from errors and has full logging available.

Also has some good switches. I like the one that logs what would be copied without actually doing the job.

I'll take a look Galax. thanks. btw, now that you're on the command line kick, how do you install .msi packages? I've only ever done it through the DOS "msiexec" command. Is there another way? just curious.

And I've heard about eSATA. Aren't those connected to motherboards somehow though? Could I get a drive and simply plug it into one of the COM ports on the outside of the machine like we do with our mice or keyboards?

Galaxiom · Nov 28, 2010

I install msi packages from DOS.

Incidentally, some Microsoft packages can extract an msi from the exe installer with a switch on the exe execution.

eSata is a special port. They seem most often found on laptops but are beginning to show up on desktops. It won't work in a USB port. With the introduction of USB3 (4 Gb/s) they may not become ubiquitous. They are nice in that a Sata drive can work without an interface.

BTW. Although USB ports show up as COM ports in Windows, old blokes like me still think of COM ports as Serial ports. Long ago they were the only port but at 33kb/s most PCs don't even bother to include them now. However many motherboards still have them available as a nine pin header.

wiklendt · Dec 15, 2010

i like the command prompt for this sort of thing.

windows key + R
cmd ENTER
then i generally use something like:

Code:

xcopy "C:\Users\Me\Music" "D:\BackupFolder" /C/D/I/K/F/S/Y >>"D:\BackupLog.txt"

this will list each and every file that it copies in the log file. if you want to know what the switches are (or which others you can use), type

Code:

 xcopy /?

in the command window.

obviously, replace the source and destination paths with your own.

edit: then, once finished (get it to run overnight, or something), right-click|properties on both source and destination folder and check the bytes are the same (not the size on disk, but the size only).

Galaxiom · Dec 15, 2010

xcopy has a weakness in that it does not recover from errors. Robocopy doesn't give up the whole job when it encounters an error. It is ideal for copying across networks.

Robocopy (robust copy) has all the same functionality as xcopy and more. It can optionally delete files from the destination.

It is part of the Server 2003 Resource Kit but it also works on XP. Technically you do have to own a Server 2003 licence to use it.

the_net_2.0 · Dec 15, 2010

good info folks. I try to use DOS whenever I can, simply because it's less work than windows, but obviously I don't know it front and back. I will be using an external hard drive for this. as a matter of fact, what I think I'm going to do is buy one and simply use it for storage. that will free up the 500+ gb of space that is currently used on my home machine.

and I'm not sure about the piping wiklendt. That seems like a lot of work. But regardless, I appreciate the info. and for switches a purposes of them, I recommend computerhope.com

That'll tell you everything you want to know about dos and more.

wiklendt · Dec 15, 2010

sorry, i probably should have said that "right-click|properties" is from windows explorer.

@Galaxiom: i've heard of robocopy but never used it. i think i checked it out once and it didn't do something i wanted, so i stuck with xcopy. might be time to re-visit robocopy. thanks for the push

wiklendt · Dec 15, 2010

ok, i just looked up robocopy help via command prompt and i SWEAR it didn't look that flexible and feature-full last time! maybe i am confusing it with something else? anyway, i'll give it a go

wiklendt · Dec 15, 2010

the_net_2.0 said:
and I'm not sure about the piping wiklendt. That seems like a lot of work. But regardless, I appreciate the info.

hellava lot less work than copying manually folder by folder. or in 50 mb chunks.

one line's not a lot of work at all. i wrote a 900 line batch file to handle my backups, which i can run at anytime now with a bunch of options, so a one-liner is really quite a no-brainer. (edit: i thought you said you liked using the command prompt??)

the thing i like about that xcopy command (with those particular switches) is that if you need to re-run it at a later date (which is easy if you save it as a .bat file), it ONLY copies new or updated files, it doesn't do all of them all over again.

Access_guy49 · Jan 5, 2011

No one has mentioned MS Synctoys, which I personally Like as a simple and quick "backup" application. If you have to stop the process at a given time and come back to it later, It wont waste a ton of time trying to overwrite the same files. plus you can choose which way to move files in the future, you can append to a disk with new files, create a duplicate copy, or even have it where it does a sync and takes newest file from either drive. Plus you can get into specific subfolder sync if needed.
It's pretty good, it's free and runs with interface or command line.

wiklendt · Jan 5, 2011

Access_guy49 said:
No one has mentioned MS Synctoys, which I personally Like as a simple and quick "backup" application. If you have to stop the process at a given time and come back to it later, It wont waste a ton of time trying to overwrite the same files. plus you can choose which way to move files in the future, you can append to a disk with new files, create a duplicate copy, or even have it where it does a sync and takes newest file from either drive. Plus you can get into specific subfolder sync if needed.
It's pretty good, it's free and runs with interface or command line.

oooo! i didn't know about synctoys... will have to check it out. thanks access guy!

as for robocopy (from earlier in the thread) i had a look at the scripting required for it, and it's quite a bit more complex than xcopy... there are GUIs out there, but they don't actually explain any of the switches, so you'd still need to learn them anywyay.

Backing up 1TB of data (1 Viewer)

Banned

Remembered

Banned

Remembered

Banned

Remembered

Banned

Remembered

Banned

Super Moderator

Banned

Super Moderator

i recommend chocolate

Super Moderator

Banned

i recommend chocolate

i recommend chocolate

i recommend chocolate

Registered User.

i recommend chocolate

Similar threads

Users who are viewing this thread