Year 5 Day 3 - File Management

I rebuilt my PC a couple months ago - my old one was a few years old and had some hard drive issues. Luckily I had backups of all my data -actually it wasn't luck, I make sure to have multiple copies of everything.

Read on for more.

I won't go in to too much detail but for the sake of this post - when I get home from shooting somewhere if the day was awesome, I off load my cards to my pc, rename files, and make a backup copy of the files, and then move the files to current location for originals.

I try to make sure my 2nd copy of the images is either DVDs or now, a second hard drive which is NOT in the same PC as the originals. I use a 1TB NAS which is full, and another 2 x 1 TB external USB drives.

So, lately I've been juggling around my files and trying to organize them better. I think I will use some of this month off from flickr to redo my scheme from scratch.

I started to use Lightroom to do all my file management a few months ago. Picasa would just consistently crash and erase the entire database of images and have to rescan everything. I know it was because I had too many files, well over 200,000 images / originals, and maybe over 300,000 files since I had shot jpgs + raws very often. And I have each high level folder as its own Lightroom catalog to keep the catalogs somewhat manageable.

The problem is that stuff is now all over the place, the catalogs are roughly:
1) 2005 through part of 2008.
2) Part of 2008 through part of 2009.
3) Most of 2009.

And for the 3rd grouping I began to store the folders from each day in a parent folder for the location. Which is good for the new stuff, but now my old stuff that is just sorted by day isn't part of this setup.

One of the challenges I face now is to reorganize the first two catalogs in to groups by location and then day. Many many of those day-folders are for multiple locations-days when I went to two or three spots. And lots of the folders are named things like "2006_07xx_MDMisc" which might be multiple days and a variety of stuff. I guess I can sort through it all, but THEN the matching backup copies are going to be differently named folders.

I could redo the backups, by deleting the backups and re-copying the freshly organized folders but if my main copy (the hard drive) dies during that process I might lose ALL the copies.

I might just have to buy another couple 1 or 2 TB hard drives so I can safely juggle EVERYTHING around.

One of the roles I have at my day job is data management and protection strategies and technologies, and testing and vetting it from end to end. The last thing I want to EVER happen is to lose some of MY DATA! So that is why this might all seem a little overkill.

--

And then I need to determine the impact of moving files around from one catalog to another , if those folders had lots of editing applied to them in LR. Most of that is just recent stuff from the past few months, so it might not be that bad. I could lose all the editing work and not be too upset. I always make my edits, and then export a full resolution jpg from Lightroom.

---

If I do go through all this hassle to move files, rename folders, regroup folders I want to do it right. A couple of the possible plans I have right now are:
1) Sort everything by year, and be done - folders will then be for a location and day (never 2 locations per folder). Keep a LR catalog per year.
2) Sort everything by location, and for places like the DC Zoo, or Great Falls, where I've been 50 or 100 or 200 times I'd make that get its own LR catalog.


And something that just now occurred to me, I need to develop a plan for storing my HD video files.

All that, and I plan to swap out my temporary install of Windows XP for Windows 7 in a month or so. That's going to rock - my AMD 3Ghz 4-core CPU with 8GB or RAM and 5+ TB of disk! Woot.

Ok, I've got my work cut out for me, and I won't stand for losing any data...

-Jon

That's enough to make your head spin huh?

Year 5 Day 3 - File Management