No More Network Video! Please
We have managed to secure device (/dev/sda1 10GB:/srv) to keep some movies and a basic image of Debian Stable (lenny) for bootstrapping Xen instances of domU.
The deployment target is (/dev/sdc1 10GB:/home) where the disk is ext3 with reasonable permissions, and the space has been liberated from network video users.
Our system has a dangerous (/dev/md0 446GB:/) that is probably pretty safe formatted as XFS, with something like 100GB free thanks to a lost 100GB filesystem that was hosting a system running Ubuntu.
So, we ought not count on that 446GB! Instead, find a way to utilize that 100GB such that it can be "snapped" and utilized en-masse by another system, like the 120GB guest OS living on my VMware Infrastructure disk.
When we have another system somewhere with 500GB that we can trust, until then, scripted deployment is king and federated virtualization is a pipe dream that we need not realize now or in the near future.
There is no need to copy hundreds of gigabytes! Your application is insignificant.
Wednesday, March 18, 2009
/home (exported from irie-arch:Tomboy)
Posted by
yebyen
at
8:49 PM
0
comments
Labels: data retention, z
Sunday, October 28, 2007
Tackling Tickets
These are all of the things that I mean to do in the next short span of time. The plan is to run down the list and generate a file for each issue, then either prepare a note on my schedule to remind me of the time for action, or write another note expanding on the task at hand, or mark it as thoroughly completed. First priority is to fulfill my government-mandated obligations, I have two tickets on the books.
- Judge in Avon
My own observation of the speedometer indicated my speed was between 80 or 82 MPH, though admittedly my first reaction upon seeing the reaction of the cloaked state trooper to my passing was not to check the speedometer, but to reduce my speed and prepare to pull over.
- Pay Bills (Check Bank Balance)
- Set up Home Office at 204-1
- Clean out Trunk of Car
- Take out the Trash
- Do the Dishes
- Video Software
- HP Warranty Status
End result: I bought a new hard drive and put it into my Dad's computer. A little white lie, I said it was covered under warranty, but really I couldn't stand to spend another 3 hours on the phone with HP to prove that the drive was faulty and get a replacement shipped.
We've got til February to orchestrate a catastrophic provable failure on this drive and get a new one under warranty. Maybe I can get Dad to load it up with porn and keep writing to it over and over again?` It probably will not crash on its own, hooked up in configuration as a secondary drive like it is now.
Would you ever know if videos, instead of system data and configuration files, were losing bytes due to corruption? There's no reason the system should crash because of incoherent data files. Is there any great symbolism to the idea of corrupting pornography? Kind of funny I guess, ha ha ha.
The unhappy conclusion to this story is that my new cheaper HP Slimline does not seem to have user-serviceable parts. I have no idea how to remove the hard drive, and if I had to replace it with another from Western Digital working on my own, I think I would become a slave to the HP warranty timetable. That sounds awful.
- NYS DMV License Plate Replacement for CVP1942 - Stolen?
Posted by
yebyen
at
10:28 PM
0
comments
Labels: data retention, english language, getting things done, issues of record, kingdon, The Causemaker Foundation
Thursday, June 14, 2007
Network Detour
Taking a quick detour to work on multi-homed storage arrays as I think this might be quite handy, with applications from off-site backup, to decreasing access times for valuable data (like FreeBSD ISOs). I mean to duplicate the specialized functionality of the Andrew File System, without quite so much focus on coding and maintaining one unified file-system -- here I hope to demonstrate that Andrew goes too far, and he does so at his own peril.
In other words, we can fudge it without losing very much functionality ... we will lose customer-oriented attention to detail in service presentation. No longer will this have the appearance to random users of being a unified file-system. I was hoping to keep random users in the loop as much as possible, but at this point I've been struggling with some of these issues for so long that I'm going to throw up my hands and answer, "this is my storage cluster. ask for help if you need anything."
I'd rather administrate a distributed storage cluster than a team of researchers anyway! So for now I will endeavor to fill my terabytes with something of value, and to make it accessible through the regular channels, whatever those may turn out to be. Ubuntu Mirrors listings and FreeBSD as well are in the sixth layer delicious notes for the day. Will probably be a few days before I can work out the kinks in the scripting and data inventory process.
I think I'll add a website with links to the data and a nice writeup about this joint interest of Tuesday Studios, Venture Creations, and the Rochester Institute of Technology. Hopefully I can justify some of the money that I have spent, and maybe if I get lucky I can get my activities tacked onto somebody's budget without too much struggle.
Today I am pretty sure that either of these commands can be used reliably inside of a cron job to create a mirror:
$ rsync -vaz --delete [rsync url] [local directory]
$ lftp -c mirror [remote [local]]
There are mechanisms to reverse the direction of this operation; see man rsync or lftp -c help mirror for more details. As long as we are only mirroring things that are already public data, then this is all you need to duplicate the effects of the AFS feature of remotely replicating read-only cells.
Unfortunately today from where I stand, it looks like there will never be a means of simple public read/write access to such a large disk. There are too many copyright issues, this is risky business -- if we're not careful about access control, we might even create something of value that doesn't technically belong to anyone, and then where will we be? Oh yeah...
This sort of policy can be enforced through other means, like access control lists on the file-system itself, or some other restriction implemented through the protocol or the server. Next step for these gigantic public mirrors are to actually make them publicly accessible and start advertising through the usual channels.
I think that I will configure both anonymous FTP and HTTP access, but I suspect that I will change my mind and use HTTP only when I get lazy. What is the value of FTP? Hmm... a convenient interface, in fact if I configure one machine with access to all of the pieces-parts of the whole storage cluster, then this might start looking like a unified file space again.
It's probably worth keeping FTP around.
Posted by
Kingdon
at
6:34 PM
3
comments
Labels: andrew file system, data retention, The Causemaker Foundation
Saturday, June 9, 2007
Data Retention Plan
You want a data retention plan? I'll give you a data retention plan. Open up your organization! Depend on free services! Put everything you own on the internet, and you will be just fine! Trust me ;-)
In related news, there's a new class of student companies joining us at Venture Creations, the business incubator sponsored by RIT, which houses Tuesday Studios and the server room! In reality I do have some servers of my own, and I have really put a lot of consideration into data retention over the last 8-10 years as I have watched my own precious data occasionally slip through the cracks when tossing an old computer to the curb. First step is to centralize your data and then give it to me... I'll take good care of it for you!
If that doesn't sound attractive to you, then my next recommendation is to give your data to Google. If you're worried about data mining, no, I'm sure it'll be MUCH safer with Google than in my own server rooms... the world's foremost experts in data mining? Nah, I really don't think they'd try anything too fishy.
Still, if that sounds too much like charity, or you like the idea of doing things in house, I am always happy to pay a visit and perform a consultation, or give a tour of my own server room. I could fill at least 1 hour's time talking about where I keep my data, and the measures that I employ to keep it safe.
In fact I will even go so far as to say, free consultation. Bear in mind this may be a limited time offer; I could get very busy tomorrow. You know, this is not a contract to work at no cost for the rest of my days.
I was going to start digging up information on those new companies at the incubator! Tune into my del.icio.us feed for the scoop, I'm going to see what Google already knows. Hopefully I'll meet some of these people soon and post again to tell you how cool their services are; I don't like to interrupt people when they're working, and everyone tends to look busy when I walk by their desks. I wonder if that's because my card says President?
Posted by
Kingdon
at
1:24 AM
0
comments
Labels: consultation work, data mining, data retention, google, kingdon, outsourcing, president, venture creations