Re: expanding ram
- From: Paul <nospam@xxxxxxxxxx>
- Date: Tue, 27 Nov 2007 16:10:00 -0500
carlbernardi@xxxxxxxxx wrote:
expanding ram
Hi, I have question about expanding ram.
Presently, I am working with some very large databases( in the 500
gig range) and I am searching for an inexpensive way to have the
entire databases in memory. Given that my knowledge of hardware is not
that in depth I was wondering if someone knew the best way to expand
memory to avoid using hardrives which are much slower than ram.
I am considering:
USB
FIREWIRE
BLUETOOTH
and any other possibilities.
Since cheap mother boards can only hold up to a few gigs unless
you can afford a super computer, I was thinking about having my
databases put on USB flash drives attached through a USB hub * 2
(because must computers have 2 usb ports) adding for example 16gig usb
drive * 2 hubs * 4 ports per hub is 128 gigs. I don't think there is
any common mother board out there that can hold that kind of ram.
Linux Ubuntu Fiesty is the operating system.
Any suggestions?
Thanks,
Carl
RAM is the fastest storage method. RAM is volatile, and must be
battery backed, in case the power fails suddenly. Of course, a
hard drive should be used to hold the backup copy, and some
designs would incorporate the backup device in the design.
At the basic cost level, RAM is $25 per GB right now. To have 500GB,
would cost $12500. For the chassis to hold all that RAM, I'd throw in
another $5000 minimum hardware cost. Companies who make real SSDs
based on RAM, charge a lot more than that. Rounding up and leaving
a tiny bit of room for expansion, means you're looking at $20000 of
materials if you design it yourself.
A RAM SSD has close to zero seek time (a couple microseconds might
be typical). The bandwidth achieved is a variable - commercial devices
might be connected by something crappy, like say Ultra320 SCSI, which
spoils the peak numbers that might be achieved. You could get 4GB/sec
via a PCI Express slot, and a good design would attempt to tap
into that kind of interface.
An ordinary hard drive, has an average seek time which
is dominated by the rotational latency (average is 1/2 the time for
one full rotation of the disk - sometimes you get lucky, and the
sector you want is right next to the head). So to seek to a
random sector on the 500GB hard drive, might take thousands of
times longer than the RAM SSD.
So the next question you might ask, would be " what if I used a RAID array
of ordinary hard drives ? ".
A RAID array is an array of disks in parallel. That achieves a higher
bandwidth, but it doesn't do anything about the seek time. The heads
of all the disks, still need the same time to change cylinders, and
they all arrive in about the same time as a single disk.
There is another solution. There are SSDs based on flash memory.
Such an SSD has a reduced bandwidth compared to ordinary RAM. Some
of these devices have abysmal write rates (slower than a hard drive),
but on the upside, the seek time can be better than a hard drive.
The abysmal read/write is why they have to be researched so carefully.
You can see in the product listings here, a vast range of capacities
and prices. For your application, you'd want to find the pricing
sweet spot, to build a storage subsystem as cheaply as possible.
Read the comments and reviews - I don't know if too many of these
products are making customers happy.
http://www.newegg.com/Product/ProductList.aspx?Submit=ENE&N=2003240636&Configurator=&Subcategory=636
Using flash based devices for this task, is not nearly as easy as you
might think. Many seek to hide their performance issues, causing
customer to have to buy and test, to find the good ones. For example,
if a device had a write performance of 8MB/sec, you'd be pissed.
Flash SSDs also have a wearout issue. Good ones make use of a wear leveling
patent. The patent is owned by one company, meaning what are the
other companies doing ? Wear leveling is intended to spread updates
to physical sectors, so that the max writes to a given sector are
not quickly exceeded.
You might also ask the question " could I put a bunch of SSD flash
devices in a RAID array ? ". Yes, you could, but the tuning parameters
would have a critical effect on performance. The write speed of
the little flash devices, changes as a function of the size of the
transaction. I'm not knowledgeable enough, to suggest what array
formats might improve the bandwidth aspect of things. After all,
if parallelism was so easy, they'd already be using it inside some
of those SSD drives.
Again, just making up some fake numbers, if a 32GB flash drive is $300,
I'd need 16 of them. I'll pretend I can find a 16 port RAID card
for $1000 (there are SATA ones like that, but the SATA form factor
SSDs are not as cheap as they could be). My material cost then
is about $5800. Better than the $20000 I was planning on spending
on an SSD based on RAM. My assumption is, that the items in the
database are small, so bandwidth is not really an issue, and seek
time is all that matters.
You could try USB devices, because they're pretty cheap, But a lot
of those probably have no wear leveling. You can fan out USB interconnect
with powered hubs, to achieve a large set of connected devices.
The 60MB/sec on the root controller, means there is not a lot
of bandwidth to go around, but if seek time is all that matters,
maybe it won't be that bad. You'd need 16 of these, and I don't
see a transfer rate spec for these - 16*280= $4480 plus the USB
hubs.
CORSAIR Flash Voyager 32GB Flash Drive $280
http://www.newegg.com/Product/Product.aspx?Item=N82E16820233061
Connecting ordinary hard drives, on a caching RAID controller, does
have the advantage that some of the information is stored in the
RAID controller cache. But for a database application, the access
pattern is probably quite random, so the cache would be rendered
worthless by such a pattern. So while it might be tempting to
buy a RAID card with a 1GB cache DIMM mounted on it, I'm not at
all certain that is going to help.
So far, you'll notice *CHEAP* was not part of the answer :-)
Motherboards can be endowed with quantities of memory, but I
don't think you'll get to 500GB that way. The largest DIMMs
(probably manufactured just so they could say they did it),
cost as much as a small used car. You wouldn't want to fill
a server motherboard with a memory product like that, as you'd
easily surpass some of my other material cost estimates.
Quickly scrolling through the list on the Tyan site, this is the most
DIMM slots I could find.
Tempest i5400PW (S5397)
http://www.tyan.com/product_board_detail.aspx?pid=560
http://www.tyan.com/support_download_memory.aspx?model=S.S5397
Using (16) 4GB FBDIMMs gets you to a total of 64GB. Still short
by a factor of 8. In round numbers, those FBDIMMs are $500 a piece,
so 16 of them is $8000. If we somehow set up eight of those
boards, and networked them together, that is $64000 worth of
RAM. (Each motherboard would function as a RAM based NAS.)
It's fun to dream though...
I think to be realistic, you should have a very good understanding
of just what the characteristics of the database are. Is there
a way to trace what sectors are accessed during your daily
activities ? If you find the set of sectors visited is not
purely random, then caching might make a big difference to
your performance.
Paul
.
- Follow-Ups:
- Re: expanding ram
- From: ~misfit~
- Re: expanding ram
- References:
- expanding ram
- From: carlbernardi@xxxxxxxxx
- expanding ram
- Prev by Date: Re: expanding ram
- Next by Date: Re: expanding ram
- Previous by thread: Re: expanding ram
- Next by thread: Re: expanding ram
- Index(es):
Relevant Pages
|
