Re: 4-way Opteron vs. Xeon-IBM X3 architecture
- From: George Macdonald <fammacd=!SPAM^nothanks@xxxxxxxxxxxxx>
- Date: Mon, 02 Jan 2006 06:04:56 -0500
On Fri, 30 Dec 2005 20:51:42 +0000 (UTC), David Wang <foo@xxxxxxxxxxx>
wrote:
>George Macdonald <fammacd=!SPAM^nothanks@xxxxxxxxxxxxx> wrote:
>> On Thu, 29 Dec 2005 19:19:38 +0000 (UTC), David Wang <foo@xxxxxxxxxxx>
>> wrote:
>
>> >George Macdonald <fammacd=!SPAM^nothanks@xxxxxxxxxxxxx> wrote:
>> >
>> >> Maybe but IF YOU"D ONLY READ, there's more than THG - this stuff is also
>> >> common knowledge in discussion forums. OTOH Micron/Crucial, among other
>> >> mfrs, for what is now a small premium, is producing PC4200/DDR500 DIMMS for
>> >> the enthusiast market; so the server market is asleep?... err, not
>> >> enthusiastic enough to pursue this?:-) If this DDR500 needs 2.8V... so
>> >> what?... specs can be changed to adapt to evolution of silicon.
>> >
>> >1. I did not see any concrete reference other than to THG. If you have
>> >anything other than citing "common knowledge from the enthusiast market"
>
>> While it may suit your agendum, that is not a quote of what I said.
>
>My "agenda" is only to report facts and data. The facts on the ground
>is that I have not seen any references that has shown that you can
>reliably add more memory to the Opteron system @ 400 Mb/s.
More than what?
When you cobble together a composed misquote, it's distorting my "facts" to
fit something you want to argue with.
>The point here is that the issue concerns both speed AND capacity.
>The references given suggest that higher speeds are possible, but
>none shows that higher speeds are possible with full capacity.
Depends what you mean by "full capacity" - if you mean eight quad-rank
DIMMs then the memory mfrs don't even make PC3200 in that form factor; if
you look at the mbrds available to us, eight slots per CPU is not something
I've seen. OTOH right in this thread is a "reference"
11r8donj9u00g9d@xxxxxxxxxxxxxxxxxx apparently demonstrating PC3200
operation with 72 devices per channel.
>> >then automatically applying it to the server market, please reference
>> >that.
>
>> I did not give any "concrete reference" URL... not even to THG - that was a
>> casual mention of one I remembered among several I've seen and I do *not*
>> keep bookmarks for such things... as common as they are. You need to get
>> out more - the "common knowledge" is in discussion groups all over the
>> place - I visit several Web Forums to gather info and troubleshoot and I've
>> seen such discussion of memory configs in almost all.
>
>As you may suspect, I read plenty about memory systems, and I would
>vigorously challenge your memory in regards to the "common knowledge"
>here. I do not believe it's common knowledge for anyone to have
>demonstrated running 64 DDR(1) SDRAM devices reliabliably @ 400 Mb/s.
>If you can, please do cite a concrete reference URL. What you remember
>to have seen may not actually be what it is.
See above.
>> >2. I don't read THG, so I am not familiar with details of their test.
>> >However, there is a difference between "memory targeted for the enthusasists"
>> >and "memory targeted for servers". For one thing, servers use RDIMMS
>> >while desktop folks use UDIMMS. The difference is that servers tend
>> >to go for capacity, so they'd want to load up on the ranks, and you'll
>> >usually see 16 x4 devices crammed into a single rank, then the DIMM
>> >provides the buffering to electrically isolate the load.
>
>> With the right devices and with registering the server market should be
>> able to do better - that's all I'm saying. There's nothing about an x4
>> device which would prohibit making the higher speed versions in x4 form.
>
>The issue that prohibits making the higher speed version with x4 devices
>is that you have to hang 16 of them (18 with ECC) on the same address
>and command busses per rank. That's a rather heavy electrical load to
>run @ 200 MHz. So, no, you can't just look at "enthusiast memory" built
>with x8 parts and automatically assume that x4 parts will work just the
>same at the same data rate, because you're going to need even faster
>parts to meet the same timing.
The memory mfrs are producing PC3200 2GB RDIMMs with 36 devices.<shrug>
>> > The "enthusiast
>> >memory" have to go for speed, so AFAICT, they don't use 16 x4 devices
>> >in any single rank, and there's no buffering there. So even if THG or
>> >whatever website may have succeeded in running 4 ranks of DDR(1) devices
>> >@ 400 Mb/s, we have to ask the next question, "What is the configuration
>> >of each rank?" If those are not fully loaded 16 device ranks, then the
>> >capacity per channel is still limited to 32 devices @ 400 Mb/s.
>
>> Ah so the servers don't "go for speed"? What I'm saying is that with the
>> devices available now I don't see why they could do better. You know
>> damned well that UDIMMs are 8 devices per rank so why do you have "to ask
>> the next question"?
>
>Because you cited some rather nebulous references in regards to memory
>from the enthusiast market and assumed that it would work in the server
>world. I was simply pointing out that's not going to work here because
>of configuration differences and electrical loading considerations.
AMD has specs. They can be exceeded and have been regularly for Athlon64
unbuffered operation; it does seem that the specs for Opteron registered
could be and in fact are easily exceeded.
>> > Even if you show that it is
>> >capable of running DDR(1) with 64 devices @ 400 Mb/s on Opteron/A64,
>> >the next question to ask is how many simulation and test hours do you
>> >have behind that configuration. The point here is that no one is doing
>> >that sort of study to get DDR(1) to work @ 400 Mb/s in the big servers,
>> >because DDR2 is here, and all of the energy is going into qualifying
>> >those parts and getting the 64 device configurations to run @ 400 Mb/s.
>
>> Ah now we have it: "conservative":-)... that's my complaint.;-)
>
>Which is what is shipping in HP's Opteron server, and guarenteed to
>work by HP. That guarentee provides the effective upper limit to
>the maximum memory capacity of an Opteron server as of today. That
>limit cannot be exceeded or changed arbitrarily. This, I believe was
>the crux of the contentious point. . .
The original claim was that anything "> half-loaded" required a decrease in
clock speed; even AMD specs go above half-loaded.
>> >> >Getting 4 ranks of DDR(1) to work @ 400 Mb/s probably doesn't take
>> >> >rocket science, as I am reasonably certain that I am using it in the
>> >> >Intel based box that I sit in front of right now. The difference is
>> >> >in guarenteeing that it'll work with sufficient margins and reliability
>> >> >that HP or anyone would feel comfortable in putting their name on it...
>> >> >in a server.
>> >
>> >> Let's hope someone else might get more umm, enthusiastic.:-)
>> >
>> >That's the enthusiast market, not the server market. I believe the
>> >topic of discussion is the 4P Opteron box and its memory capacity.
>> >If gaming ethusiasts start shelling out the dough to cram 16 GB of
>> >memory into those boxes, then some vendor will certainly step up, do
>> >the required engineering support and get it to work, but until then,
>> >the Opteron is limited to running DDR(1) with only 32 devices @ 400 Mb/s,
>> >because that's the maximum amount of memory that HP is willing to
>> >stand behind at that datarate.
>
>> No, the topic was *not* limited to 4P Opteron box, not that it's make much
>> difference with Opteron anyway. I'm not sure how you're counting your "32
>> devices" but HP only makes the rules for its boxes... not Opteron in
>> general.
>
>32 devices is not counting ECC. 36 counting ECC.
>
>The limit is 2 ranks of 18 x4 DDR(1) devices running @ 400 Mb/s, and
>that's the same number I am seeing over and over again. Not just HP,
>but Tyan as well. So the limitation isn't just HP Opterons, or even
>Opterons of any brand of servers, but DDR(1) SDRAM memory controllers
>@ 400 Mb/s. The Opteron just happen to have a DDR(1) SDRAM memory
>controller that has to follow the same constraints as everyone else.
>If you claim that the limit can be exceeded, please show me where
>you're getting your impression from, because I'd certainly like to
>see where someone is getting a fully loaded DDR(1) memory system to
>run @ 400 Mb/s. **
Look up the specs again - looks like AMD is saying 3 ranks of 18 x4
devices.
>** By fully loaded, I mean 4 ranks of 16 x4 devices, for a total of
>64 devices (not counting ECC) per channel. With 1 Gbit devices, you'll
>get 8 GB of memory per channel, and 16 GB of memory per Opteron
>processor. In a 4P config, that would push your total memory
>capacity to 64 GB instead of the current limit of 32 GB @ 400 Mb/s.
>
>> >> >HP was being conservative and drops the capacity down to 2 ranks when
>> >> >running DDR(1) @ 400 Mb/s. That shouldn't be a surprise. IMO, seeing
>> >> >DDR(1) @ 400 Mb/s in a server is itself a bit of a surprise,
>> >> >since DDR(1) @ 400 Mb/s is really "overclocked" memory with the 2.6 V
>> >> >spec.
>> >
>> >> DDR(1)400 is no more of an overclock than PC100 was over PC66 and PC133
>> >> after that - it's just progress and enhancement of the processes... happens
>> >> all the time.
>> >
>> >There are sets of baseline spec's for SDRAM, DDR SDRAM, DDR2 SDRAM,
>> >respectively, and DDR(1) 400 is a push product that broke the
>> >baseline spec for DDR SDRAM. PC133 (and even PC166) uses the same
>> >voltage specs as PC100. DDR(1) 400 does not use the same set of
>> >spec's as DDR 266 or DDR 333.
>
>> So a recommended .1V difference for DDR400 makes it overclocked and a
>> different "set of specs"?<boggle> As I've already tried to point out the
>> "baseline spec's" for DDR are >3 years old - if you're going to deny the
>> ability to push as the silicon opportunity presents, I have to ask why? We
>> "allow" that CPUs, GPUs, etc. increase voltage and speed over a silicon
>> design lifecycle... why not memory?
>
>The difference is that CPU's and GPU's are single vendor to single customer
>parts. That is, Intel can change the specs of these devices to whatever
>it wants, whenever it wants, as long as its customers doesn't mind
>following the new spec. Same with AMD, NVIDIA, ATI, etc. DRAM doesn't
>work that way. The dynamics of the commodity market means that the parts
>are supposed to be completely interchangeable. So the "interchangeable"
>aspect of things greatly limits the standards definition process.
Times have changed - we can now tweak the voltages of about every part of
the system from software. We now have gone as far as CPUs with variable
voltage specs. For memory there's up to .5V boost for some mbrds in .05V
increments; saying you shouldn't do that because it violates some old spec
is nuts. Would I personally run server memory at 3V - no! Would I go up
to 2.75V?... if it gave me a worthwhile performance advantage, probably.
>For example, Samung can probably crank out much faster DDR parts because
>it has excellent process tech, but some of the less-well funded fabs
>can barely meet the spec, and they would be hard pressed to meet these
>push spec parts, so they would be resistant to changes in the JEDEC
>standards definition.
>
>The limitation of the JEDEC standard means that the faster guys can't
>really run ahead of the slow guys, although they're finding some ways
>around that with the push spec parts designed for the "enthusiast
>market". So, no, you can't just take advantage of opportunities
>made available with faster process technology to make your own faster
>DRAM parts. You have to wait until sufficient number of DRAM manufacturers
>can agree with you on the new addendum to the spec, and a suffient
>number of design houses (Intel, IBM, Sun, AMD, etc etc) agree to the
>same set of proposed addendum to the spec, before the standard can be
>created, and you can sell you part as (JEDEC) DDRx xyz MHz, and customers
>can be reasonably certain that your DDRx xyz MHz parts can operate
>interchangeably with parts from Infineon, Samsung, Micron, Elpida,
>etc.
Every DIMM mfr is selling unbuffered boutique parts rated at PC4000/DDR500
(2.8V seems the norm) - where they get the chips from matters not...
whether it violates some enshrined, 3-year-old JEDEC document is of no
importance to the people selling or buying.
>> >> > The 400 Mb/s bin was supposed to have been reserved for DDR2,
>> >> >but market demand from the desktop segment pushed 400 Mb/s down into
>> >> >DDR(1) territory and made it happen. So no other server that I know of
>> >> >use DDR(1) @ 400 Mb/s, and they're all waiting for DDR2 to run 4 ranks
>> >> >@ 400 Mb/s.
>> >
>> >> Not expecting some evolution and progress in functional specs is a bit
>> >> myopic IMO. No need to slavishly follow specs which are >3 years old.
>> >
>> >If you're selling servers and your reputation depends on the reliability
>> >of said servers, you too would gladly follow specs regardless of their
>> >age. If you're pushing out a product that has better spec than others
>> >in the market, that's great, but I'd want to know how many test hours
>> >you had behind it. I wouldn't want to hear that you just came up with
>> >the product with Micron's help last week.
>
>> Oh you mean like FBDIMMs... with AMBs which "will brown a burger better
>> than a George Foreman Grill can" [quote from this NG]?:-)
>
>FBD's have been in developement for more than 2 years, and they're still in
>development/testing/tweaking.
Yeah and the point is that Intel dumped a buncha $$ into Micron to get it
kick-started... thus the hint of irony in your remark about Micron.;-)
>They'll enable servers with incredible amount of memory, and the power
>headache that comes with it. The AMB is just part of the problem. With
>16 device per FBD, you can get the ratio of AMB device power to DRAM
>device power down to 15~20%.
But the AMB power usage must depend on its position in the channel... so
are they going to be able to be able to lose the heatpipes on them?:-)
Surely the Primary is always going to get hammered.
--
Rgds, George Macdonald
.
- Follow-Ups:
- Re: 4-way Opteron vs. Xeon-IBM X3 architecture
- From: David Wang
- Re: 4-way Opteron vs. Xeon-IBM X3 architecture
- Prev by Date: Re: Intel Inside no more
- Next by Date: Re: Intel Inside no more
- Previous by thread: Re: 4-way Opteron vs. Xeon-IBM X3 architecture
- Next by thread: Re: 4-way Opteron vs. Xeon-IBM X3 architecture
- Index(es):
Relevant Pages
|
