AMD Rome Second Generation EPYC Review: 2x 64-core Benchmarked
by Johan De Gelas on August 7, 2019 7:00 PM ESTPCIe 4.0
As the first commerical x86 server CPU supporting PCIe 4.0, the I/O capabilities of second generation EPYC servers are top of the class. One PCIe 4.0 x16 offers up to 32 GB/s in both direction, so each socket offers up to 256 GB/s in both directions, for a full 128 PCIe 4.0 lanes per CPU.
Each CPU has 8 x16 PCIe 4.0 links available which can be split up among up to 8 devices per PCIe root, as shown above. There is also full PCIe peer-to-peer support both within a single socket and across sockets.
With the previous generation, in order to enable a dual socket configuration, 64 PCIe lanes from each CPU were used to link them together. For EPYC, AMD still allows for 64 PCIe lanes to be used, but these are PCIe 4.0 lanes now. There is also another feature that AMD has here - socket-to-socket IF link bandwidth management - which allows OEM partners to design dual-socket systems with less socket-to-socket bandwidth and more PCIe lanes if needed.
We also learned that there are in fact 129 PCIe 4.0 lanes on each CPU. On each CPU there is one extra PCIe lane for the BMC (the chip that controls the server). Considering we are living in the age of AI acceleration, the EPYC 7002 servers will be great as hosts for quite a few GPUs or TPUs. Density has never looked so fun.
Facebook
Twitter
RSS
180 Comments
View All Comments
close - Thursday, August 8, 2019 - link
VMware licenses per socket. I'm not sure what kind of niche market one would have to be in (maybe HPC on Windows with the HPC Pack?) to run Win server bare metal on this thing. So I'm pretty sure the average cores/VM for Windows servers is relatively low and no reason for concern.schujj07 - Thursday, August 8, 2019 - link
@deltaFx2 Most people purchase more cores than they currently need so that they can grow. In the long run it is cheaper to purchase a higher SKU right now than purchase a second host a year down the road.@close There are companies that are Windows only so they would install Hyper-V onto this host to use as their hypervisor. However, even under VMware if you want to license Windows as a VM you have to pay the per-core licensing for every CPU core on each VM. I looked into getting volume licensing for Server 2016 for the company I work for we have 2 hosts with dual 24 core Epyc 7401's and we would need to get 16 dual core license packs for each instance of Server 2016. It ended up that we couldn't afford to get Sever 2016 because it would have cost us $5k per instance of Server 2016.
DigitalFreak - Thursday, August 8, 2019 - link
@schujj07 Just buy a Windows Server Datacenter license for each host and you don't have to worry about licensing each VM.schujj07 - Thursday, August 8, 2019 - link
AFAIK it doesn't work that way when you are running VMware. With VMware you will still have to license each one.wolrah - Thursday, August 8, 2019 - link
@schujj07 nope. Windows Server licensing is the same no matter which hypervisor you're using. Datacenter licenses allow unlimited VMs on any licensed host.diehardmacfan - Thursday, August 8, 2019 - link
This is correct. You do need to buy the licenses to match the core count of the hypervisor, however.Dug - Friday, August 9, 2019 - link
You still have to pay for cores on datacenter. Each datacenter license covers 2 cores with a minimum purchase of 8. So over 8 cores and you are buying more licenses. 64 cores is about $25kMDD1963 - Friday, August 9, 2019 - link
Windows license (Standard or Datacenter) covers 2 *sockets* for, a total of 16 cores....; if you have more than 2 sockets, you need more licenses...; if you have 2 sockets, filled with 8 core CPUs, you are good with one standard license... If you have 20 total cores, you need a standard license, and a pair of '2 core' add ons... If you have 32 cores, you need 2 full standard licenses....MDD1963 - Friday, August 9, 2019 - link
Datacenter is still licensed for 16 cores, with little 2 pack increments available, or, in the case of a 64 core CPU, effectively 4 Datacenter licenses would be required...($6k per 16 cores, or, roughly $24k)deltaFx2 - Friday, August 9, 2019 - link
@schujj07: Of course I get that. The OP @Pancakes implied that Rome was going to hurt the wallets of buyers using windows server. The implication being this would not happen if they bought Intel. I was questioning those assumptions. How can Rome cost more money for windows licenses unless rome needs more cores to get the same job done or enterprises overprovision Rome (in terms of total cores) vs. Intel. That would make sense if the per-thread performance is worse but it's not.