A Messy Transition (Part 2): Windows XP, Vista, and the 2GB Barrier
by Ryan Smith on July 19, 2007 12:00 PM EST- Posted in
- Software
Final Thoughts
Among all of our tests, there are three identifiable patterns, all relating to some difference between Vista and XP. Those are:
- Vista is using more address space than XP in all situations
- The amount of address space used with Vista seems to be related to the amount of video memory on our video card
- XP on the other hand does not fluctuate at all, the address space usage is the same no matter what card we use.
We suspect that the issue is related to some combination of how Vista handles video cards, and Vista being more resource-hungry as a side effect of it being a heavier operating system in general. With the latter generally being unavoidable and unchangeable, we're left with trying to figure out what about the former is having the greatest impact.
One of the most notable features introduced in the Windows Display Driver Model(WDDM) for Vista was virtualized memory, which allows for Vista to use the normal computer memory hierarchy as part of a memory hierarchy for video memory. The reason we mention this is that tucked away in one of Microsoft's documents on WDDM is a note with the following: "Memory allocations are limited to the application address space primarily for security reasons. WDDM provides increased security by isolating applications and their resources from each other." Our suspicion is that this feature is causing video memory to eat up more of the application's address space than under XP, which would explain why the extra address space is being used, why it's variable with video memory, and why XP doesn't show something similar.
Unfortunately we have been unable to find any confirmation that this is the case. When presented with the some of our numbers, only NVIDIA would comment, cryptically saying that "Microsoft is aware of the issue and they are working on it. More details will be provided as they become available." Whether this means that the problem is what we're guessing or something entirely different is not something we can get an answer at this point.
As far as Windows Vista is concerned, all of this weighs heavily on a fledgling operating system that many people are waiting to get its first service pack before trying. While the performance problems we saw earlier this year with Vista have largely cleared up, now it seems there's a definite issue with address space usage in conjunction with applications that make heavy use of a video card (e.g. games). In turn, it's hard to understate the situation, since hitting the 2GB barrier typically results in crashing.
The situation isn't entirely unmanageable, but the number of questions left is uncomfortably larger than the number of answers provided so far. The problem can be solved by users in several ways we've outlined in part 1 of this series, but none of these solutions are truly satisfactory. The real solution is going to have to come from Microsoft, who thankfully is aware of the problem and we presume we'll be hearing about a solution from them sooner than later. In the mean time however serious gamers will want to take a critical look at Vista if they are currently using it or are thinking about switching to it. This isn't a severe problem with so few serious gamers currently on Vista, but it may be best to hold off on switching to Vista until Microsoft has the issue sorted out.
On a lighter note, not everything we've seen today is bad news. The address space usage on XP is significantly lighter than Vista, making the situation far less dire. Instead of having to resort to any of the solutions we outlined in part 1, the majority of XP users should be able to get away with not needing to take any action at all, at least for the next few months. The 2GB barrier still exists and still will be a problem, but ideally by the time the situation is as bad on XP as it is on Vista today, we'll have a solution to this address space issue from Microsoft, along with a clearer path on the migration to 64bit.
And getting back to Supreme Commander for a moment, our problems have not fallen on deaf ears over at developer Gas Powered Games. They let us know that the next version of Supreme Commander, Forged Alliance, will feature several engine improvements related to the issues we've covered. They are switching memory allocators to an in-house solution which is lighter and uses less memory, and are also attempting to put in a notification system for alerting users if the game has run out of address space to work with. Finally the game will be shipping as large address aware and will be tested as such. Hopefully we'll see other developers taking similar steps over the next year.
We'll have more on address space usage in Windows Vista as we find out more.
Facebook
Twitter
RSS
57 Comments
View All Comments
totallycool - Thursday, July 19, 2007 - link
The reason could also be the way memory is allocated in vista as opposed to xp. If i am correct, then with superfetch it really tries to bring as much of the program in as possible, and this could relate to vista allocating more user space to program as opposed to xp. Simply put xp could be more conservative in allocating memory then vista is.The best way to test it out would be to check other programs memory usage, like maybe Word2007, photoshop etc on the two operating systems.
Just my 2 cents
The other solution is to enable the A20 line already ;)) <- Joke
Ryan Smith - Thursday, July 19, 2007 - link
It's not SuperFetch. Unfortunately I can't find the MS article at this specific time, but there's one I read specifically talking about SF having its own allocations.BUL - Thursday, July 19, 2007 - link
What registry setting is the equivalent of "DOS=HIGH"? And wouldn't it be the A36 line? (I'm just messing with you...)strafejumper - Thursday, July 19, 2007 - link
i've noticed on a lot of game boxes that the minimum system requirements are different for vista and xpusually its something like:
1gb ram (2gb if using vista)
as a gamer this kind of thing put me off and i stuck with xp
leexgx - Thursday, July 19, 2007 - link
as i have got vista 64 my self and XP 32 dual boot on 2x2 80gb hdds RAID (2 for xp 2 for vista) i find games are basicly Unplayable in some cases on vista with 2gb of ram due to the game stuttering
i got 4gb of ram in it now (xp picks up 3.2gb but that hardy matters as i never see ram use pass 1.8gb)games run mostly fine DX10 games are flaky
Nvidia could fix this bug maybe setting there drivers to Not used shared ram as from the way it looks its the New way vista handles shared ram now on the 7800 256 it would waste an extra 256mb in VM but the 8800gtx has 756mb so it uses up 500mb more VM then the 7800GTX
ChristopherO - Thursday, July 19, 2007 - link
Am I the only one who thinks Microsoft should release a Knowledge Base article on this issue? (Ryan, have you been in contact with them?) At least the creation of said article would be an "official" source to look for, acknowledgement, updates, or a hotfix. Until they create a KB article and admit to a fix, it is pure speculation that MS's opinion isn't, "Game developers need to learn to program correctly, not our problem."Speaking of which, they had better get this done ASAP. I see two huge problems looming in their future. 1.) Back to School. 2.) Christmas. First, the college kids getting computers are going to be awfully mad if their games don't work. I'm sure all the developers and MS will get blamed equally for this. Same thing goes for Christmas. People are going to want to test new machines with the latest and greatest games/apps.
As for me, I have Vista 64, 4GB memory, with a 512MB ATI X1950. I had to edit the headers on FSX. Also C&C3. C&C 3 is such an allocation hog that it will run out of 3GB of address space. It will always do this in about 15-25 minutes on large maps, and 30-45 on smaller ones. Sure I can turn down the detail, but why the heck would I do that when I can run at "ultra-high" quality settings and get nothing less than 30fps? Fortunately I can tell when C&C 3 is about to crash (the icons go "pink"), which gives me time to save, quit, re-load and keep playing. Unfortunately I can't play online due to this problem.
What confuses me is that memory allocation is not a difficult problem... I don't know if the issue is becoming more prominent because game developers are really barely "adequate", or if they are off-shoring the labor to poorly trained (i.e. cheap) coding teams. I have no idea what a game developer earns, so I really couldn't make an educated guess as to where the problem is being introduced (other than the fact that Windows Vista isn't helping, and XP is really papering-over bad software architecture).
JCheng - Thursday, July 19, 2007 - link
Umm... maybe so the game stops crashing? You'd rather "save, quit, re-load and keep playing" and stop playing online, rather than turn down your settings??
ChristopherO - Thursday, July 19, 2007 - link
Yes, because otherwise I wasted a large amount of cash buying the fastest video card I could get my hands on. If I realized Vista was going to be problematic I should have bought a $60 card since I can apparently only make use of $60 quality.
elpresidente2075 - Sunday, July 22, 2007 - link
Perhaps you should have done a little more research before basing a (presumably) $1000+ system on a young and mostly untested OS that is released from a company that is known for releasing its final betas under the guise of a "final product" or "golden master" on release date.Don't worry, Microsoft will have the bugs ironed out in a year... or two... or three...
They're gonna release a service pack in November! Maybe that'll make Vista not suck so much memory! *crosses fingers*
nullpointerus - Thursday, July 19, 2007 - link
Typically, system developers create elegant, well-designed API's with flexible error handling mechanisms, which application developers then completely ignore until their applications begin doing wierd things. This failure to use API's as intended was the reason why XP and Vista's 32-bit PAE kernels were crippled: driver developers could NOT be trusted to use the system API's as published at least a decade ago, so the whole MMIO remapping thing could not be enabled, which meant that PAE kernels had the same < 4 GB physical memory limitations as non-PAE kernels.