I'm think this one is probably to neutrino. Currently I'm totally sure that game working better on GPU with 2GB (or 2.5GB) of GDDR5 than on GPU with 1GB of GDDR5, but there is no understanding for larger VRAM. Question is will be there some advantage for PA if GPU will have enormous amount of VRAM like 6GB GDDR5? Can engine use large amount of VRAM efficiently (e.g store more detail textures in VRAM with less loading on zoom) or it's won't give any advantages? Or those amount of VRAM just aren't useful for current gen of GPUs (i'm doesn't know much about hardware itself, so explain please)? I'm just thinking about my future GPU upgrade and also I'm just want to know if PA's virtual texturing tech will scale depends on available VRAM.
Actually i have the same concern : Today PCGaming magazines state that if you do not run a higher than 1920x1200 resolution, then you don't need more than 2gigs of VRAM. However, the most powerfull graphic cards (in terms of processing power) are also the ones having big amount of memory (3gigs or more). I don't know which parameter is the most important for PA : Processing power or Memory ? It of course may make sense to consider that the more powerfull gpu is, the more memory you need. However, having allways read that 2gigs of memory is useless for resolutions lower or equal to 1920x1200, I feel quite a bit puzzled. Hopefull a smart Uber guy will provide clarification
Forget about todays PCGaming magazines, they make generic assumptions based on the inner workings oft last generation games. With megatextures and alike, this is no longer true. With past engines, you would require high resolution textures, but they were still tiled so the memory usage was rather static and limited, at some point all textures would have fit the memory and you would no longer gain any advantage from more memory. With mega textures, textures are no longer tiled and can be several gigabyte in size (when rendering the whole texture in full resolution at the same time). Although they can be streamed dynamically, so the system can still make up for a lacking memory size up to a certain point, a lot better than with the old system where textures would have to be reloaded as whole leading to bad performance. Memory management works completely different from the days of the old because now only the lowest resolution thumbnail is actually hold persistent in memory and higher resolution textures are regenerated when needed. In the classic approach, you would instead always hold the highest resolution texture in memory and generate all mipmap levels at the same time (whenever the texture had to be reloaded). PA can use any VRAM you have to offer to store/cache the texture of the planets surface as it is one giant image and if the VRAM is to small, parts of the image will get evicted to soon. It is safe to assume, that PA would be capable of using up to 2-4GB of VRAM on large planets with ease (per planet that is, you will already notice some texture streaming at 1GB). Although you will possibly not gain any visible advantage from more than 3-4GB VRAM since regenerating the mega texture isn't that expensive and you can make up for a "small" VRAM with a fast CPU and a fast interlink between CPU and GPU.
But it does work automatically. I'm not sure if that is what Uber does, but I believe you can just allocate new memory (to be used in the megatexture) until the VRAM is full (you get out of memory errors), and once it is full use LRU based eviction inside the reserved memory. You could also use one of the platform specific APIs to get the VRAM size instead of risking OOM errors. The shader informs the engine whenever a part of the texture was required in a resolution which is not in memory yet and will use the available low resolution version for now. With a recent OpenGl extensions you no longer need to implement the memory management on your own, but PA makes no use of this extension yet.
Thanks for very detailed answer. I'm not 3D programmer and don't really know all those details. Yes, but it's might be not designed for enormous amounts of VRAM. E.g it's can continue to stream textures even if there possibility to store all of them in VRAM. Am I wrong? As long as I understand texture streaming managed by engine itself and not by any OpenGL extension. Isn't it?
It is handled partly in the engine and partly in the shader itself. It could be done using a new OpenGL extension, but that isn't used in PA. Eviction will only kick in when the reserved memory is full. So if the VRAM is big enough, no eviction will occur and streaming will only be used when the texture is required for the very first time. There is no preloading though, that means streaming will always be visible shortly after startup.
You mean GL_AMD_sparse_texture, right? Thanks for this info, but anyway I'll be happy if somebody of devs confirm it just to be sure PA won't have any specifics. It's also interesting if there possible some bottlenecks in hardware itself. E.g for example there is HD 7970 GPU which have 6GB VRAM, but is this chip can handle it? I'm asking it because previously (many years ago) on market was GPUs which can't really handle lot of VRAM properly and only have it for marketing purposes.
We don't do texture streaming in the "traditional" use of virtual texturing. Games like id's Rage or Splash Damage's Brink popularized the idea of streaming vast gigabytes of pre-cooked texture data from your computer's harddrive. This meant you can have a level with 10s of gigs of textures be streamed in while keeping fixed (but trivially configurable) VRAM usage. We generate the virtual textures on the fly on the video card, we're not streaming from the disk at all. When you move the camera we re-render the terrain in to the virtual texture at the resolution needed. An advantage of this is the game doesn't need to generate and store the hundreds of gigs of data several large planets could require for the full resolution textures across the surface, and we only have to pay the "cost" of rendering the surface when we need a new resolution. It also means we can support much higher res textures when at the closest ranges. Rage and Brink both get pretty blurry when you get close to a wall because it would have taken hundreds of gigs of storage space (and several days of computation time on their end). The Virtual Texture setting in the options almost directly correlate to VRAM usage. There are some other factors like the screen resolution, UI textures, and unit textures, but right now low sets the virtual texture to 1024 x 1024, medium is 2048 x 2048, high is 4096 x 4096. Those are actually fairly low res compared to Rage, which defaults to 4096x4096 on consoles and PC and can be overridden to 16384 x 16384 on the PC. For our game we currently use 4 32bit virtual textures. On the OpenGL extensions, AMD has an extension to support virtual textures, referred to as Partially Resident Textures, that may be a good performance improvement for us to use, but is only supported on the latest Radeon cards and requires OpenGL 4.2. NVidia's Bindless Textures and are similar in that it allows large amounts of texture data to be handled, which is part of what virtual textures do, but is narrowly focused on textures being streamed from a disk. It is also only available for the latest NVidia cards (higher end 600 series and 700 series). To answer the specific question of if more memory will help, yes, but probably not to the 6gigs range, at least not yet. Before optimizations a single planet used multiple gigs of data just for the mesh data, not the textures. Running multiple views with high quality texturing in all views will also require more memory, which could be considered an "advantage". Exactly how much memory depends on the size and number of planets and what we do with ongoing optimizations. We may be able to have cards with as little as 512 megs of VRAM look nearly as good as a card with 8 gigs. Either way CPU & GPU perf is going to be more important that VRAM over 1 gig.
Thank you for such informative answer, it's this is big piece of information about PA engine and it's really interesting. I'm finally ready to ask few small questions about that: Is generating textures on the fly is reason why on zoom "glUseProgram" goes up to 200-500 while normally it's 10x lower? Can you expose some control over this process into UI or just with some debug flags? Probably I'm wrong, but texturing process seems to be main reason of lag on zoom, personally I'm prefer to see few glitches with playable FPS, than get lags on zoom. Is there some kind of cache? If no, why? Textures size for all planet is too large to store any "cache"?
pinbender would be better at answering these question, but hopefully I won't be too inaccurate. 1. Yes. There's a lot of work that goes into generating the textures. There's likely still optimization to be had there, and there's more experimentation to be done with load balancing. It took id several years to get it right for Rage. 2. I expect we'll expose more options at some point. There's a lot of variables beyond just the texture resolution that goes in to it, and I think most of them are controlled by hard coded numbers right now. 3. There's no cache for the textures. Part of the reason is the amount of storage space required, especially since it would basically be worthless data after any game is done. The other part is just rendering the surface again should end up being faster than reading it from cache on any kind of hard drive, even an SSD.
Thanks for answers! I don't mean HDD, SSD or even RAM cache, but some kind of VRAM cache (I'm have no idea how it's works on GPU, sorry if this wrong question). Is result of texture generation stay in VRAM (obviously if there enough free memory) after I'm changed zoom level or it's created each time from scratch? Will VRAM size affect this process? I'm toyed with planet generator a lot and I'm feel it's working this way, but want to ask anyway.
Also I want to clarify that I'm asking all this because I want to generate extremely large planets and solar systems and play with generation process lot. I'm understand that you'll optimize normal game for non high-end GPUs, but this question isn't about playing actual game and more about finding best hardware for PA. I just want do some crazy things like this: https://forums.uberent.com/threads/some-crazy-fun-without-gpu.51186/ And I'm want to buy GPU which will be better suitable for such activity. I'm already find nice 64GB RAM pack.
Sorry, was busy with some back-end stuff, and Ben didn't tell me about this thread for some reason. TL;DR: We are using more video memory for larger planets, but less than you might think. We have a virtual texturing cache that keeps the results of compositing the terrain, which is what gets turned into what you actually see every frame. How much of that cache gets used for a given planet is dependent upon screen resolution, rather than texture resolution or planet size. That allows us to get full texture resolution for every pixel, even when you're zoomed all the way in. (I have some interesting ideas about other things we can use that cache for, but it will probably have to wait a while. People are really busy around here. I might be about 3 levels deep on "things that have been interrupted to work on something more important", and rendering features surprisingly aren't the first or last thing on that list. In fact, we recently hired another guy who's a total rendering ninja to help out with rendering tasks, and he already got interrupted to work on profiling tools, and just recently got back to doing rendering stuff.) Anyway. We need an extra texture per planet for referencing into that cache, and the size of that texture is a function of surface area. That means a big planet needs a big page mapping texture. It's only a single channel, however, and it maps 4096:1 to virtual texels, so it's quite a bit smaller than the 3-channel full resolution virtual texture that would take up 768GiB if it were ever actually rendered out. We do cap it at 4kx4k, but that's a 64mb texture even without mips. If you start adding more big planets into the system, it can turn into quite a lot of data. Additionally, our vertex density is also a function of surface area, so a bigger planet will also mean more of that. It's not as much data, however, so it's more like a 10% additional cost than a 100% additional cost. Newer cards have some tricks we might be able to use that will allow discarding large page map mips when they are not in use. We have a lot of other features queued up for work before that, though. (The sad reality is that the cards that support those features are also the ones that need it the least, since they're more likely to have extra memory.) At some point I would like to experiment with re-creating the textures dynamically to emulate the feature when it's not available, but that's also quite a bit down the ol' to-do list. At the end of the day, having more memory on your video card isn't going to slow us down. We're working really hard to make sure the game runs well on a GPU with a reasonable amount of memory. If you have a bigger, beefier GPU, however, there are extra bells and whistles you may be able to turn on later when we have a little more wiggle room to actually implement those bells and whistles. In the mean time, turn everything up to "Uber", and we'll take up whatever extra memory we can for now.
Also, as a follow-up apology, I just double-checked our virtual texture format, and one of the channels is currently 16-bit. So the full, maximum virtual texture size is "only" 640GiB. Terribly sorry about the confusion.
Fantastic reply, thanks for taking the time to post this. Always good to read technical stuff, at least the stuff that I can follow
