Page 1 of 2

Improving data load times - how?

Posted: Tue Jan 31, 2006 20:45
by Enjay
I have noticed with GZdoom (and Zdoom before it) that there can be a momentary stutter when loading new data (at least that's what I assume is going on). This seems is more apparent if the thing you are seeing/hearing requires a lot of data to be loaded. For example, I've been playing around with an object that uses quite a number of fairly large sprites very quickly - they are a flame effect and a smoke effect (Solarsnowfall, you know the ones I'm talking about ;) ). When you see one of these for the first time in a level (especially when I'm using the flame and smoke together), there's a momentary slowdown, then the game gets back to full speed and only really slows down again if I throw loads of these things into the map - but that's obviously for a different reason.

Most of the time, it's barely noticeable (but something I'd like to improve on if possible) however, yesterday (for my sins) I downloaded the Risen3D version of the hi-res texture textures and tried them with GZDoom. Once they were up and running, tings were fine - but if I moved into a new area where a lot of previously unseen textures suddenly came into view (opening a door is a perfect way to make this happen) there was a very noticeable performance hit as the data was loaded. I reckon once, opening the door nearest to the player on the right in E1M2 the hiccup was the bast part of 1/2 a second. Not a life shattering delay, but one that could certainly interfere with gameplay, especially when similar delays were happening quite a lot.

I also get the feeling that it's more common when the mod being played has a lot of data in it, even if the thing being encountered isn't that big. Perhaps search time in big WAD files or something is an issue?

I'm sometimes aware of it in regular Doom/2 too. However, then the data being loaded is so small that I think I'm really only noticing it because I'm aware of, and sensitive to the issue.

So, are there any suggestions for a way for me to improve on this?

Posted: Tue Jan 31, 2006 20:53
by Deathlike2
This is what exactly Precaching GL Textures was meant to fix. You'll suffer slightly longer map load time, but you won't have delays like you are referring to.

Posted: Tue Jan 31, 2006 21:06
by Enjay
OK, thanks. That certainly fixed the problem with the hi-res textures. The multi-sprited smoke and flames still has a noticable hit when you see one first.

Posted: Tue Jan 31, 2006 22:14
by Graf Zahl
The engine can't precache what it doesn't know it has to use. Some things are simply unavoidable. The biggest hit doesn't come from creating the textures anyway. It's loading the data from the HD that creates the biggest slowdown. If you know that certain data is used at the same time it might help to group it together in the WAD/ZIP.

Posted: Tue Jan 31, 2006 22:38
by Enjay
Graf Zahl wrote:The engine can't precache what it doesn't know it has to use. Some things are simply unavoidable.
I figured that was probably going to be the ultimate difficulty.
If you know that certain data is used at the same time it might help to group it together in the WAD/ZIP.
I'm usually pretty ordered about how I put stuff into my WADs, so I already have things like that arranged pretty optimally. I suppose I'll have to look at seeing what I can minimise and then just live with the occasional delay. They're not massively problematic, I just wanted to try and improve on things.

I assume it's just not practical to load the entire WAD into memory or something.

Heh, I remember "back in the day" one of the tricks those of us who were lucky enough to have loads of RAM could do was set up a RAM disk, copy Doom.wad to it and then run it from there. Happy days. :)

Posted: Tue Jan 31, 2006 23:09
by Graf Zahl
Enjay wrote: I assume it's just not practical to load the entire WAD into memory or something.

I have to do that when you put a WAD into a ZIP and load that. You can try and see whether it helps.

Posted: Wed Feb 01, 2006 9:41
by Enjay
I'm having difficulty working out if that's having much impact or not. I think maybe it is. For the fire items, a slight reduction in sprite size combined with a tailoring of their scale factor to suit made a bigger difference though (not too surprisingly). They were scaled down quite a bit anyway, so this hasn't had a huge visual impact at all. Thanks for the help.

Oh, to get further clarification: precaching of GL textures, what does that mean? Does it mean all textures are precached or is it just special ones - such as hi-res ones. Judging by my perception of a slight increase in load times for all levels, I'm guessing it's all of them not just special ones. Also, does it take account of things like animations and switches or is it just the textures being used when the level starts?

Posted: Wed Feb 01, 2006 11:12
by Graf Zahl
Precaching means that every texture the level uses and every sprite frame of any actor spawned as load time is created when the level loads. Animations and switches are not being checked. Creating single textures on occasion is not that critical. That happens anyway when you are using warped textures. It only becomes a problem if a larger amount of textures has to be created in a short amount of time. Just type IDBEHOLDV when hires textures are on and you see what I mean.

Posted: Wed Feb 01, 2006 21:30
by Alter
My map09 loads about 10 seconds due to

Map09 details
4206 verticles!!!!!!!!!!!!
5663 linedefs!!!!!!!!!!!!!
10092 sidedefs!!!!!!!!!!!!!
1486 sectors!!!!!!!!!!!
872 things!!!!!!!!!!!!!!!!

look in the projects section for the demo of my megawad which includes this serious map!

Posted: Wed Feb 01, 2006 21:45
by Phobus
Errm, I've made a map with the following statistics that takes less time than that to load up:

20963 vertices
24607 lindefes
41399 sidedefs
2580 sectors
2958 things

And get this! It's currently unbalanced and needs work to sort it out, both gmaeplaywise and architecture.

Besides, many other people have made maps bigger than mine (and well) on their own.

Posted: Wed Feb 01, 2006 21:51
by Alter
map09 was my first big detailed map so check it out

Posted: Wed Feb 01, 2006 22:43
by wildweasel
alterworldruler: I don't know about you, but I think it's in your best interests not to be advertising your project in this thread.

Posted: Wed Feb 01, 2006 22:48
by TheDarkArchon
alterworldruler wrote:My map09 loads about 10 seconds due to

Map09 details
4206 verticles!!!!!!!!!!!!
5663 linedefs!!!!!!!!!!!!!
10092 sidedefs!!!!!!!!!!!!!
1486 sectors!!!!!!!!!!!
872 things!!!!!!!!!!!!!!!!

look in the projects section for the demo of my megawad which includes this serious map!
I've picked one paticular map in my GZDoom folder and:

25528 vertices
34171 linedefs
59989 sidedefs
8022 sectors
2196 things.

Posted: Wed Feb 01, 2006 23:21
by Graf Zahl
Sounds like a not quite up-to-date version of KDIZD MAP03... ;)

BTW, the current version is:

2193 things
25580 vertices
8093 sectors
34299 linedefs
60229 sidedefs


And MAP04 is even larger.

Posted: Thu Feb 02, 2006 3:59
by Tormentor667
Graf Zahl wrote:2193 things
25580 vertices
8093 sectors
34299 linedefs
60229 sidedefs
"The Tower of Suffering" right now is about 35% done and has the following stats:
15583 Vertices
18278 Linedefs
33471 Sidedefs
2725 Sectors
739 Things
So the estimated size could be even larger than MAP03 and MAP04 of KDiZD, maybe I can break the record :)