Image Resolution Degradation With WorldBuilder's 2D Maps Recombining Feature

:scratchhe

The map art for the first floor of the inn has never been altered. There are override files in place in the mod for that map, but for a different reason than to change the art. There is one .jpeg chunk (00230020 to be specific) that is problematic. It is strangely larger in size than the others, although like the others, it is a 96 dpi image. If you recombine that map in WorldBuilder and then split it again however, WorldBuilder turns that chunk it into a 72 dpi image, for reasons I don't know. This all seems to be handled well internally with the vanilla image, but ironically, not with a WorldBuilder processed image, like yours.

What I did for the mod was to save a version of 00230020 with the same exact specs, resolution, and relative data size of its neighbors, and then simply included all .jpeg chunks in that map in the mod for a total override. The reason for this is that I had fears that the game might not have been handling it well internally and possibly causing some of the issues we see with that map - long load times and maybe even the onset of exterior Hommlet jerkstop. I don't believe that's proven to be the case, but it's still probably best to have consistent .jpeg map chunks running. More to the point though, all of this results in no effective visual change for the user. None of the content of the image for that map was changed whatsoever, as you can see in the images below.

Vanilla



Co8



However, we do see the problem I described above arrive in a recombined version of your .jpeg chunks.

Zoltec



And here is a highlighted version to show the specific chunk with issues. As you can see, it is too large to fit in its allotted square.



The game seems to resize it correctly in-game, but the condition is doubtfully ideal. The Co8 version of the file is 14 kb in size, 96 dpi. Your version is 85 kb in size, 72 dpi.

Finally, here's a composite to make comparison easier.



The bottom line is that there is no need to either delete or replace the art for this map from the mod, and it might even be best if you did neither. Doing so will provide no visual change or 'restoration' whatsoever.

All that said, I'm interested in your transparency fix for the black and white mystic garb. If it tests okay, we may use that for the mod if you don't mind.

 

Re: Aziraxis Modification Pack

The .jpeg chunks for the inn map in vanilla and Co8 are functionally identical. 00230021, for example, is a 256 x 256, 96 dpi, 8,931 byte image in Co8, and a 256 x 256, 96 dpi, 8,936 byte image in vanilla. See 100% side-by-side comparison below.



I don't believe that WB even does any real image processing. Its main function in the 2D Maps app is to simply split your existing map, along with its existing resolution and various other characteristics, into 256 x 256 sections, or to recombine existing 256 x 256 sections of maps into a large unified image for editing by the user in Photoshop or whatever. There is no resizing going on or any other editing to lighting, colors, etc. (the issues with the trouble chunk notwithstanding).

Can you post some screens to show what you're talking about? I assume that on your 1024 x 768 display, the same side-by-side exercise as I showed above should reveal noticeable differences between the images. At any rate, if this were really an issue we could simply copy the .jpegs from vanilla (minus the goofy one) directly into the Co8 mod.

 

Re: Aziraxis Modification Pack

In where? I'm pulling 00230021 straight from the dats. It's 8,936 bytes.

No, it's not. No visual difference that I can see, no data size difference, no indication from Agetian that he ever employed anything but the most basic functions for splitting and recombining.

No offense Z but I guess I need some of what you're smoking. Looking at those images reasonably closely (e.g. not with a magnifying glass but with a studied eye), I can see no effective difference. I looked at the .tgas as well, still not seeing it. Can anybody else see what he's talking about?

I think maybe you should try posting a screen capture to try to demonstrate what's happening in-game. Also, what other maps do you see this problem in?

[edit]

Okay, after scrutinizing the images very closely at 200% magnification in Photoshop, I do see a difference. The thing is, my WB and dat versions are identical, so it's still not WB. Where are you getting your vanilla chunks from, and what utility did you use to extract the dats?

 

Re: Aziraxis Modification Pack

Okay, I used Zane's. I wonder if that might be the real culprit, because ...

... that's exactly what I don't see. For me, the vanilla chunk from the dat is identical in size and image quality to the same vanilla chunk recombined and split by WB.

Okay, but a little less with the lecturing please. Believe it or not, I'm trying to make your modding better by holding you to a higher standard.

I will have to take a broad look at all our modded maps, because if image quality is suffering throughout, that will have to be dealt with. Unfortunately, in the case of all our custom maps, there is no other way to make a map besides splitting all those chunks by hand. That would be a daunting task to put it mildly, which is why I'm hoping the culprit is not WB here. The large exterior maps can contain hundreds if not thousands of chunks.

I mean my WB and my dat versions - the chunks as taken straight from the dats and the chunks as split by WB.

Me too. I'm using the same tools as you. The only difference is that I extracted the dats with Zane's dat extractor as opposed to the app in WB.

I will have to try re-extracting the dats in WB and comparing them against those I got with Zane's.

 

Re: Aziraxis Modification Pack

Okay, it's WB, specifically in the recombining step.

I think I inadvertently had WB-split chunks in my first floor inn dat folder, which is why I couldn't see the difference initially.

So anyway, being as the game doesn't seem to have any issues handling that one weird chunk in the inn itself, we'll just remove the override jpegs to go back to the better resolution for the mod. Have to look at a bunch of other maps too unfortunately.

~

Also I split this off from your thread so as not to take it over, Z. You may want to repost your fixes and whatnot there.

[edit]

Additional info ...

Upon looking more closely at things, I see that vanilla .jpeg chunks are actually 72 ppi, and for reasons that are not clear to me, WB resamples them to 96 ppi. In my admittedly limited understanding of DPI, I don't think this should affect anything, but who knows? Also, from what I can gather (according to some people at least), there is some inherent degree of image quality loss that occurs with opening and saving any .jpeg due to the format's lossy high compression characteristics.

 

Okay so far I have 'restored' the inn and the other three Hommlet interiors that were flipped horizontally for accuracy purposes. In the case of the latter three I just rebuilt the maps with flipped vanilla .jpeg chunks and left WB-recombining out of the equatiion. Also re-did Emridy Meadows, with the only override files being the chunks where the fissures are.

Unfortunately, in the case of anything that was customized from original ToEE art (which is basically everything, like the fissures above), the customization was done off of a WB-recombined version of the map. So we will be stuck with the image quality loss for those, as I'm not going to re-do all the art in the mod for the next 10 years.

 

JPEG, being a lossy format, is pretty difficult to preserve absolutely intact when different compression schemes and internal algorithms are used [when storing the original image and when resaving it in a different program]. There is nothing in ToEEWB that would purposefully downsample the recombined images or store them at a deliberately lower quality. All ToEEWB does is load up the images, put them all on a combined canvas, and save them back in the same basic format (jpeg, that is). The process of splitting is much the same. The loss in quality comes from two facts: the first is that .NET Framework might utilize a somewhat different approximation algorithm for pixels than originally used, and the second (I believe, more influential) is that the JPEG file format, being lossy, always utilizes at least some approximation to reduce the file size. As such, when resaving a modified JPEG file (split/recombined), some loss is bound to happen because we are talking about reapproximating an already approximated picture (and using a different algorithm, that is). I wish there were something I could do, but sadly I don't think I can - not unless ToEE used a different format (lossless, such as PNG) for its maps.

- Agetian

 

Could WB convert the files to bitmap, or simply operate with bitmaps or other lossless formats? (e.g., the user could convert vanilla chunks from JPEG to BMP, WB could recombine/split as BMP, and the user could reconvert the WB-processed BMPs to JPEGs.) When I manually work with the JPEG chunks and then manually resplit them one by one in Photoshop using the highest quality JPEG setting, I get no perceptible image degradation.

To put it another way, I believe that if I open a vanilla JPEG map chunk in Photoshop and convert it to BMP, there is no image degredation. Likewise, if I convert the BMP back to JPEG using the highest quality setting, there is no image degradation. What this means ultimately is that the original JPEG and the reconverted JPEG would be functionally and visually identical; I know that a native BMP image would show a marked loss in quality if simply converted to JPEG, even at the highest quality setting, simply because as you say JPEG is a lossy format. (It's reason for existence is in fact to be compressed, much like an .mp3 audio file.)

 

Sure thing, that sounds like a good idea! I'll post a version of ToEEWB later today that will output BMPs instead of JPGs (hope to find time... but I'm pretty sure I will), you can use that then to preserve quality within ToEEWB.

- Agetian

 

OK, posted an update to ToEEWB v2.0.5a that has a few extra buttons to recombine and split into the BMP format. Remember that it'll recombine from JPGs (the original game format) into a BMP, and will split from BMP to BMPs (so you'll have to manually convert those resplit BMPs into JPG for the game to use them; most advanced graphics editors, such as Photoshop and GIMP, should contain a way to batch-convert those tons of BMPs into JPEGs, but I haven't looked into that). Consider this release experimental, please let me know if it works for you and does the right thing - if not, I'll patch it up. Hope this helps!

- Agetian

 

Sweet Ag, thanks. :thumbsup:

I likely won't try it for a bit (busy with other stuff, v7.8 due next Sunday), but I'll let you know when I do.

 

Sure thing! I'll go ahead and code the new TFE-X in the meantime (no way I can make it before v7.8, so it's likely to go into the next revision, e.g. v7.9). Good luck with everything, my friend! Let me know if you need any assistance, I'll try to check back regularly.

- Agetian