Jump to content
  • 0

memory_decompress not working?


RickDangerous
 Share

Question

I'm experiencing problems using the memory_decompress kernal function.

I have some assets, that I compressed using "lzsa -r -f2 ... " and this I then decompress using the kernal function.
The first few kb of data are decompressed correctly, but then the decompression output starts to differ from the originally uncompressed data.

Does anybody else experience the same behavior?

for the record, my compressed data is ~8kb in size, the uncompressed size is ~22kb
I compiled the lzsa tool from the latest github sources (on windows) using the VC17 project - I'm using VC19 and it said it needs to upgrade the project to the windows SDK I'm using, but otherwise it compled fine.

Link to comment
Share on other sites

14 answers to this question

Recommended Posts

  • 0
On 11/13/2022 at 11:56 AM, desertfish said:

Sounds like you are either overwriting your compressed data with decompressed data, and/or are writing over the I/O and VERA registers starting at $9f00

 

Yes, but I don't see how that should even be possible when I use the function in "VRAM" mode - meaning the decompressor should always only write to the VERA d0 address.

I can see that it decompresses to VRAM, the display changes to some garbage remotely resembling the image it should be.
So maybe my input data is garbage? Causing the decompressor to go off the rails?

And of course there's also the chance that the kernal routine is buggy.. but I kind of doubt that.

Therefore I wanted to ask around if others have that problem too.

Link to comment
Share on other sites

  • 0

I've used the Kernal decompress function without any issues to unpack a help text to banked RAM. I've not tried to do anything else with it though.

Looking at the source code for memory_decompress in the file kernal/lzsa.s:

  • Line 52-57: If output address MSB (r1H) is $9f it sets register r3 to the address of putdst_io, else putdst_ram
  • This will happen if the destination is anywhere in the $9f00 page
  • putdst_ram stores a byte and increments the destination address. putdst_io will never increment the destination address
  • I guess decompress will work only if the destination address is $9f23 (DATA0) or ($9f24, DATA1, if you're using that)
  • And it will only work if you set an auto increment value

Are you doing all of this, and it still doesn't work, we might have a bug in the Kernal.

Edited by Stefan
Link to comment
Share on other sites

  • 0
On 11/13/2022 at 7:42 PM, Stefan said:

I've used the Kernal decompress function without any issues to unpack a help text to banked RAM. I've not tried to do anything else with it though.

Looking at the source code for memory_decompress in the file kernal/lzsa.s:

  • Line 52-57: If output address MSB (r1H) is $9f it sets register r3 to the address of putdst_io, else putdst_ram
  • This will happen if the destination is anywhere in the $9f00 page
  • putdst_ram stores a byte and increments the destination address. putdst_io will never increment the destination address
  • I guess decompress will work only if the destination address is $9f23 (DATA0) or ($9f24, DATA1, if you're using that)
  • And it will only work if you set an auto increment value

Are you doing all of this, and it still doesn't work, we might have a bug in the Kernal.

Yes, I'm doing all that, and I do get output in VRAM, its just garbage.

I tested uncompressing the file to regular RAM and compared it in the monitor to the uncompressed source data - the results where ok at the beginning, but after a few KB it started to differ wildly.
I'm still all safe in the basic ram here, so nothing should be overwritten.

I don't believe the kernal is wrong, that would be odd, especially if it has worked for you in the past.

Therefore I wonder if my input data is wrong? I compressed it with the right params, but I compiled lzsa myself, usibg VC19 instead of VC17, etc.

I've uploaded my uncompressed (intro_back.bin) and compressed (intro_bg.bin) files.
Would you mind compressing it with your lzsa and posting it here again?

Thanks,

Erik

intro_back.bin intro_bg.bin

Link to comment
Share on other sites

  • 0

Hi Stefan,

Thanks for your compression output.
However this has opened a can of worms:

  1. your compressed file differs from my compressed file  -  so I probably need to look into my compilation output
  2. When decompressing your file into RAM, it is "almost" correct, but still not correct - so it seems there is a problem with the decompression code?
  3. When decompressing into VRAM, it produces just garbage - which means that probably the kernal function writing to VRAM also has a bug.

Ok, so I'll go on a bug hunt on multiple fronts

Link to comment
Share on other sites

  • 0

BTW, if you are wondering what this all is about, here's a screenshot of the current compression output, when decompressing your file to memory and then copying to VRAM.

It clearly decompresses the 2k of the screenbuffer correctly and then about 1/3 of the tile data. 😕

nibbly.PNG

Link to comment
Share on other sites

  • 0
On 11/15/2022 at 8:17 PM, Ed Minchau said:

With SD storage, is  compression even necessary?

It's not so much about load speed but rather about storing more assets (sprites, etc.) in RAM than I can in VRAM.

And I just was delighted to find the uncompress function in the Kernal and wanted to play with it. By now I'm pretty sure that there is a problem in the Kernal, and I'm going to debug it. Just my daytime job is keeping me from doing it right now.

  • Like 4
Link to comment
Share on other sites

  • 0

Ok. so for whom is interested, I have a few updates on this topic:

  1. the kernal routine for decompressing into regular RAM works fine. (It seems I was drunk when I first tested it)
  2. However, the routine for decompressing into VRAM is broken

The reason for 2 is that the decompression algorithm wants to reference memory it has already decompressed. A backreference as it is called in the decompression sourcecode.
Basically, it detects that it needs to output a few bytes it has already seen, and wants to copy them from previously output memory to the current output location.

So when decompressing to VRAM the algorithm needs to use both VERA dataports one for reading and one for writing.
I'll work that out and will submit a pull request once it's done

  • Like 1
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

×
×
  • Create New...

Important Information

Please review our Terms of Use