More Awesome Than You!
Welcome, Guest. Please login or register.
2019 October 22, 10:32:36

Login with username, password and session length
Search:     Advanced search
539951 Posts in 18023 Topics by 7846 Members
Latest Member: Bitick
* Home Help Search Login Register
  Show Posts
Pages: [1]
1  The Bowels of Trogdor / The Small Intestines of Trogdor / Re: New DBPF library on: 2007 June 01, 15:36:10
New release that should fix the unistd.h problem.
2  The Bowels of Trogdor / The Small Intestines of Trogdor / Re: New DBPF library on: 2007 May 30, 23:59:45
I just posted a new version of the library. The changes aren't very interesting. If you're a programmer, see the changelog in the zip archive for a list. If you're not a programmer, you can switch to the new version of dbpf-recompress.exe, but you probably won't notice any difference.

Mere mortals, as opposed to talented programmers such as yourself, would get laughed at for even considering paying the $40.

Oh, I know it's a spoof. But what it looks like is a domain with nothing on it but the spoof pages. You'd never guess that there's a real forum with real Sims 2 content on the same site.

Quote
Is this the way they look before compression (i.e. 0x95 0x01)?

Yes, that's before compression (or after decompression).
3  TS2: Burnination / Peasantry / Re: The Compressorizer! Mass DBPF Compressing Program on: 2007 May 30, 20:02:32
I originally posted this reply in the "New DBPF library" thread, but I'm copying it here since the original message was in both threads.

Normal compressed SHPE ref, in Hex view in SimPE: 66 42 72 61 6e 64 69... etc.  This is 1 byte for the string length (0x66 - 102 bytes) and then immediately followed by the actual string (Brandi...).
For a compressorised SHPE ref, in Hex view in SimPE: 95 01 62 72 61 6e 64 69... etc.  This is 1 byte for the string length (0x95 - 149 bytes) and then a 01 and then followed by the actual string (brandi...)

SimPE uses System.IO.BinaryReader.ReadString() and its BinaryWriter counterpart for most string serialization. These do not use a single byte for the length -- they use a weird base-128 encoding documented here: http://msdn2.microsoft.com/en-us/library/system.io.binarywriter.write7bitencodedint.aspx. In this encoding a length of 0x95 is serialized as 0x95 0x01. I'm almost certain that this is your problem, not anything to do with compression.

I don't know whether this is a bug in SimPE or whether it actually is the encoding used by The Sims. If the former, you may have to use an ugly workaround anyway.
4  The Bowels of Trogdor / The Small Intestines of Trogdor / Re: New DBPF library on: 2007 May 30, 16:41:58
Hi, following up on a bunch of stuff:

The problem with compressed duplicate entries is that there are two different entry lists in the file. One lists the type, group, instance, file offset, and (compressed) size for all entries, and the other lists the type, group, instance, and uncompressed size for all compressed entries. In order to figure out what's compressed you have to merge the two lists by type/group/instance, and there's no way to do that when there are duplicate entries. It's a stupid design, but it's not SimPE's fault.

Normal compressed SHPE ref, in Hex view in SimPE: 66 42 72 61 6e 64 69... etc.  This is 1 byte for the string length (0x66 - 102 bytes) and then immediately followed by the actual string (Brandi...).
For a compressorised SHPE ref, in Hex view in SimPE: 95 01 62 72 61 6e 64 69... etc.  This is 1 byte for the string length (0x95 - 149 bytes) and then a 01 and then followed by the actual string (brandi...)

SimPE uses System.IO.BinaryReader.ReadString() and its BinaryWriter counterpart for most string serialization. These do not use a single byte for the length -- they use a weird base-128 encoding documented here: http://msdn2.microsoft.com/en-us/library/system.io.binarywriter.write7bitencodedint.aspx. In this encoding a length of 0x95 is serialized as 0x95 0x01. I'm almost certain that this is your problem, not anything to do with compression.

I don't know whether this is a bug in SimPE or whether it actually is the encoding used by The Sims. If the former, you may have to use an ugly workaround anyway.

And out of curiosity, do you actually play the game? How did you find us here?

I'm not really a gamer, I'm just interested in game design. When I do play games it's usually with as many cheats and walkthroughs as I can get my hands on, which for some reason strikes many people as somehow immoral.

I think I found you through a thread in ModTheSims2. How are people supposed to find you? Or aren't they? It isn't exactly obvious how to get here from the top page, unless paying the $40 actually works.
5  TS2: Burnination / Peasantry / Re: The Compressorizer! Mass DBPF Compressing Program on: 2007 May 12, 23:08:48
People who are experiencing crashes: if you have the time, please try to narrow down the problem. If you can find a single file which causes a crash when it's recompressed, and post it here, I can probably fix the bug.

The command-line tool verifies that all of the recompressed files match the originals, and backs out the changes if they don't, so it should be safe, but I may have overlooked something.
6  The Bowels of Trogdor / The Small Intestines of Trogdor / Re: New DBPF library on: 2007 May 04, 02:25:18
And welcome to the ranks of Peasants At Least As Tall As The Beefy Arm, whoever you are.

Um, I guess I should have introduced myself. Hi, I'm Ben. I've never posted on a Sims forum before. I'm a PhD student in the theory of programming languages. One of my hobbies is writing decompilers for obscure virtual machines, like these: http://www.darkweb.com/~benrg/if-decompilers/. I thought that a SimAntics decompiler and recompiler might be an interesting project, and as a first step I needed a DBPF read/write library, so I looked around for one, and discovered that it was a most-wanted item, so I wrote it. Of course it took four times as long as I expected (Hofstadter's law), and the decompiler will probably never see the light of day, but in any case the library is finished (modulo bugs), so I'm happy.

As everyone else has figured out already, Jack Sparrow's hair has index entries sharing the same type/group/instance ID, and because compression status is indexed in the file by type/group/instance ID, it's impossible to have duplicate entries unless none of them are compressed. How should the library handle this? It could:

  • Refuse to even open files with duplicate index entries, because they're broken. Probably a bad idea, since at least one exists.
  • Open them but refuse to write them. Clients like dbpf-recompress would have to check for duplicates and... do what? Discard all but one? Compare them to make sure they're equal?
  • Allow opening and writing, always writing the duplicate entries uncompressed, unless the write_disposition makes that impossible, in which case fail. This is maximally flexible but seems pointlessly complicated for a rare and useless case, and makes pointlessly larger files. It wouldn't be too hard to implement, though.

-- Ben
7  The Bowels of Trogdor / The Small Intestines of Trogdor / New DBPF library on: 2007 May 03, 21:11:52
I think this belongs in the Bowels of Trogdor, but I don't seem to have posting permission there.

This is a DBPF (*.package) library written in C++ with a C interface, GPL licensed. It has full write support with incremental and in-place updates and compression. The compression code is based on zlib's deflate code with lazy matching, and almost always compresses better than SimPE or the game itself (sometimes a lot better). I also included a sample app / useful utility called dbpf-recompress which uses the library to recompress package files. It shaves about 100 megabytes off the size of a base Sims 2 installation, and everything seems to still work.

This is a very preliminary release and some things are completely untested. Please test dbpf-recompress on every package you can lay your hands on (after backing them up!). And please use the library to build useful tools, otherwise it will all have been in vain.

Edit: New version 20070530. See the changelog in the zip archive.

Edit: New version 20070601.
Pages: [1]
Powered by MySQL Powered by PHP Powered by SMF 1.1.21 | SMF © 2015, Simple Machines Valid XHTML 1.0! Valid CSS!
Page created in 0.175 seconds with 19 queries.