Good News for the DS port

Last week, I restructured the DS loader to use a SeekableReadStream for file reading. Once that was in place and working, I restructured the PS2 loader to use a SeekableReadStream as well and tested it (this needed to be done anyway and it helped me ensure there was nothing inherently wrong with the way I was using SeekableReadStream in the DS loader).

I then had problems reading the symbol table in successfully. After some wasted time :P, I realized it was just a silly mistake. I had neglected to initialize _symtab_sect to anything in dsloader.h and the value of uninitialized ints is undefined in C++. Thus, the loader was detecting, from the value that _symtab_sect had, that the symbol table was already read in when it wasn't.

At this point the loader was getting to the point where I had to deal with the relocations. I used consolePrintfs to make sure that the relocation types the loader was detecting were the same as the ones detected by arm-eabi-objdump (they were). I then began coding for these relocations. After a day of work on the relocations with no success as far as getting games to run with dynamic plugins enabled on the DS, I decided to disable the thumb-interworking on the ds temporarily so I could work on getting the DS with plugins in a runnable state without having to deal with the complication of thumb instructions/relocations.

I worked further on the relocations yesterday; I started dumping the instructions to be relocated and immediately noticed a MAJOR problem. Every instruction was 0x00000000! The addresses of the instructions seemed reasonable, though, so I suspected it had something to do with how the plugins were being built and not how I was retrieving target addresses. I dumped Mapfiles with "-Map" during plugin linking and discovered that whole input sections were being discarded, among other problems. After switching out the linker script for a modified version of the default "arm-eabi-ld" linker script, these problems seemed to be fixed, perhaps because the modified linker script I was using (based on the script used by the main executable) used the MEMORY command to map out different blocks of memory but I was trying to force the start address of the ".text" section to be 0. I am not sure whether switching out this linker script will cause problems.

I then did a fair bit of reading on the ARM instruction set and continued tweaking/testing the relocation code. Things still weren't working like they should have. Yotam suggested comparing completely unrelocated code and semi-relocated code to see what ld was already doing for me, and through a few dumps, I realized I was trying to do A LOT that ld already did redundantly and thus messing up things that were already fixed! After removing this redundant code, I got Beneath a Steel Sky to run successfully with Dynamic Plugins enabled on the DS!

My next order of business, then, is to test the rest of the engines (some of which have relocation types I still haven't dealt with). Then, I'll reintegrate the thumb-interworking and deal with any major complications that brings (if any :D).

~Tony

Also, I plan to update my wiki schedule shortly: http://wiki.scummvm.org/index.php/User:Toneman

a size-able problem with plugins

While working on changing how the loader opens and reads plugin files on the DS this weekend, I noticed that the ".plg" files produced in my builds were significantly smaller than expected (as in under a hundred kilobytes).

So I looked through verbose build output to investigate what the problem could be. The plugin linking output included all the necessary object files, so I knew the problem wasn't there and decided to look closer at the plugin flags.

Last week when I added the PLUGIN_LDFLAGS into the ds/arm9/makefile, I copied over the regular LD_FLAGS to make sure all the special DS optimizations that occur when linking the main executable would be used for the plugin files as well (stuff like "-mthumb-interwork"). One of these flags was "--gc-sections" which I believe is meant to garbage collect sections full of unused functions and/or data. Since both the main executable and the plugin files were using this flag (but plugins aren't truly linked in to the main executable until run-time), functions and data in the main executable and the plugins that point to each other and nowhere else were mistakenly garbage-collected. After removing the "--gc-sections" flag from LDFLAGS and PLUGIN_LDFLAGS, plugins are a much more reasonable size. Unfortunately, this means the main executable is a bit more bloated than before, which runs counter to the goal of using as little memory as possible on the RAM-starved DS...

But for now, I'll be moving on to reworking the loader to use "Common::SeekableReadStream" to read plugin files.

~Tony

plugins linking on DS

I worked through some build errors, switched my plugin.ld to use a modified version of the correct linker script and added a number of PLUGIN_LD_FLAGS (including "-mno-crt0 $(DEVKITPRO)/devkitARM/arm-eabi/lib/ds_arm9_crt0.o"), and plugins started linking. I had some trouble with plugin.ld syntax errors, but figured it out (I was doing things in SECTIONS like "{ contents } >region =fill :phdr" when it needed to be "{ contents } >region :phdr =fill"). Using objdump on the plugins, it seems most only have one relocation type to worry about (R_ARM_ABS32), but the cine and cruise engines also use R_ARM_THM_CALL and R_ARM_TARGET1. I'll be working on completing dsloader.cpp (including the ARM-specific relocations) now, my first milestone being getting one of the engines that uses just R_ARM_ABS32 working.

DS Update

Work on the DS dynamic module loader is progressing. In the tail end of last week, I put the basic loader code in for the DS and modified the default arm-eabi linker script to make a plugin linker for the DS. This week I found (in a .spec file included in one of the makefiles for the DS) that the DS port doesn't use the default arm-eabi linker so I'll have to modify the one it does use (ds_arm9.ld) instead. At the opening of this week, I took some time to try to understand the Makefiles for the DS (it's a bit more complicated than the other ports I've worked with) and then started modifying them to use dynamic modules. I had a bit of trouble getting make to attempt to link the plugins, but after correcting some dependencies, linking of plugins is attempted (though not completed successfully, which hopefully will change once I switch to a modified version of ds_arm9.ld for plugin linking). I'll post again once the plugins are successfully linking.

:-)

Tony

This week has not been the most productive thus far (though it's only just begun). I've been trying to get engine plugins building for the ds (using the custom "plugin.ld" linker script) so I can begin work on the ARM-specific relocations needed for the DS, but the way I've modified the DS makefiles, engine plugins don't build. (I currently get things like "warning: overriding commands for target 'plugins/scumm.plg'"). I'll be contacting Niel to better understand the DS make system shortly.

New Schedule up on Wiki

Check it out :-)

http://wiki.scummvm.org/index.php/User:Toneman