That's my mega-commit you've all been waiting for.
The code for the shell share more routines with userspace apps than with kernel
units, because, well, its behavior is that of a userspace app, not a device
driver.
This created a weird situation with libraries and jump tables. Some routine
belonging to the `kernel/` directory felt weird there.
And then comes `apps/basic`, which will likely share even more code with the
shell. I was seeing myself creating huge jump tables to reuse code from the
shell. It didn't feel right.
Moreover, we'll probably want basic-like apps to optionnally replace the shell.
So here I am with this huge change in the project structure. I didn't test all
recipes on hardware yet, I will do later. I might have broken some...
But now, the structure feels better and the line between what belongs to
`kernel` and what belongs to `apps` feels clearer.
Instead of expecting a `USER_CODE` symbol to be set, we expect `.org` to be
set in all userspace glue code. This gives us more flexibility with regards to
how we manage that.
Moreover, instead of making `USER_RAMSTART` mandatory, we make it default to
the end of the binary, which is adequate in a majority of cases.
Will be useful for my upcoming mega-commit... :)
Most of register fiddling routines (which is now the only thing contained
in care.asm) are used by almost all userspace apps, often in inner loops.
That makes the penalty of using jump tables for those a bit too high.
Moreover, it burdens jump tables needlessly.
Because this unit is very small (now that string routines are out), it makes
sense to always include it in binaries.
There's a lot of looping through that table. At first, I wanted to add some
bisecting, but 16-bit additions and multiplications involved made the idea a
bit less appealing. I went with a very basic, hardcoded index which should
speed things quite a bit at a minimal complexity cost.
There's a couple of bit fiddling instructions that didn't have their
IX/IY variant implemented yet and without this commit, each of them
would have required a special routine. Not anymore.
The goal is to avoid mixing those routines with "character devices"
(acia, vpd, kbd) which aren't block devices and have routines that
have different expectations.
This is a first step to fixing #64.
* Optimised parsing functions and other minor optimisations
UnsetZ has been reduced by a byte, and between 17 and 28 cycles saved based on branching. Since branching is based on a being 0, it shouldn't have to branch very often and so be 28 cycles saved most the time. Including the initial call, the old version was 60 cycles, so this should be nearly twice as fast.
fmtHex has been reduced by 4 bytes and between 3 and 8 cycles based on branching.
fmtHexPair had a redundant "and" removed, saving two bytes and seven cycles.
parseHex has been reduced by 7 bytes. Due to so much branching, it's hard to say if it's faster, but it should be since it's fewer operations and now conditional returns are used which are a cycle faster than conditional jumps. I think there's more to improve here, but I haven't come up with anything yet.
* Major parsing optimisations
Totally reworked both parseDecimal and parseDecimalDigit
parseDecimalDigit no longer exists, as it could be replaced by an inline alternative in the 4 places it appeared. This saves one byte overall, as the inline version is 4 bytes, 1 byte more than a call, and removing the function saved 5 bytes. It has been reduced from between 52 and 35 cycles (35 on error, so we'd expect 52 cycles to be more common unless someone's really bad at programming) to 14 cycles, so 2-3 times faster.
parseDecimal has been reduced by a byte, and now the main loop is just about twice as fast, but with increased overhead. To put this into perspective, if we ignore error cases:
For decimals of length 1 it'll be 1.20x faster, for decimals of length 2, 1.41x faster, for length 3, 1.51x faster, for length 4, 1.57x faster, and for length 5 and above, at least 1.48x faster (even faster if there's leading zeroes or not the worst case scenario).
I believe there is still room for improvement, since the first iteration can be nearly replaced with "ld l, c" since 0*10=0, but when I tried this I could either add a zero check into the main loop, adding around 40 cycles and 10 bytes, or add 20 bytes to the overhead, and I don't think either of those options are worth it.
* Inlined parseDecimalDigit
See previous commit, and /lib/parse.asm, for details
* Fixed tabs and spacing
* Fixed tabs and spacing
* Better explanation and layout
* Corrected error in comments, and a new parseHex
5 bytes saved in parseHex, again hard to say what that does to speed, the shortest possible speed is probably a little slower but I think non-error cases should be around 9 cycles faster for decimal and 18 cycles faster for hex as there's now only two conditional returns and no compliment carries.
* Fixed the new parseHex
I accidentally did `add 0xe9` without specifying `a`
* Commented the use of daa
I made the comments surrounding my use of daa much clearer, so it isn't quite so mystical what's being done here.
* Removed skip leading zeroes, added skip first multiply
Now instead of skipping leading zeroes, the first digit is loaded directly into hl without first multiplying by 10. This means the first loop is skipped in the overhead, making the method 2-3 times faster overall, and is now faster for the more common fewer digit cases too. The number of bytes is exactly the same, and the inner loop is slightly faster too thanks to no longer needing to load a into c.
To be more precise about the speed increase over the current code, for decimals of length 1 it'll be 3.18x faster, for decimals of length 2, 2.50x faster, for length 3, 2.31x faster, for length 4, 2.22x faster, and for length 5 and above, at least 2.03x faster. In terms of cycles, this is around 100+(132*length) cycles saved per decimal.
* Fixed erroring out for all number >0x1999
I fixed the errors for numbers >0x1999, sadly it is now 6 bytes bigger, so 5 bytes larger than the original, but the speed increases should still hold.
* Fixed more errors, clearer choice of constants
* Clearer choice of constants
* Moved and indented comment about fmtHex's method
* Marked inlined parseDecimalDigit uses
* Renamed .error, removed trailing whitespace, more verbose comments.
* addHL and subHL affect flags, and are smaller
Most importantly, addHL and subHL now affect the flags as you would expect from a 16 bit addition/subtraction. This seems like it'd be preferred behaviour, however I realise any code relying on it not affecting flags would break. One byte saved in addHL, and two bytes saved in subHL. Due to the branching nature of the original code, it's difficult to compare speeds, subHL is either 1 or 6 cycles faster depending on branching, and addHL is between -1 and 3 cycles faster. If the chance of a carry is 50%, addHL is expected to be a cycle faster, but for a chance of carry below 25% (so a < 0x40) this will be up to a cycle slower.
* Update core.asm
* Reworked one use of addHL
By essentially inlining both addHL and cpHLDE, 100 cycles are saved, but due to the registers not needing preserving, a byte is saved too.
* Corrected spelling error in comment
* Reworked second use of addHL
43 cycles saved, and no more addHL in critical loops. No bytes saved or used.
* Fixed tabs and spacing, and made a comment clearer.
* Clearer comments
* Adopted push/pop notation
I've tested RAM usage when self-assembling and there weren't as high
as I thought. zasm's defaults now use less than 0x1800 bytes of RAM,
making it possible, theoretically for now, for a Sega Master System
to assemble Collapse OS from within itself.
Yup, that's ultimately why I've just made this whole big zasm
refactoring in the previous commits. To allow for this.
But also, zasm is in much better shape now...
I'm about to split the global registry in two (labels and consts)
and the previous state of registry selection made things murky.
Now it's much better.
The dual scraptchpad thing doesn't work. Things become very
complicated when it's time to write that back to the file. We
overwrite our contents and end up with garbage.
This hard-binds ed to the filesystem (I liked the idea of working
only with blockdevs though...), but this is necessary for the
upcoming `w` command. We need some kind of way to tell the
destination to write to truncate itself.
This only has a meaning in the filesystem, but it's necessary to
let the file know that its registered file size has possibly
shrunk.
I thought of alternatives that would have allowed me to keep ed
blkdev-centered, but they were all too hackish to my own taste.
Hence, this new hard-bind on files.
During expression parsing, if a local label was parsed, it would
select the local registry and keep that selection, making
subsequent global labels register in the wrong place.
That was an interesting bug. It didn't cause a problem in emulation, but
in an RC2014 on an SD card, an include that didn't end with two newlines
would cause an infinite loop.
Needed if we want to compile the kernel and zasm from within a SD card.
I didn't go straight for 32-bit because it was significantly more
complex and 24-bit give us 16M. Enough to go on for a while...
A new app to stress test the SD card driver. Also, accompanying this
commit, changes solidifying the SD card driver so that stress tests
actually pass :)
This huge refactoring remove the Seek and Tell routine from blockdev
implementation requirements and change GetC and PutC's API so that they
take an address to read and write (through HL/DE) at each call.
The "PTR" approach in blockdev implementation was very redundant from
device to device and it made more sense to generalize. It's possible
that future device aren't "random access", but we'll be able to add more
device types later.
Another important change in this commit is that the "blockdev handle" is
now opaque. Previously, consumers of the API would happily call routines
directly from one of the 4 offsets. We can't do that any more. This
makes the API more solid for future improvements.
This change forced me to change a lot of things in fs, but overall,
things are now simpler. No more `FS_PTR`: the "device handle" now holds
the active pointer.
Lots, lots of changes, but it also feels a lot cleaner and solid.
What used to be `tools/emul/user.h` was in fact specific to zasm, so I
moved it there.
To avoid name confusion, I renamed what used to be kernel.h and user.h
to kernel-bin.h and user-bin.h.
To run a parseExpr on first pass would always return a false success
with dummy value because symbols are configured to always succeed on
first pass. This would make expressions like ".fill 0x38-$" so bad
things to labels because "0x38-$" wouldn't return the same thing on
first and second pass.
Revert to parsing literals and symbols after having scanned for
expressions and add a special case specifically for char literals (which
is why we scanned for literals and symbols first in the first place).
It was a bad idea to remove it. Now that I'm introducing the concept of
a per-app glue file, it becomes much easier to build emulated zasm as a
userspace app.
Instead of buffering input in memory one line at a time, we go in "just
in time" mode and always read contents directly from I/O, without
buffering.
It forces us to implement a `ioPutback` scheme, but on the other hand it
greatly simplifies cases where multiple tokens are on the same line
(when a label is directly followed by an instruction).
The end result feels much more solid and less hackish.
This will soon allow use to seek and tell on input, which is necessary
for a second pass which is necessary for forward symbol references.
This require making `blkSel` a bit more flexible. Rather than having one
global selected blkdev, each app can select its own, in its own memory
space.
