Native assembler, preferably open source

[Stefan]

Hello,

Is there an assembler for 6502 that could be made to run natively on the X16?

Preferably it would:

• Be open source
• Read source code from files
• Assemble directly to files
• Support at least named labels, basic compiler directives (for instance .byte, .word, .text) and macros, simliar to the Commodore Macro Assembler that you could use on the C64

I've been searching, but it seems hard to find.

I used the Commodore Macro Assembler a lot with my C64 in the 80s. Even though I had a disk drive, it was cumbersome and took quite some time to edit, save, assemble, load and test each change in a program. If the real X16 reads and writes SD card files at the same speed as the emulator, we would be getting much closer to a modern experience with a file based assembler.

[Stefan]

Hi,

Yes the editor reads the whole file, and then refreshes the screen.

EDIT: I think it's sufficient if the assembler has one common read_line function, so that it's easy to expand it in the future to receive code from other sources (e.g. memory). It would also be nice to have one common write_byte function, so that it's easy to let the assembler write program code to different destinations, if need be (file, memory or whatever). These are just my thoughts.

Edited January 12, 2021 by Stefan

[desertfish]

I need some help to find out where the difference in loading speed comes from..... My program doing just the bare minimum (load a byte with CHRIN, store it in $02, loop until eof) takes almost 8 seconds to finish.

Reading the same file into x16edit takes around 4 seconds, and it is doing a lot more while loading ... I don't get it

Can anyone take a look perhaps?   Here is my program:

.cpu "65c02"
*=$0801
        ; 10 sys 2061
        .byte $0b, $08, $0a, $00, $9e, $32, $30, $36
        .byte $31, $00, $00, $00

start

    phx

    ldx  #<_filename
    ldy  #>_filename
    lda  #10
    jsr  $FFBD      ; SETNAM
    jmp  _go
_filename
    .text  "romdis.asm"

_go
    lda  #0
    tax
    tay
    jsr  $FFDB      ; SETTIM  0,0,0

    lda  #1
    ldx  #8
    ldy  #0
    jsr  $FFBA       ; SETLFS   1,8,0
    jsr  $FFC0       ; OPEN
    ldx  #1
    jsr  $FFC6       ; CHKIN, use #1 as output channel
    ;lda  #'.'
    ;jsr  $FFD2       ; CHROUT

    ; load the file ....
_loop
    jsr  $ffb7       ;READST
    bne  _eof
    jsr  $FFCF       ;CHRIN
    sta  $02         ; store...
    jmp  _loop

_eof
    ; close stuff
    jsr  $FFCC       ;CLRCHN
    lda  #1
    jsr  $FFC3       ;CLOSE

    ; print the time taken
    jsr  $FFDE       ; RDTIM -> A,X,Y
    tay
    txa
    jsr  $fe03       ; GIVAYF
    jsr  $fe81       ; FOUT
    sta  2
    sty  3
    ldy  #0
_printlp
    lda  (2),y
    beq  _endstr
    jsr  $FFD2    ; CHROUT
    iny
    bne  _printlp
_endstr
    plx
    rts

 

And an sdcard image containing the "romdis.asm" text file is also attached including the 'loadtest.prg' program above

 

loaddtest-img.zip

[Stefan]

Hi,

I cannot see anything obvious.

It seems that X16 Edit loads the complete file in about half the time. That's 8,208 lines of code. Blocks free in the editor before load=1,983 and after=1,692, diff=291 blocks x 251 bytes => the file is about 73,041 bytes. Seems to match the reported file size if the sdcard image is mounted in the host file system.

The secondary address is different in your code compared to the editor, but that doesn't seem to affect load time.

Another difference is that the editor appends ",s,r" to the file name for sequential file read. But this doesn't either seem to affect load time.

We have to keep looking...

[desertfish]

That's really odd.   Is X16edit doing something before loading files?  Changing vectors, disabling IRQ, or something else?

Otherwise I may have to try to integrate your I/O routine in my program to see if that makes a difference...

[Stefan]

The editor is essentially a custom IRQ handler. Everything, including the file read function, is run within the interrupt handler. So a file read blocks further interrupts from happening.

However, if interrupts are disabled in your code, that doesn't help (much). The timer doesn't work then, but I manually clocked it to be 7,8 s.

I don't know which vectors you are thinking about. But I would say the program does nothing special before starting the file load.

[desertfish]

Yeah I tried several things already but they didn't change anything (disabling interrupts, adding ",s,r" to the filename, using a different secondary device number).    Very peculiar,  I'd like to solve this issue but it seems it will require a lot more trial and error time than I anticipated so I think I'll put it on the back burner for now.     For smaller files it's less noticable but we didn't want to do this project for small files 

[Stefan]

I have done some more tests, and written a completely new test program. These are my findings.

• My test program is as fast or slow as your test code, i.e. loading the source file in about 8 seconds.
• That is if the test program is run as a "normal program", and not within an interrupt.
• If I setup the IRQ vector in $0314-15 and makes it point to the start of my program, so that it is run as part of the interrupt (the same as X16 Edit) the file read is done in about 4 seconds without changing the code that's responsible for reading the file.
• Can this be the true behavior of the machine? Or is it some kind of bug in the emulator?

test.asm

Edited January 13, 2021 by Stefan

[desertfish]

Thanks a lot Stefan, to take the time to investigate and discover the unexpected factor of difference here!

Running the I/O inside an IRQ handler sounds too much of a weird kludge and strikes me as not reflecting the true behavior of the physical machine, but rather an artifact of the emulator.
I'll stick with just writing our I/o loops as we were doing now and not think much more of it for the time being.

[desertfish]

Here's a version of the assembler to play with.   It doesn't support symbols yet so stick with basic assembly.  It also only writes output directly to memory for now.

The video shows it in action on a simple hello world program:

assem.prg

[Stefan]

From the discussion in the File I/O Performance thread I think we know how to optimize the speed of file reading:

• Disable interrupts
• Select Kernal ROM before the read loop
• Maybe getting the status value directly from memory instead of calling READST

The assembler you're working on is very promising.

I guess the next big step is supporting symbols/labels.

These are the thoughts I have had on that subject:

• The symbol table entries could have a fixed length, making it easier to store and search them
• The symbol names could be long but not more than maybe 256 characters; to limit the size of the table you could opt store not the whole name, but for instance this information that is used for searching/matching symbols:
  • A suitable number of characters from the start of the symbol name (for instance 6 characters)
  • The total length of the symbol (Forth does this)
  • An 8 bit checksum (for instance just adding the character ASCII values together).
• The symbol search could start from the end of the table. This makes it possible to redefine symbols, as the assembler will find the symbol that was defined last. This method is used in Forth.
• Redefined symbols could be used to support scopes, which I find very valuable in ca65. When starting a new scope the assembler needs to store the index of the last item in the symbol table. New symbols defined within the new scope are added to the symbol table at the end. When checking for not allowed duplicate symbols, the search would start from the end of the table and stop at the index where the scope started. And when leaving the scope the pointer to the last symbol would be reset to the index before the scope started, effectively forgetting all symbols created within the scope. These are not in any way my ideas, it is just a combination of Forth and David Salomon's book that I mentioned above in this thread.

[desertfish]

I think I'll make a separate github repo for the assembler. 

It's now "an example" in the prog8 repository but I feel it is growing way beyond that.

Also I don't think I have the time to implement everything myself and I much rather would like to see this as a joint effort!  It's meant to be open-source after all

Great tips though Stefan. I am quite confused still though by the "select kernal ROM" -- isn't it there by default? How else could we call CHRIN()  and its brethren?

[Stefan]

Great! Github is a great alternative for a project like this.

As to the "select kernal ROM" issue. I have not done any thorough investigation, but this is what I think is happening.

Basic is in ROM bank 4, and the Kernal is in ROM bank 0. You may see how the X16 ROM is put together in the makefile. You may also do PRINT PEEK($9F60) at the basic prompt, which returns a 4.

When you start a program from Basic, you will continue to have ROM bank 4 selected unless you change it.

In the Basic ROM bank, there are corresponding jump instructions to Kernal public functions at the same addresses as in ROM bank 0. But is is not the same content, and the actual Kernal functions are not also stored in ROM bank 4. For instance, FFD2 in bank 4 = JMP $FD53. In bank 0 this is JMP ($0326).

If you follow the execution path in bank 4 from $FD53 I would say that you will eventually end up at an instruction that makes a bank switch, and then calls the function in ROM bank 0. And when it's done, it will switch back to bank 4.

There's quite a lot going on preparing for the bank switch, explaining the delay.

The solution seems to be an easy one. An assembly program that uses Kernal functions should switch to ROM bank 0 at start, and restore the ROM bank before program exit.

Edited January 16, 2021 by Stefan

[desertfish]

That’s a nice description I think I’m starting to understand now what’s going on. Will try it in the assembler tomorrow. Not a big fan of disabling the Irq though, I wonder if we can leave that on default. 

 

edit:  it works - selecting rom bank 0 for the kernal rom, saves a couple of seconds in the assembler, the large file now takes 9.1 seconds to assemble where previously it took 11.5 seconds. 
I think we have to load the file at least two times soon (when substituting symbols) so this time saving is going to add up.  This nice little hack is staying for now

 

Edited January 17, 2021 by desertfish

[mjallison42]

Sorry for being late to the party. I'm currently using CC65 tool chain since it provides relocatable object models and a capable linker, plus integration with C. On top of that, since it runs on my modern machine, I have access to GIT (or what ever SCS you want). With out a source code control system, life ain't worth living!

I totally get the idea of having a native development system, but modern editors (although I use Emacs) and source control are too much of an ask for me. 

Combine with the X16 emulator, and it's a pretty fast edit-compile-debug cycle.

[desertfish]

The file-based assembler project can now be found in its own github repo:

https://github.com/irmen/cx16assem

 

 

[TomXP411]

10 hours ago, mjallison42 said:

Sorry for being late to the party. I'm currently using CC65 tool chain since it provides relocatable object models and a capable linker, plus integration with C. On top of that, since it runs on my modern machine, I have access to GIT (or what ever SCS you want). With out a source code control system, life ain't worth living!

I totally get the idea of having a native development system, but modern editors (although I use Emacs) and source control are too much of an ask for me. 

Combine with the X16 emulator, and it's a pretty fast edit-compile-debug cycle.

Ugh. I started looking at cc65 the other day and kind of got overwhelmed. It's like starting over, with completely different code for project management than I'm used to. I'm starting to feel like that's where I need to go, however, since I'm coming up short with other tools, and cc65 is what the ROMs are being developed in, after all...

 

[Stefan]

1 hour ago, TomXP411 said:

Ugh. I started looking at cc65 the other day and kind of got overwhelmed. It's like starting over, with completely different code for project management than I'm used to. I'm starting to feel like that's where I need to go, however, since I'm coming up short with other tools, and cc65 is what the ROMs are being developed in, after all...

 

The assembler and linker in the cc65 suite are very nice. It's true that you feel overwhelmed at first, especially learning to use the linker config files that control where code ends up in memory.

It's not likely that a native assembler will ever compete on par with cc65.

That said, I am convinced that we may create a really good native assembly programming experience on the X16. That's worthwhile, as without native programming alternatives, the X16 could easily become nothing more than a game console with a cumbersome way of starting the games. 

Not one person can do all that work, creating source code editors, assemblers, and compilers or interpreters for different languages. We need to divide the tasks and work together on projects wherever that's effective. Using standards, such as plain text files, is a reasonable step in that direction.

I lived a long time before ever hearing about source control software. That was a life worth living too.

[desertfish]

I started programming seriously on the Amiga and my source control consisted of a stack of floppies (and later a set of numbered directories on the hard disk drive).  It sortof kinda worked mostly because I was the sole programmer, and because the tool support (editor, compilers) on the Amiga itself was extremely good for the time. While it's impossible to obtain that same level on the commander x16 I still think Stefan has some valid points - being able to create nontrivial programs on the target system itself is an important selling point.  And I dont think basic v2 can fill that spot because it is very lackluster (in my opinion).

 

[pzembrod]

Great work, @desertfish! Really cool to use one's own programming language for such a project.

In the meantime I have also done a little research. I asked for likely porting candidates on 6502.org: http://forum.6502.org/viewtopic.php?f=2&t=6475. The Editor Not Included assembler mentioned there looks interesting to me; I'm currently trying to find out if the sources are available.

And my VoksForth partner, an old Atarian, pointed me to Bibo Assembler - https://atariwiki.org/wiki/Wiki.jsp?page=BiboAssembler - where the sources are available. Unfortunately the handbook is in German.

[Stefan]

40 minutes ago, pzembrod said:

In the meantime I have also done a little research. I asked for likely porting candidates on 6502.org: http://forum.6502.org/viewtopic.php?f=2&t=6475. The Editor Not Included assembler mentioned there looks interesting to me; I'm currently trying to find out if the sources are available.

And my VoksForth partner, an old Atarian, pointed me to Bibo Assembler - https://atariwiki.org/wiki/Wiki.jsp?page=BiboAssembler - where the sources are available. Unfortunately the handbook is in German.

Very interesting. Hope you find the sources. I guess there is some tweaking to get those to work on X16. Wonder how much tweaking.

[Dejital]

On 12/27/2020 at 10:16 AM, Stefan said:

Hello,

Is there an assembler for 6502 that could be made to run natively on the X16?

Preferably it would:

• Be open source
• Read source code from files
• Assemble directly to files
• Support at least named labels, basic compiler directives (for instance .byte, .word, .text) and macros, simliar to the Commodore Macro Assembler that you could use on the C64

I've been searching, but it seems hard to find.

I used the Commodore Macro Assembler a lot with my C64 in the 80s. Even though I had a disk drive, it was cumbersome and took quite some time to edit, save, assemble, load and test each change in a program. If the real X16 reads and writes SD card files at the same speed as the emulator, we would be getting much closer to a modern experience with a file based assembler.

Back in 1986 I wrote an assembler for the C64. It has everything you list above, except macros. It is also BY FAR the fastest assembler I have ever seen for the 64, assembling itself (6K object code, from 19K of source code) in only 12.5 seconds. To compare, Compute! magazine's LADS assembler, quite popular in the day, took almost two minutes to self-assemble RAM-to-RAM.

I still have the source code on floppy. Whether it's readable is another question. Porting to the X16 should be pretty easy given Commodore Kernal and BASIC V2.

If anyone can point me to a conversion service in the Ottawa, Canada area I'd be willing to try recovering the source code, posting to Github, and porting to X16.

[pzembrod]

On 2/7/2021 at 3:57 AM, Dejital said:

Back in 1986 I wrote an assembler for the C64. It has everything you list above, except macros. It is also BY FAR the fastest assembler I have ever seen for the 64, assembling itself (6K object code, from 19K of source code) in only 12.5 seconds. To compare, Compute! magazine's LADS assembler, quite popular in the day, took almost two minutes to self-assemble RAM-to-RAM.

I still have the source code on floppy. Whether it's readable is another question. Porting to the X16 should be pretty easy given Commodore Kernal and BASIC V2.

If anyone can point me to a conversion service in the Ottawa, Canada area I'd be willing to try recovering the source code, posting to Github, and porting to X16.

That's fascinating! Did you ever publish it anywhere? What is it called?

By conversion service, you mean a service to get 1541 disks onto a modern machine, i.e. create a d64 disk images from them? Do you still have your 1541? And possibly your 64?

 

On a separate note, I got lucky and found the author of ENI Assembler: http://forum.6502.org/viewtopic.php?f=2&t=6475

Caveat is that there are a few challenges involved in accessing the source code. Best read his post linked above.

[Michael Parson]

On 2/6/2021 at 8:57 PM, Dejital said:

I still have the source code on floppy. Whether it's readable is another question. Porting to the X16 should be pretty easy given Commodore Kernal and BASIC V2.

If anyone can point me to a conversion service in the Ottawa, Canada area I'd be willing to try recovering the source code, posting to Github, and porting to X16.

Find a local C-64 group, a city the size of Ottawa probably still has one.  I'm sure it's full of people that could help you with that.

[Dejital]

2 hours ago, pzembrod said:

That's fascinating! Did you ever publish it anywhere? What is it called?

At the time I called it the Commodore 64 TurboAssembler. Probably not so suitable today as Borland beat me to it.

It was actually released on the Ottawa Home Computing Club disk of the month, perhaps back in 1990 or thereabouts. Good luck finding THAT.

2 hours ago, pzembrod said:

By conversion service, you mean a service to get 1541 disks onto a modern machine, i.e. create a d64 disk images from them? Do you still have your 1541? And possibly your 64?

I mean a way to read my 35 year old floppies and hope that they are still readable. I no longer have my 64 or 1541. If I can borrow a 1541 I'm willing to spring for one of the various adapters that allow connection to a Linux machine and the use of special software.

Once I get the files I can easily fire up an emulator, make the one change that I'd like to make, and then post to GitHub.

There are a couple of issues that will make a port to the X16 a bit challenging but we can cross that bridge when we come to it.

 

2 hours ago, pzembrod said:

On a separate note, I got lucky and found the author of ENI Assembler: http://forum.6502.org/viewtopic.php?f=2&t=6475

Caveat is that there are a few challenges involved in accessing the source code. Best read his post linked above.

 

[desertfish]

Worked a bit more on my assembler.

Symbol support is better now and you can use .str to enter petscii strings directly.

More advanced stuff such as expressions ("pointer+1" for example) and macro's, are way too much work for now...

 

 

Edited February 21, 2021 by desertfish