Native assembler, preferably open source

[Stefan]

Hello,

Is there an assembler for 6502 that could be made to run natively on the X16?

Preferably it would:

• Be open source
• Read source code from files
• Assemble directly to files
• Support at least named labels, basic compiler directives (for instance .byte, .word, .text) and macros, simliar to the Commodore Macro Assembler that you could use on the C64

I've been searching, but it seems hard to find.

I used the Commodore Macro Assembler a lot with my C64 in the 80s. Even though I had a disk drive, it was cumbersome and took quite some time to edit, save, assemble, load and test each change in a program. If the real X16 reads and writes SD card files at the same speed as the emulator, we would be getting much closer to a modern experience with a file based assembler.

[Ender]

Well, there is the "super monitor" X16 Assembly Environment that @mjallison42 is working on. It's still in the works, but it's largely complete already: https://sites.google.com/view/x16asmenv/home

There's also an assembly language editor being worked on by @Ed Minchau that looks cool. He's mentioned on Facebook that it should be finished soon.

 

Edited December 27, 2020 by Ender

[TomXP411]

Obviously, there is not a conventional native assembler for the Commander X16, yet. 

@mjallison42's monitor/assembler is very useful an dfun to use, but it has a shortcoming of not being able to read text files as input. So you can't write your assembly code in a text editor and assemble it using his program. I'm going to pester him to see if there's some way to import/export his label files for debugging, at least. 

Here's a page of 6502 software; there's at least one assembler on there that can be modified to run on the Commander:

http://www.6502.org/source/

 

[Stefan]

Thanks.

The assembler made in 1979 by Robert Ford Denison looks really interesting judging from its documentation. It's rare to see such documentation - I'm impressed to say the least. However, it is not free software. To use the assembler you would need to get hold of Mr. Denison and get his approval. That might be possible, but where do we find him?

I'm not really a fan of solutions where you bundle the assembler with a development environment of sorts. These solutions tend to become tightly bound to the computer they were written for. An assembler using text files as input and outputting object or program files could on the other hand with reasonable efforts be run on any 6502 based computer. Also I am a strong believer in the Unix approach, to make programs that do one thing, and do it well.

While longing for such an assembler, you may read this nice online book by David Salomon on how to make one:

http://www.davidsalomon.name/assem.advertis/asl.pdf

Edited December 28, 2020 by Stefan

[desertfish]

 I’ve watched many episodes of 8 bit show and tell and Robin mostly uses the Turbo Macro Pro assembler. It provides an integrated editor and assembler. The edit assemble run cycle is really smooth. This is mostly because it can use a Ram expansion/banking to store stuff in and the ability to immediately restart the editor after testing the program. But, I think it allows for assembly to disk as well. 

Is something like that what you’re after? 

For the Unix like experience I think I just want to stick to using a cross assembler on pc 

i wonder if TMPro can be ported 

[TomXP411]

52 minutes ago, desertfish said:

 I’ve watched many episodes of 8 bit show and tell and Robin mostly uses the Turbo Macro Pro assembler. It provides an integrated editor and assembler. The edit assemble run cycle is really smooth. This is mostly because it can use a Ram expansion/banking to store stuff in and the ability to immediately restart the editor after testing the program. But, I think it allows for assembly to disk as well. 

Is something like that what you’re after? 

For the Unix like experience I think I just want to stick to using a cross assembler on pc 

i wonder if TMPro can be ported 

It's certainly possible to port the code, but the program itself is not open source. It started out as a commercial product and was subsequently pirated and modded by users over the years. I don't think I'd use it as a starting point for a new assembler, especially when there are simpler and more straightforward assemblers out there. 

 

Edited December 28, 2020 by TomXP411

[aex01127]

On 12/27/2020 at 4:16 PM, Stefan said:

Hello,

Is there an assembler for 6502 that could be made to run natively on the X16?

Preferably it would:

• Be open source
• Read source code from files
• Assemble directly to files
• Support at least named labels, basic compiler directives (for instance .byte, .word, .text) and macros, simliar to the Commodore Macro Assembler that you could use on the C64

I've been searching, but it seems hard to find.

I used the Commodore Macro Assembler a lot with my C64 in the 80s. Even though I had a disk drive, it was cumbersome and took quite some time to edit, save, assemble, load and test each change in a program. If the real X16 reads and writes SD card files at the same speed as the emulator, we would be getting much closer to a modern experience with a file based assembler.

I would really recommend looking into CC65. I think it has everything you ask for. If you work on small projects you can have everything in one file. And if you want to work on larger projects you will also see that it scales. Linking together multiple files, combining assembly / C language. The options are endless. I also see that CC65 is used for building the X16 kernal / Basic. cc65 is also available on most modern operating systems using packet managers.

To compile a simple program I would use something like this: cl65 --cpu 65C02 mylife.s -t cx16 -C cx16-asm.cfg --start-addr 16384

Load your program and SYS16384 to get it started. Attached is Hello World using Vera.

Edit: I misread that you wanted it to run on the X16. So my answer does not apply.

hellow.s

Edited December 30, 2020 by aex01127
misread platform requirement

[SlithyMatt]

3 hours ago, aex01127 said:

So my answer does not apply.

Not completely, but the ca65 code may be a good starting point. You could actually use cc65 to build a hacked down version of ca65 for the X16. It's all open source, so why not? You'll at least get all the syntax parsing stuff already implemented.

[Stefan]

Yes, I've used CA65 quite a lot. It's a great assembler.

I realize that it will be hard for a native assembler to match all features of CA65. But it would still be possible to make a very good assembler that is easy to run on many different 6502 systems.

In order for the X16 to become something else than a game console with a complicated way of starting your games, native programming is probably important.

I suspect that there is no easily portable open source assembler that you could build upon. Hopefully I'm wrong.

If my suspicion is true, it's a bit sad considering the wast amount of time and effort poured into developing various assembly programming environments for different 6502 based systems since the processor was born.

If you would go about making yet another assembler, it should be made portable. Using raw text files as input is probably be the best option.

Bundling assemblers/compilers makes them harder to port. Also, making a good source code editor is a huge task in itself, and takes a lot of time and energy that could be saved for making the best possible assembler/compiler.

So if you would go about making that assembler, please let it read source code from text files and output object files controlled by a minimal text based interface.

Edited December 30, 2020 by Stefan

[desertfish]

Something like the Merlin assembler then, Stefan? 

 

[Stefan]

According to Wikipedia, the heir of the author of that program released all his "Apple software to public domain" in August of 2000.

https://en.wikipedia.org/wiki/Merlin_(assembler)

However, the Merlin assembler was also bundled to an editor, initially a line editor, which very few of us would stand to use. I don't know how much that complicates porting the program.

Anyway, I think it would be a good idea to look at the source code to get a feeling for how hard it would be.

Edited December 30, 2020 by Stefan

[Stefan]

Some initial thoughts after reading manuals for the Merlin assembler:

• If I understand, it only assembles source code residing in memory, not directly from files, not what I would prefer.
• I read somewhere that it uses a custom binary format for the source code, not plain text. May or may not be true.
• And it is very particular when it comes to source code formatting.
• My guess is that porting the Merlin assembler, requires porting the whole system including the editor (probably not a good option) or splitting out just the assembly function and doing a lot of rework to that.

[desertfish]

I prefer plain text files as well but mainly because I'm still focused on manipulating them on other systems outside of the Cx16 itself.  If we're strictly limiting ourselves to working on the cx16, the benefits of working with plain text files is less clear to me.  

There are some problems I think with text files as source.

- they take up much more memory and disk space than a binary format, so a source listing is quickly larger than the available memory. There is enough banked RAM to load it but dealing with data split up over different banks may complicate processing it as a single file a lot?
- text is inherently more complicated and slower to process than a dedicated binary format.   Although maybe the 8 mhz can just brute force negate this?
- editing can be problematic (naive inserting and deleting requires moving a lot of data around in memory).  Although you seem to have solved this admirably in your text editor already !

 

[lamb-duh]

On 12/30/2020 at 12:01 PM, SlithyMatt said:

Not completely, but the ca65 code may be a good starting point. You could actually use cc65 to build a hacked down version of ca65 for the X16. It's all open source, so why not? You'll at least get all the syntax parsing stuff already implemented.

now I'm wondering.. has anyone tried to make cc65 self-hosting? I wonder what's holding it back

[Stefan]

12 hours ago, lamb-duh said:

now I'm wondering.. has anyone tried to make cc65 self-hosting? I wonder what's holding it back

Interesting idea. Maybe you actually could compile the CA65 assembler with the CC65 compiler.

When I compiled the CC65 suite for my desktop computer, CA65 became 211 kB.

Obviously you need to strip out some functionality, as @SlithyMatt mentioned above, to make it fit.

[lamb-duh]

26 minutes ago, Stefan said:

When I compiled the CC65 suite for my desktop computer, CA65 became 211 kB.

Did not realize it was that big! Maybe it wouldn't be such a useful starting place

[Stefan]

Hi @desertfish 

The great benefit of using standards (such as ASCII encoded plain text files) is that it makes it possible for programs to work together.

And that makes it possible to easily divide programming tasks between different programmers or teams.

In the long run, we will be using our combined efforts more effectively, hopefully ending up with better applications.

Edited January 1, 2021 by Stefan

[Stefan]

3 minutes ago, lamb-duh said:

Did not realize it was that big! Maybe it wouldn't be such a useful starting place

I still think it is an interesting idea.

[desertfish]

I find it an interesting topic and so I am right now toying around with the first steps of such an assembler ....   

let's see how far we can take this

 

 

[Stefan]

3 hours ago, desertfish said:

I find it an interesting topic and so I am right now toying around with the first steps of such an assembler ....   

let's see how far we can take this

 

 

That would be a very nice project.

A stumbled upon David Salomons book about build an assembler (mentioned above in this thread). It's very hands on. And freely available online:

http://www.davidsalomon.name/assem.advertis/asl.pdf

I found the discussion on how to layout the symbol table and how to support different scopes especially interesting.

[desertfish]

First I've got to get the instruction argument parsing has to work on more than just the trivial cases shown above i.e. first goal is to correctly reassemble a plain disassembly.
Dynamic data structures are a pain on the 6502. Maybe it's possible to start with a limited fixed size symbol table to keep it simple initially.
Thanks for that document.
 

[pzembrod]

I've been wishing for quite some time to learn the current whereabouts of Dirk Zabel, the author of ASSI/M, which I considered, back then, to be the flagship macro assembler on the C64; would be lovely if he could be convinced to open-source it. I hope he's still alive.
ASSI/M assembles from file to file, and it's quite powerful.

[Ed Minchau]

Now available on Github, version 1.2 of my assembly language editor.  This is the native assembler you are looking for.  Write your programs directly on the x16, save them, write programs in banked RAM too.

https://github.com/edrobotguy/cx16/blob/master/METAL1_2_38_0664.zip

This editor is a little different than most - it's an interpreter, in the same way that BASIC is an interpreter, rather than a compiler.  That means there's no source code stored anywhere in RAM.  The only source code is in VERA as it is being displayed.  Your code is interpreted as you type it and stored in machine language and metadata, which is things like labels and comments and so on.  The metadata is stored separately from your program.  So when you're executing your code, it's the actual code.  There's no stepping or breakpoints.

This also means that you can load in already-compiled programs and read them - and add your own labels and comments to make it more understandable.  I'm thinking of doing precisely that with David Murray's PETDRAW, converting it for the X16.

There is also a non-standard program notation: every command has a unique four-character code.  The first three are the same as the standard notation, and the fourth character indicates addressing mode.  It might irk veteran 6502 programmers, but it doesn't take long to get the hang of it.

I put every single "wouldn't it be nice if" that I could think of into this program.  Have fun.

Edited to add: Here's the video playlist; only two so far but more to come.

 

Edited January 9, 2021 by Ed Minchau

[desertfish]

Worked a bit more on my attempt at a file-based assembler. Basic assembler functionality more or less done, it can more or less assemble a raw disassembly now.  No labels, symbols or other things yet

I also ran it on a 8200 lines long disassembly dump of the Commander X16 ROM region of $c000-$ffff (16 KB) and it took about 12 seconds to assemble that (into memory).  Half of that time is spent just loading the source code from disk...  and we're not yet writing the result to a new file so that would probably add about six seconds more...  adding labels and symbols functionality will require re-reading the source file at least one extra pass so that adds six seconds once again.
A disk-to-disk assembler will be pretty slow compared to an in-memory assembler...

For now it's still an example that's part of the Prog8 compiler repository so the source is here https://github.com/irmen/prog8/blob/master/examples/cx16/assembler/assem.p8

 

Edited January 12, 2021 by desertfish

[Stefan]

Really impressing work.

Maybe you can improve the speed of the file read function. I tried to load a 433 kB file into X16 Edit. That took 26 seconds, i.e. 16 kB/s.

Another thing. At the heart of the assembler I/O could be a read_line function. There's nothing wrong letting the assembler using different sources where the line is read from (file, memory).

But again. Impressing work in such a short time.

[desertfish]

Thanks Stephan. It seems the effort that I've put into Prog8 is now starting to pay off...

The assembler logic is indeed already independent from where the main loop gets the lines from (it can also run in interactive mode like by line that you enter manually). Are you suggesting to add a third source, a source file completely read in memory?  I avoided that for now because I can't deal with banked memory yet and the source files will not all fit into main memory 

I am not sure why my file I/O seems to be a lot slower than your x16 edit ....  a dummy loop to just read the file (and not process anything), basically calling CHRIN() in a loop and storing the bytes in memory, takes 6 seconds for a 16 kb file ....   I may need some help to debug that to see what could be the cause.  I'm thinking: could it be an emulator thing? that my emulator is just reading things a lot slower than yours?   I'll try reading the large file in x16 edit tonight myself to compare the load speed.     Is x16 edit reading the full fine into memory when you open it?