Jump to content
pzembrod

New community dev tool uploaded: VolksForth X16

Recommended Posts

VolksForth X16

View File

VolksForth Readme

VolksForth is a 16bit Forth System produced by the German Forth Gesellschaft e.V.

The main repository lives here: https://github.com/forth-ev/VolksForth

Major development of this system was done between 1985 until 1989. The VolksForth Project was revived in 2005 with the goal to produce a managable Forthsystem for computer systems with restricted system resources.

Some modern Forth Systems were influenced by or were derived from VolksForth (GNU-Forth, bigForth).

The current Version of VolksForth is 3.81. Version 3.9.x will be interim versions on the way to sync all VolksForth targets and move towards compliance with the 2012 Forth standard.

Version 3.8.x is based on the Forth 83 standard, Version 4.00 will be based on the current 2012 Standard (https://forth-standard.org).

At this time VolksForth is available for this Systems:

  • VolksForth MS-DOS (Intel x86 architecture i8086/i186/i286/i386/i486 etc)
  • VolksForth 6502 (Commodore 64, Commodore Plus 4, Commander X16, Apple 1, Apple ][, Atari XL/XE)
  • VolksForth Z80 (CP/M, Schneider CPC)
  • VolksForth 68000 (Atari ST, Amiga with EmuTOS)

 

Copyright

The VolksForth Sources are made available under the terms of the BSD Lizenz - http://www.opensource.org/licenses/bsd-license.php

The Handbook is Copyright (c) 1985 - 2020 Forth Gesellschaft e.V. ( Klaus Schleisiek, Ulrich Hoffmann, Bernd Pennemann, Georg Rehfeld, Dietrich Weineck, Carsten Strotmann).

(most of the Information is still in german. We are planning to provide future versions with englisch documentation)

Have fun with VolksForth the VolksForth Team


 

  • Like 4

Share this post


Link to post
Share on other sites

Nice work! Forth is an interesting language, but I do not know almost anything about how to use it.

I've been programming a text editor for the platform (X16 Edit). It would be nice to use the editor for forth coding.

I managed to write the smallest hello world program and saved it as a text file on the SD card image. I could import the source file to your interpreter with the include command, and it worked as expected.

Even though the interpreter seems to be running in ISO character mode, the source file I imported was interpreted as PETSCII. I don't know if that's a limitation. Comments are, for instance, marked with backslash, which is not available in PETSCII.

Please let me know if there is anything I can do to make the editor play nicely with Forth.

  • Like 1

Share this post


Link to post
Share on other sites

Actually, I still have a lot to learn about the details of the X16 programming model. So, I don't think VolksForth is running in ISO mode, at least not by default, as far as I can tell. I do print a chr$($0e) during startup to switch to lower/uppercase, but $41 emit prints a lowercase a and $61 emit an uppercase A, as expected in PETSCII mode. In CBM Forth, comments are marked with the pound sign, chr$($5c), where backslash sits in ASCII. I hope this helps.

FWIW, the X16 VolksForth is built with a target compiler running on C64 VolksForth in VICE, and the originally Linux ASCII sources are first converted to PETSCII before fed into VICE, so the X16 VolksForth should be PETSCII DNA through and through.

But great that your hello program worked as expected, and I would expect your editor to work just nicely for Forth code without any modification since it supports PETSCII. I'll try it out myself locally and let you know how it works for me.

One thing that would of course be great is if one could call your editor directly from Forth. Do you think there would be a way to place X16 Edit into a ROM bank and make it callable from other programs, e.g. Forth?

Edited by pzembrod

Share this post


Link to post
Share on other sites
3 hours ago, pzembrod said:

One thing that would of course be great is if one could call your editor directly from Forth. Do you think there would be a way to place X16 Edit into a ROM bank and make it callable from other programs, e.g. Forth?

I have another thing that would be great: if the X16 edit keeps the data of the text file in memory, it would useful, after returning back to VolksForth, to be able to get the pointers to the start and the end of the text from inside VolksForth (or any other application). That way VolksForth could parse and compile the source code from memory without the need of (slow) disk I/O.

Edited by cstrotm

Share this post


Link to post
Share on other sites

@pzembrod

OK. Let me know if there are issues. One thing: As far as I know there is no real text file format standard for the X16, especially how line breaks are to be coded. I have chosen to use Unix style line breaks, that is LF=character code 10 (decimal). That could be a problem for VolksForth if it assumes another form of line break. I would not like to use CR as line break, as that is not recognized by other OS. And don't ask me about those Windows two byte line breaks...

I'm not part of the group that decides what's going into the ROM. If they would like to include the editor, that's fine with me. The editor is released under GPL v 3, and may be distributed by anyone under the terms of that license.

Would it otherwise help to make a high memory version of the editor, that could stay in RAM at the same time as VolksForth? The program has become a bit large (at present 15KB). I could shrink it by 2 KB by moving the built-in help screen to a file. I'm in the process of cleaning up the code, and might be able to shrink to code further, but it will not be easy to make it substantially smaller than 13 KB.

Share this post


Link to post
Share on other sites
18 minutes ago, cstrotm said:

I have another thing that would be great: if the X16 edit keeps the data of the text file in memory, it would useful, after returning back to VolksForth, to be able to get the pointers to the start and the end of the text from inside VolksForth (or any other application). That way VolksForth could parse and compile the source code from memory without the need of (slow) disk I/O.

The head of the text is always bank 1, address $a000.

The memory model is not a simple stream of bytes. The memory pages are organized as doubly linked lists, and every memory page has some metadata to make this possible.

Edited by Stefan

Share this post


Link to post
Share on other sites
2 hours ago, Stefan said:

@pzembrod

OK. Let me know if there are issues. One thing: As far as I know there is no real text file format standard for the X16, especially how line breaks are to be coded. I have chosen to use Unix style line breaks, that is LF=character code 10 (decimal). That could be a problem for VolksForth if it assumes another form of line break. I would not like to use CR as line break, as that is not recognized by other OS. And don't ask me about those Windows two byte line breaks...

Ah! I stayed with the CBM standard CR for line breaks so far, given that I also stayed with PETSCII. How would you feel about using LF or CR depending on whether X16 Edit runs in PETSCII or ISO mode? Fully agree wrt Windows style CRLF, btw.

2 hours ago, Stefan said:

Would it otherwise help to make a high memory version of the editor, that could stay in RAM at the same time as VolksForth? The program has become a bit large (at present 15KB). I could shrink it by 2 KB by moving the built-in help screen to a file. I'm in the process of cleaning up the code, and might be able to shrink to code further, but it will not be easy to make it substantially smaller than 13 KB.

I figured that this would be its realistic size range, hence my ROM suggestion as a ROM bank is 16k. The question, as I see it, would be whether your code is a) romable and b) could bank-switching call the KERNAL. X16 BASIC is doing this already, so there must be a reasonably established way to do it.

Share this post


Link to post
Share on other sites

It's not a problem supporting CR line breaks. But is it a good idea to have different standards?

I haven't done much thinking on what's needed to make the editor "romable".

  • The editor would fit in a 16 K ROM bank without any further size optimization
  • The editor uses the following Kernal routines:
    • GETIN to read the keyboard
    • SETNAM, SETLFS, OPEN, CHKIN, CHKOUT, CHRIN, CHROUT, READST, CLRCHN and CLOSE for reading and writing files
    • MEMTOP to get banked RAM size
  • I think you would need a bridge in RAM for each of those Kernal functions. A bridge would need to
    • Switch to the Kernal ROM bank
    • Call the Kernal function
    • Switch back to editor ROM bank
    • And return without affecting the communication registers of the Kernal function
  • Looking at the memory map, it seems that the bridge functions could be placed in 0400-07ff.
  • The bridge functions could be stored in the editor ROM bank and moved to 0400-07ff on each startup.
  • Disclaimer: I haven't tested any of this.

Share this post


Link to post
Share on other sites

CR as end of line was also used by Apple II/III and Mac OS 9 and lower. There are a lot of conversion programs for this text file standard, so I guess it's best to not change if one want X16 to be compatible with C64 and PET. In Linux/Unix exists unix2mac, mac2unix, unix2dos, dos2unix, see man unix2dos.

Share this post


Link to post
Share on other sites

Was there ever a standard text file format for the C64? I don't really remember. But I was not doing a lot of text editing on the C64 in the 80s.

My feeling is that everyone invented their own solution on the C64.

  • BASIC uses NULL (0) as end of line marker.
  • At least some of the word processors/text editors of the time used screen codes instead of PETSCII. One such example is SpeedScript, that I've tested in VICE.
  • Speedscript furthermore used screen code 31 as end of line marker, i.e. the left arrow character.
  • The text: "HELLO WORLD!<ENTER>" is stored as hex values 00 25 08 05 0C 0C 0F 20 17 0F 12 0C 04 21 1F, where the first three bytes are not part of the text, and 1F is the line break marker.

By not using LF as line break marker, we loose compatibility with all modern OS:es. Do we really gain anything by using $OC?

Share this post


Link to post
Share on other sites

In my opinion, the "natural" text line break character of the C64 is defined by what the KERNAL does on print, and the KERNAL breaks lines on $0d.

Also, the text editors that came with the assembler ASSI/M and with Kyan Pascal use CR as line terminator.

So  think, to keep C64 compatibility, which I guess is a clear goal for the X16, there is no way around supporting CR as line terminator as an option.

I also think this fiddling with different dharsets and encodings is quite annoying, but if CBM is part of the game, that's how it is.

 

As for the KERNAL API that X16 Edit uses, since that is all in the standard list of API calls, you could look at how X16 BASIC handles those. IIUC you can call those standard APIs with the BASIC ROM switched in, and they magically end up calling the KERNAL, so BASIC must have a nifty bank bridge that I would expect you could re-use.

 

Share this post


Link to post
Share on other sites

In the nano editor for Linux you can choose to save as old Mac format (CR=$0D), but default is Unix format (LF=$0A).

nano-save-mac-cr.png

Share this post


Link to post
Share on other sites

OK. I will look into supporting CR line breaks and maybe even CRLF line breaks as alternatives. The user interface could work about the same way as Nano, i.e. you select format after issuing the save command.

As to being romable:

  • There's the JSRFAR Kernal function, but I guess that's not available if your code resides in another ROM bank.
  • The JSRFAR is general and there seems to be some overhead for each call that you could avoid by using a specialized bridge, at least for Kernal calls made a lot of times in a loop.
  • I've looked at the BASIC source, but as I'm not familiar with it, it's very hard to exactly understand what's going on. And the BASIC ROM code is also not available if your code resides in another ROM bank.
  • A bridge function in RAM need not be more complicated than this, I think, using a call to CHRIN as an example:
    • bridge_chrin: stz $9f60   ;ROM bank 0
    • jsr CHRIN
    • ldx #6   ;Return to editor ROM bank, here assumed to be no 6
    • stx $9f60
    • rts

Share this post


Link to post
Share on other sites

Is it possible to write stand alone Forth programs using this VolksForth? I.e. they should be possible to autostart using the emulator without having to type anything in Forth. Preferably it should be one file with VolksForth baked in, but if that is not possible some program that loads VolksForth and the Forth program automatically.

Share this post


Link to post
Share on other sites
On 11/17/2020 at 6:38 PM, mobluse said:

Is it possible to write stand alone Forth programs using this VolksForth? I.e. they should be possible to autostart using the emulator without having to type anything in Forth. Preferably it should be one file with VolksForth baked in, but if that is not possible some program that loads VolksForth and the Forth program automatically.

Yes, that is possible. The usual way to compile an application with VolksForth is to compile the application code on top of the base Forth system, and then use SAVESYSTEM to store the compiled code together with the underlying Forth system into one file.
There are a few hooks aka deferred words, pre-populated with noop, e.g. 'cold, that are called during system startup. They can be used to start an application on Forth startup.

As for how BASIC in ROM bank 4 jumps into the KERNAL in ROM bank 0: This is implemented in https://github.com/commanderx16/x16-rom/tree/master/kernsup. I haven't fully grokked (yet) how the code works in detail, but it seems pretty clear that the essence is that the kernsup_*.s files assemble a jump list at the end of the BASIC or the MONITOR rom that mirrors the KERNAL jump list and implements it via cross-bank long jumps. I could imagine that replicating this mechanism at the end of an X16 Edit ROM could work. I wasn't suggesting to use the BASIC ROM; I may have caused a misunderstanding there.

  • Like 1

Share this post


Link to post
Share on other sites

@pzembrod and @mobluse

Inspired by you, I have made a romable version of X16 Edit. It's published in the downloads section. Sources on Github. It's a bit fiddly to set up for testing, but a proof of concept that the editor ROM version works as expected.

I had three main issues to solve

  • Bridging Kernal calls. I seems that I could have used the built in JSRFAR, but in the end it was very easy to implement my own specialized bridge. This was the easy part.
  • There was quite a few variables embedded in the code section that I had to move to RAM. This was not hard, but took some time to do.
  • The hardest part was to get interrupts working. Very frustrating before I found a solution. I looked mostly on how the built-in monitor had solved this, but it's always hard to get to know code you are not familiar with.

I have also changed the way line breaks are encoded.

  • If the editor is in PETSCII mode, it uses CR as line break
  • If in ISO mode it uses LF as line break
  • Internally it still uses LF as line break in all cases. When reading a file into the buffer it converts all CRs to LFs. If in PETSCII mode, all LFs are converted back to CRs when you save the file.

I guess you're right that this is the most likely setup to be play nicely with other programs from the C64 era. After all Commodore 8 bit computers didn't have the LF control character.

If you would like to save an ISO file with CR line breaks or a PETSCII file with LF line breaks, it's still possible. Edit the file in the preferred mode. If you want LF line breaks, change to ISO mode before saving or to PETSCII mode before saving if you want CR line breaks.

Edited by Stefan
  • Like 2

Share this post


Link to post
Share on other sites

You need to start the ROM version of the program in machine code.

In Basic you may load the code from a DATA statement. I attach an example.

I don't know how to do the same in Forth.

I'm very interested in knowing how well the editor works with your Forth compiler. Let me know if there are issues.

STARTER.BAS

EDIT: Isn't there an inline assembler in Forth? If so you could enter the following assembly code to start the editor:

LDA $9F60
PHA
LDA #$07
STA $9F60
JSR $C000
PLA
STA $9F60
RTS

 

Edited by Stefan

Share this post


Link to post
Share on other sites

I'm trying to setup V4TH to start learning the language.

Is it true that the download published on this site is just a base system, and that you need additional files to actually do something interesting? For instance inline assembly.

Could you tell a total FORTH beginner what files you need to put on the sdcard besides v4th-x16.prg.

Share this post


Link to post
Share on other sites

Hi Stefan,

in case you want to do inline assembly, you would indeed want to add 6502asm.fth and possibly trns6502asm.fth from https://github.com/forth-ev/VolksForth/tree/master/6502/C64/src to the sdcard, but to start and learn the language I would actually advise to leave out assembly at first. Of course, to call your editor, you need assembler, that's true. I may also take a stab at that integration over the Christmas break. I'm currently working on porting my C compiler cc64 (written in VolksForth) to the x16 - there your editor would of course also come in extremely handy.

But basically I would say you don't need any other files to do something interesting. You may want to add tracer.fth from the same location above, if you want to do single-step debugging. But that should be it. cc64 is really based on just core v4th and the 6502asm.fth files from the VolksForth sources.

One thing we should figure out to make the two tools easy to integrate with each other is zero page usage: Maybe we could make it so that we keep away from under each others' feet, so to speak, that we use different zero page areas. What is X16Edit using? VolksForth currently starts to use ZP from $30, and I would have to look closer how long exactly the used zero page segment is, but it seems to be ~50 bytes; among others the inner interpreter lives there, much like charget in C64 BASIC.

Cheers

/Philip

  • Like 2

Share this post


Link to post
Share on other sites

Hi,

Thanks, I'll try including those files. Learning Forth is a struggle, and I don't feel like the smartest of people when reading tutorials about it...

Good idea to coordinate memory usage.

At this stage, the program uses 57 bytes of zero page ($22-$5B). Of these, 10 bytes must be in zero page, as they are used for indirect addressing.

The program uses $0400-04DA for variables and temorary storage.

$0500-05FF is the buffer used when prompting the user for input to commands.

$0600-06FF is the current file name buffer.

$0700-077F is a buffer holding the result of last disk status read.

$0780-07FF is bridge code needed to handle ROM banking.

All of banks 1-255 in the banked RAM are also used. Bank 1 for the program's memory map (1 kB) and clipboard buffer (3 kB) and banks 2-255 for the actual text buffer.

I could easily use less zero page addresses.

I could probably also use a lot less space in the $0400 page. There are a lot of temporary backups of values that could be moved to the stack instead.

And it is of coarse possible to backup the zero page on starting X16 Edit, and to restore it on exit. I have got 4 kB of unused space in banked RAM, bank 1, that could be used for this purpose.

  • Like 1

Share this post


Link to post
Share on other sites

Hi Stefan,

Great stuff! 🙂

Regarding learning Forth: Yes, that is quite a step. It doesn't actually really have to be hard, but it's very different in a number of respects from many other programming languages, so quite some concepts to wrap your mind around, esp. the distinction of compile time and run time which, as opposed to most other languages, in Forth you have explicit control of - and want to use, at times.

There are 2 books by Leo Brodie I would recommend, btw. The generally agreed best book for learning Forth is "Starting Forth" which is e.g. available online at Forth, Inc. The other book is even more remarkable, imho, and it is "Thinking Forth" which even has its own web site now, and it contains a lot of really insightful design advice and principles when using Forth; at least some of it should be relevant beyond Forth, too; I at least consider it one of the top influential books on my own programming and design style.

 

Really good that you have leeway wrt zero page usage. But I also just saw that I could likely let VolksForth use the area $00A9-$00FF; in the Commander X16 Programmer's Reference Guide it says "In a machine language application that only uses KERNAL (no BASIC or floating point), the following zero page locations are also available:" - that should apply easily to VolksForth; when exiting it I'm invoking a BASIC cold start so overwriting BASIC's zero page addresses should not be a problem.

Anyway, clearly something we can get figured out and synced on going forward.

As for the $0400-$07FF area - VolksForth doesn't use any of that (methinks, at least); it is based on and follows the C64 VolksForth very much. 🙂

VolksForth also doesn't use any banked RAM right now; when a time comes where that might be desirable, e.g. to back the Forth virtual memory manager for reading disk blocks (which the x16 implementation doesn't do at all yet), I'm sure we could also arrange some (possibly configurable) split of the RAM banks between the different usages.

Cheers

/Philip

 

  • Like 1

Share this post


Link to post
Share on other sites

I've done some initial work reducing X16 Edit memory usage.

It's now only using ZP addresses $22-$37 = 22 bytes.

This was done by moving variables from ZP to $0400 page.

I have also removed some variables that were stored in $0400 page (mainly temp data backups). These values are now temporarily stored on the stack instead.

The range $0400-$04c6=199 bytes is used for variables.

As before the editor uses all of $0500-$07ff.

As to the banked RAM:

  • I need one bank for house keeping and buffers. This is bank 1 at the moment
  • I would like to have a fixed head of the text buffer not to overcomplicate the program. This is bank 2/$A000 for now.
  • In bank 1 I already have a memory map or BAM, if you like. It would not be hard to make a function that lets you reserve parts of banked RAM that is not touched by the editor. 
Edited by Stefan

Share this post


Link to post
Share on other sites

@pzembrod I've now given it some thought how to best let other programs limit what parts of banked RAM is used by the editor.

I have got a test solution that seems to work: Before starting the ROM version of the editor you specify the first bank used by the editor in the X register and the last bank in the Y register. Then you call the editor entry point in $c000.

Another thing: I have not yet succeeded in including 6502asm.fth. I get compiler errors that seem to be linked to the character encoding. I first made a Python script to convert ascii to petscii. I then saw that you in the Makefile called a utility for this conversion. I also found the source code of that utility, but I still didn't get it working properly. I need to continue another day... But I really like the idea of Forth from a conceptual standpoint. And I'm still using my old HP12c calculator at work, so I'm kind of used to reversed Polish notation 🙂

Edited by Stefan

Share this post


Link to post
Share on other sites

Hi Stefan,

congratulations, awesome progress on your side. Just a quick update from here as I recently found some time again for my X16 coding.
I feel I am in the late stages of porting my cc64 Small C compiler to the X16. It is building now (I figured out a way around the smaller contiguous RAM compared to C64 and C16), I am currently cleaning up the code, and then I'll have to adapt the runtime library, and see if the tests pass. The next step to make it usable would then be adding a call stub to X16 Edit. The same call stub would obviously also work for plain VolksForth, as cc64 is written in VolksForth.

On 12/10/2020 at 9:54 PM, Stefan said:

@pzembrod I've now given it some thought how to best let other programs limit what parts of banked RAM is used by the editor.

I have got a test solution that seems to work: Before starting the ROM version of the editor you specify the first bank used by the editor in the X register and the last bank in the Y register. Then you call the editor entry point in $c000.

This is pretty great! I am currently using one RAM bank for VolksForth; with your X/Y solution I can adapt X16 Edit easily; I love it.

On 12/10/2020 at 9:54 PM, Stefan said:

Another thing: I have not yet succeeded in including 6502asm.fth. I get compiler errors that seem to be linked to the character encoding. I first made a Python script to convert ascii to petscii. I then saw that you in the Makefile called a utility for this conversion. I also found the source code of that utility, but I still didn't get it working properly. I need to continue another day... But I really like the idea of Forth from a conceptual standpoint. And I'm still using my old HP12c calculator at work, so I'm kind of used to reversed Polish notation 🙂

Happy to help there if things haven't cleared up already; we can sync one of these days if you like.

  • Like 1

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...

Important Information

Please review our Terms of Use