Standard interface for extending Basic

[mgkaiser]

How about introducing a standard interface for extending BASIC? My thought is that it would work something like this. We add a single new BASIC statement, "@", that tokenizes and takes a parameter list. The first parameter is an unquoted string, which is follow by a variable number of parameters. All the command does is parse the parameters and then call a vector. By default the vector will point to a "do nothing" function. When people write extensions, they will hook the vector. It will be the standard to remember what the vector was previously set to and call that at the end of your call if you did not process the command. That would allow multiple extension commands to be hooked to the interface written by different people.

Example
@MYCOMMAND1,2,3
@ANOTHERCOMMAND,"SOME TEXT"

That would call the vector and pass 4 parameters to it. The vector handler would look at param1 and go "I handle MYCOMMAND" and handle it. If it didn't handle "ANOTHERCOMMAND" then it would just pass it along to the next vector handler.

I know we have the "USR" function, but that only allows a single function with a single parameter. What I am proposing allows as many commands as desired, allows multiple commands from multiple authors, and does not break tokenization. The only down size is a custom command will be bigger than tokenized commands.

Double bonus is that these will work both in code and in immediate mode with no extra effort.

EXTRA POINTS if we add 2 KERNAL routines, one to hook and one to unhook commands so the commands don't even need to know how to hook and unhook themselves and can be loaded and unloaded in any order.

[skaratso]

What you're proposing here sounds very similar to the "&" (ampersand) command in Applesoft BASIC on Apple II computers.

What & did was an immediate jump to $3F5, called the ampersand vector. The difference is that people writing extended commands would do their own command parsing using Applesoft BASIC's own command parser. This would, of course, require that certain entry points within BASIC would need to be preserved, but it would be much simpler to implement.

[TomXP411]

My thinking was something slightly more sophisticated, but not by much...

basically, I was thinking about creating a custom escape token that would be "execute user defined token", which would then point to a user created token table somewhere else in memory.

The user token table would basically be two lists:
List 1: Address Table
The address of each user routine, starting at token $80. Each entry is: 1 byte bank, 2 byte address.
List 2: Names Table
A list of token names. Last character of name is $80+character (same as CBM table.)

[skaratso]

I'm not super familiar with Commodore 64 or Commander X16 BASICs, as I grew up using Apple II machines, but I just received notice that my X16 will be shipping very soon. But, given the similarities between Applesoft and Commodore BASIC, is the CHRGET routine still at $B1 on the zero page?

In Applesoft, when the ampersand is encountered and it jumps to $3F5, all you have to do is put a JMP <addr> to your parser. The character or token immediately after the ampersand is pointed to by TXTPTR and in the accumulator, so you can do a CMP <whatever> and then branch accordingly. If the high bit is clear, it's interpreted as an ASCII character, but if the high bit is set it's interpreted as whatever command token corresponds to that value. So CMP #$41 would be the letter A, but CMP #$BA would be the PRINT token. So you could have &A or even &PRINT, for example. Some ampersand parsers would go further and do full on string comparisons, like &"FOOBAR" but those are more complex. Are some of the extended commands in X16 BASIC implemented with 16 bit tokens? That would make things a little more difficult.

What you propose, though, could work a little better in terms of implementing a list of additional commands after the "escape code" or character, but it seems that the parsing after the command would still need to be done with internal BASIC routines, no?

[TomXP411]

What you propose, though, could work a little better in terms of implementing a list of additional commands after the "escape code" or character, but it seems that the parsing after the command would still need to be done with internal BASIC routines, no?

In this case, the BASIC interpreter would know to look at the user-loaded token tables and parse and tokenize user tokens, as well as ROM based tokens. Obviously, this requires extending BASIC to make it work, but it would allow for any number of additions, especially if you just need a few commands to optimize an existing program.

[skaratso]

Ok, so what you're proposing is more or less a "standard" way for "escape character" commands to be passed to the code that's interpreting them. If we say, follow Apple's convention and use the & as the escape character, then when the interpreter sees "& FOO" it'll look it up in the user token table and pass that to the appropriate routine.

The next question is, how will the user routine parse any arguments? Would the BASIC interpreter do it automatically and pass a list of pointers to each argument to the user routine (which could either limit the arguments to a certain format or create more complexity for you guys to allow maximum flexibility), or would the user routine be responsible for parsing, as it is done in Applesoft?

In Applesoft, ampersand commands just use internal Applesoft routines to parse all the arguments passed. None of them were "official" entry points, though, so if Apple had made substantial changes to BASIC any user routines would have needed to be updated. If all the X16 project is guaranteeing is an API (meaning that internal entry points might change from ROM version to ROM version), then if you go the route of letting user routines parse their own arguments with BASIC's own parser then you might want to create a jump table or some other standard way of finding those routines regardless of where they might move.

If you would like to see some sample code for Applesoft ampersand commands as an illustration of how it's done I have some I can send you. I also have a list of the common internal entry points used as well. With Michael Steil's C64 BASIC & KERNAL ROM Disassembly (https://www.pagetable.com/c64ref/c64disasm/) with multiple sets of comments from several sources, we should be able to map the Applesoft entry points to the appropriate ones in X16 BASIC.

[TomXP411]

Ok, so what you're proposing is more or less a "standard" way for "escape character" commands to be passed to the code that's interpreting them. If we say, follow Apple's convention and use the & as the escape character, then when the interpreter sees "& FOO" it'll look it up in the user token table and pass that to the appropriate routine.

To be clear, I'm NOT proposing a printable escape character, like some BASIC extensions use. It's just a 2 byte token, with the first byte being one designated for user tokens. So if you wrote a command named FOO, it would just be written as FOO in the listing, but internally, BASIC would see it as something like $FF $85.

This method makes FOO a first-class command, just like PRINT or POKE.

The next question is, how will the user routine parse any arguments? Would the BASIC interpreter do it automatically and pass a list of pointers to each argument to the user routine (which could either limit the arguments to a certain format or create more complexity for you guys to allow maximum flexibility), or would the user routine be responsible for parsing, as it is done in Applesoft?

You'd just do the same thing BASIC does. There are functions to read the next parameter in the line buffer, based on what you're expecting. So you can read an integer with GETNUM, a byte value with GETBYT, an address with GETADR, and so on.

So basically, each command is responsible for its own parsing, through the use of the helper functions.

[skaratso]

Ok, so the command parsing itself is done the same way, essentially, with the same routines that BASIC uses. But user commands get their own entries as two byte tokens but are essentially first-class commands.

That does raise the question of how the end user will distinguish between "native" commands and "user" commands when looking at a listing.

I presume that if the commands aren't loaded and such a program is listed, the user commands will be replaced with some kind of "placeholder" text? Would it be possible to somehow highlight the user commands in a listing so that they can easily be distinguished? Making user commands first-class citizens is a much nicer idea than using an "escape" character. But the one advantage the & provides in Applesoft is that it is very easy to tell the difference between "native" and user commands.

I look forward to seeing this. I've written a handful of ampersand commands for Applesoft BASIC in the past and I'd love to try my hand at porting them over to the X16.

[TomXP411]

That does raise the question of how the end user will distinguish between "native" commands and "user" commands when looking at a listing.

I can't think of a reason that's necessary. However, if someone wanted to write their own LISTer, they could do that.

The only obvious issue I can see is that if the BASIC extender is not present, then tokenized programs with extended tokens won't list or run properly. Right now, if you have an invalid character in a BASIC line, you'll either get a Syntax Error, or you'll get a repeat of a different token.

There's not really a good way to solve that with tokenized BASIC, and running non-tokenized code slows things down a lot... which is why BASIC programs are tokenized to begin with (well, aside from saving 2-4 bytes per instruction.)

Anyway, it's just an idea. You'd have to get someone to actually implement it to make it useful, and I don't think anyone on the current dev team has the bandwidth for that. I'm playing around with a few ideas myself, but I'm also a long ways away from being able to write BASIC extenders, yet.