CommandTracker: Mostly ideas for a music tracker sparse file format

[m00dawg 107]

Given VERAsound will now replace the SAA1099, and given some other conversations on the forums (such as the conversation about envelopes in this post), I opted to take another look at my proposed tracker file format, which you can find here:

https://gitlab.com/m00dawg/commander-x16-programs/-/blob/master/command_tracker/index.md

I've talked about this before on the FB group, but I've opted to stop using FB and thought it would be a better conversation had here anyway.

The main issue is that there are 26 channels so to optimize for storage on playback, a sparse format is probably worth the extra complexity in playback routines. I had the idea of supporting multiple effects since this is a common feature in modern trackers (even FamiTracker has this). That may go by the wayside, but even if each channel only had single volume and effect columns, I came up with 5 bytes per channel for VERASound and FM and 3 bytes for DPCM. So a single row would be 126 bytes and a 64 row pattern would be 8064 bytes! The file format I came up with can support multiple effects per channel, and when you include these it balloons to a staggering 41.6k if I did my math right.

But given even complicated patterns have empty space in them, by using a sparse format these requirements go down considerably. Only rows which have data are defined and, within that, only channels which have data are defined. So a row can be as few as 3 bytes if there's only one channel playing on that row with no effects.

I made an attempt to show the proposal row format here but I can't find a way to insert code blocks and it was really hard to read without a monospace font. So I recommend looking at the link (specifically the sparse patterns link)

The trade-off is a playback routine has to track a lot more things as opposed to just reading a pattern row by row. I don't think it would be too terrible since there could be a row counter and if the current row read is greater than the counter, nothing is done. When the counter equals the current row, it then can parse out the row to do the things. This ignores envelopes and some automatic commands (like how the volume slide works in Famitracker as compared to how it works in Impulse Tracker) as those could be firing while there is no row data.

Figuring out how to efficiently edit pattern data is another task entirely though. If one adds data to a previously empty row, the sparse file would have to be reorganized. The best solution here is having a non-sparse pattern buffer - which would be fine for a dedicated tracker where we have room to move about. But given the space requirements, it means patterns would span multiple pages and that could get interesting when adding in things like envelope tables and things.

I should say I'm not an awesome assembly programmer - just a musician who is very excited about the prospects of a tracker, but given the vast sound capabilities of the X16, it feels like it will take some thought to do well given the "limited" space (which is itself far more than the 8-bit systems of yesteryear). That's why I thought it might be good to try and start the conversation by coming up with at least something that can serve as talking points.

 

[Matej 44]

Nice. Will do gui, icons etc.

[TomXP411 308]

4 hours ago, m00dawg said:

I made an attempt to show the proposal row format here but I can't find a way to insert code blocks and it was really hard to read without a monospace font.

The CODE tag here is broken. You can use the BBCODE tag, but blank lines cause weird indentation. 

	line 1
line 2
line 3

I've taken to just attaching code as text files. 

 

Edited January 16 by TomXP411

[m00dawg 107]

Ah thanks for the info! Yeah valid workaround, although the Markdown files are the easiest to see I think. Hopefully interested folk are compelled to click on those since there's support for rendering tables and things. Although Markdown was meant to look "good enough" when viewing as text, so I suppose attaching the raw markdown would work too!

[TomXP411 308]

1 hour ago, m00dawg said:

Ah thanks for the info! Yeah valid workaround, although the Markdown files are the easiest to see I think. Hopefully interested folk are compelled to click on those since there's support for rendering tables and things. Although Markdown was meant to look "good enough" when viewing as text, so I suppose attaching the raw markdown would work too!

Except I don't know of any commonly-used PC programs that render markdown properly. If you're going to go to those lengths, you might post it on GitHub or something and link to it.

[m00dawg 107]

36 minutes ago, TomXP411 said:

Except I don't know of any commonly-used PC programs that render markdown properly. If you're going to go to those lengths, you might post it on GitHub or something and link to it.

You mean like the link in my initital post? It's Gitlab, but basically the same thing:

https://gitlab.com/m00dawg/commander-x16-programs/-/blob/master/command_tracker/index.md

I actually found out today, kinda by accident, that Atom has a Markdown preview and it seems to work rather well!

[kliepatsch 71]

Hi @m00dawg and others,

just as a heads up, I am preparing to publish an early version of my PSG sound engine in the next week or two. I know we are maybe going for slightly different routes because you are really into the tracker format, whereas I am more into the modern style DAW. So there may be a couple of things that probably won't fit into your concept, but if you take my code it may be more a matter of rearranging stuff rather than rewriting everything from scratch. I have made things like multitimbrality, envelopes, volume rescaling and pitch slides (for portamento in my case).

I know it may be more fun to make all this by yourself (it's why I did it in the first place) but if you want some starting point you can take what I made. I will be willing to explain my code and discuss how things would need to be rearranged to suit your needs.

As a clarification: I am currently talking only about the sound engine, NOT the playback mechanism. You can provide the playback mechanism. There an API for using the sound engine. Triggering and stopping notes. It still needs to be extended to support things like volume slides or pitch slides which I would definitely want to be possible with it.

[m00dawg 107]

Actually on the contrary, while it may be fun to do all this myself, that's a pretty big lift! And the ultimate goal for me is having a tracker on the X16 so I can make music with it. Once there is a MIDI interface, perhaps in tandem with a modern DAW; but I'm looking forward to the offline isolation of a dedicated tracker running right off the X16. I suppose there's a small part of Nostalgia at work there but not much as I do find using my modern DAW can be pretty distracting. Even if I just unplug the damn network cable, Windows still wants to make sure that I know that my online account isn't sync'd just in case I changed my mind, in addition to myriad of other nonsense. So yeah, anything that gets the closer to the finish line I'm all about!

The portmento in particular is something I didn't know quite how to implement. I assumed it was similar to what I was thinking a pitch slide would be (in the tracker case, it's a shift and offset is how I thought about it, though I haven't yet looked at, say, Famitracker, to see how it does it) but with a end frequency to shoot for (namely the note that was played).

In any case, when you get down to it, what makes a tracker is mostly the UI and approach. I think the only thing potentially special about a sound engine for it is, like we discussed previously, channels are typically hard set to particular hardware voices where the composer decides how to use what channels. This has benefits for being of being able to saturate channels more. I like I could use PSG channel 1 for both a chip bassdrum, hi-hat, and maybe an arpeggiator within the same track.

The above is no longer true some modern trackers, like Renoise but in the case of the X16 is how I was going to approach it as it's still sort of "standard" when looking at say Famitracker or Deflemask and this makes the file format more straightforward. This concept doesn't really save any screen real-estate either - I think in the case of Renoise, being largely a sampler at the end of the day (though it has VST and MIDI support), it makes a lot of sense, especially with its out of tracker automations it can do.

Not sure how familiar you are with trackers so forgive me if I'm rehashing what you already know, but essentially you define notes and effects on a piano roll using, typically, a concept of instruments. The instruments would set up defaults and be where you might define a default envelope, etc. They make more sense for FM but even with the PSG make sense - you can setup a "pluck" sound that has a short volume envelope on it, optionally along with a PWM envelope to give it a nice timbre for the pluck. Then you might have a bassdrum which is a fast pitch slide down also with a volume envelope.

Then in the patterm data, I tell it what note, octave, instrument, and optional effects I want. Something like:
 

   | VeraSound 01         | V02    | ... | F1  | FM 2          | ... | PCM |
## | -------------------- | -->    |     | --> | ------------- |     | --> |
00 | ... .. .. .. ... ... | C-1 02 |     | G-3 | D#2 01 FF M01 |     | C-4 |
01 | ... .. .. .. ... ... | C-1 03 |     | ... | ... .. .. ... |     | ... |
02 | C-4 01 .. .. A37 ... | D#1 02 |     | ... | ... .. .. ... |     | ... |
03 | ... .. .. 20 ... ... | D-1 02 |     | ... | ... .. 50 ... |     | ... |
04 | D#4 01 L. 20 A00 V10 | D#1 03 |     | D#3 | --- .. .. ... |     | ... |
05 | D#4 01 .R 10 D04 V1A | G-1 03 |     | ... | ... .. .. D0A |     | ... |
06 | ^^^ .. .. .. ... ... | D#1 02 |     | ... | ... .. .. ... |     | ... |
...
06 | ... .. .. .. ... ... | C-1 .. |     | ... | ... .. .. ... |     | ... |
	

My current file format definition allows multiple effects, so there could be D04, G37, ... I'm not sure if I will keep that yet as another way to do combination effects is with macros (or instruments with their own automation if the effects don't need to be modulated). I also though about allow different envelopes to be selected on the tracker data but I need to see how that might fit into the effect column while trying to keep the pattern data small.

I said all that to say a tracker could surely use a sound engine probably without too many modifications? I wonder if any modifications that would be needed might be something that can be added to the engine. It would be pretty amazing if both an X16 tracker and DAW used the same engine. In fact that might not be the only thing that can be shared - a MIDI solution could be one that might be shared by both. A tracker tends to be a little more limited here - one could set it up to output MIDI but that seems like a bit of an odd solve on the X16. It could read incoming MIDI as an alternative means to edit patterns. Primarily for me, I'd like MIDI sync so I can use the X16 is sort of another instrument in my greater orchestra as it were.

I'm super rambling sorry haha I thing long story short, it makes a ton of sense to me for really all the music and sound applications to be able to reuse and share as much code as possible. A tide lifts all boats kinda thing. So yeah I'd be quite interested in learning more about your engine, absolutely!

Edited January 16 by m00dawg
Added better pattern example

[m00dawg 107]

Overriding my previous comment as I made sort of a BIG change which I think could help reduce the amount of bytes for non-zero rows:

	              | If Note Flag Set       | If Vol Flag Set | If Effect Flag Set                        |
Channel : N/V/E Flags | Note   : Octave : Inst | Pan    : Vol    | Next Effect Flag : Effect # : Effect Data |
5-bits  : 3-bits      | Nibble : Nibble : Byte | 2-bits : 6-bits | 1-bit            : 7-bits   : Byte        |

So now instead of an effect length, there's flags that explain what the next bytes are. N is note, V is vol, and E is effect.  his allows for having rows that are just manipulating notes, volume, effects or all these. So say if we're just manually changing volume, that would only require 2 bytes. Order matters in that, if we have all 3 flags set, the next byte after the flag is the note data, followed by the volume column, then at least one effect (could be more, see below).

Here is where it gets interesting. If the effect flag is set, we know there will be at least one effect, but now there is an effect flag that says if there is another effect defined. This means there could be, in theory, N-effects but the storage requirements are the same - it just means there are now "only" 127 definable effects. That seems like way too many, even with macros, since effects can be unique per sound-chip, though many effects would be common to at least VeraSound and FM (like volume slides, portamento, arpeggiation, etc.).

Edited January 16 by m00dawg
further updates to row format.

[kliepatsch 71]

5 hours ago, m00dawg said:

I suppose there's a small part of Nostalgia at work there but not much as I do find using my modern DAW can be pretty distracting. Even if I just unplug the damn network cable, Windows still wants to make sure that I know that my online account isn't sync'd just in case I changed my mind, in addition to myriad of other nonsense. So yeah, anything that gets the closer to the finish line I'm all about!

Not so much the distraction for me, but the sheer size/quantity of the tools available. Anyway, seems we're headed in a similar direction there.

Thanks for your explanations of a tracker. I have actually played around with a couple of trackers from the DOS era in order to get an idea what it is like to use one. I didn't finish anything with them but I sort of know the concept of the "effects" and the general feel of a tracker. But it's good to read anyway, so I know that it's exactly that what you are after.

5 hours ago, m00dawg said:

In any case, when you get down to it, what makes a tracker is mostly the UI and approach. I think the only thing potentially special about a sound engine for it is, like we discussed previously, channels are typically hard set to particular hardware voices where the composer decides how to use what channels. This has benefits for being of being able to saturate channels more. I like I could use PSG channel 1 for both a chip bassdrum, hi-hat, and maybe an arpeggiator within the same track.

I think I have mentioned in the other thread that the layout for the synth engine is channel-based, so it is indeed suited for the stuff that you are mentioning. But it won't be a one-to-one representation of the 16 PSG channels. We would be giving away *huge* potential if we would let people only define instruments using 1 oscillator at a time. I mean -- what would you do with 16 independent voices, unless you are going for an orchestra? In my opinion, the 16 voices of the PSG are more than enough to combine several PSG voices into one sound and thereby vastly increase the palette of sounds available.

5 hours ago, m00dawg said:

I said all that to say a tracker could surely use a sound engine probably without too many modifications? I wonder if any modifications that would be needed might be something that can be added to the engine. It would be pretty amazing if both an X16 tracker and DAW used the same engine. In fact that might not be the only thing that can be shared - a MIDI solution could be one that might be shared by both. A tracker tends to be a little more limited here - one could set it up to output MIDI but that seems like a bit of an odd solve on the X16. It could read incoming MIDI as an alternative means to edit patterns. Primarily for me, I'd like MIDI sync so I can use the X16 is sort of another instrument in my greater orchestra as it were.

It's really nice to read that you think it could work!
The same overall engine for a tracker and a piano roll DAW would not be possible. Simply because of the timing differences (piano roll allowing for more flexible timing) and the data format made up of rows and "cells", and the concept of effects. Those would overcomplicate the matter.
However, a shared synth engine seems useful. Sounds could be shared between the platforms (much like presets for NI Massive can be used by both FL Studio users and Renoise users). This would be possible I think.

In its current state, the synth engine doesn't allow for a lot of real-time modulation (such as tracker effects), but it is a high priority for me. My idea is to make pitch bend one channel for modulation that is always available (per voice), and to provide a second channel (much like Modwheel) that can be freely mapped to one of the parameters in the synth engine - and even be remapped during playback, but only one parameter at a time per voice.

This freely available modulation parameter could then be used by the tracker effects to achieve whatever they do.
In other words, e.g. if we have a vibrato effect, the tracker would automatically map the "Modwheel" to vibrato depth and apply the correct values to it. That's how I am currently imagining it.
Or we leave the abstraction layer of classical "tracker effects" away and simply do something like:

bit 7 - pitchbend or modwheel
bit 6 - set level or slope
bit 5 - remap modwheel destination
bits 4 to 0 - data

EDIT: one bit should also tell us whether there will be another modulation statement after this. (Much like in your post above). Both slopes and levels should be able to be set at the same time.

Additional bytes for more data as needed by the settings in bits 7 to 5 (e.g. pitchbend setting should be accurate. And I don't know how many bytes a remapping of the modwheel would take, yet)

I don't know how that would suit a proper tracker user

Technically, there is nothing that holds us back from adding more modulation complexity, but it will quickly become CPU intensive. That's when we need to decide: do we want a software that keeps running smoothly under all possible circumstances, or do we tell the user: "Use at your own risk" -- don't modulate too much at the same time, or playback stuttering might become an issue. I have done the math. 16 voice playback seems possible at 8 MHz, even with some modulation going on. But we have to be clever...

5 hours ago, m00dawg said:

The portmento in particular is something I didn't know quite how to implement. I assumed it was similar to what I was thinking a pitch slide would be (in the tracker case, it's a shift and offset is how I thought about it, though I haven't yet looked at, say, Famitracker, to see how it does it) but with a end frequency to shoot for (namely the note that was played).

That's it! The details depend on whether you want a constant rate or a constant time portamento. In any case, you will keep sliding until you hit the target and then disable the slide.

Edited January 16 by kliepatsch

[m00dawg 107]

Haha well I'm having a lot of trouble trying to do a multiquote so I'll just reference some of the points:

Multi-Voice Instruments

I actually didn't think about this, but it's a good point! Can you elaborate on the types of things you'd want to do with more than 1 voice? Chorus and supersaws came to mind here. Delay also. A lot of things I'd do as instrument automation in DAW land I'm used to doing in trackers by way of channels. Here there's definitely a trade-off in terms of file size though - a 3-tap delay would use 3 channels. A chorus would use at least 2 and sort of "manually" creating these effects would use more notes and thus bytes. Likewise if you want to have a long tail on a note, you do that by way of multiple channels. Same is true if you wanted, say, a pulse-saw or tri-saw kinda thing like you'd find in a normal synth.

If channels and voices are not 1:1 mapped, my initial feeling is to let the sound engine figure it out But for peristence based trackers (like FamiTracker), many effects "stay on" until you turn them off or change them. This can be really convenient but might complicate things a lot here. Likewise, there isn't a concept of a "voice engine" in a tracker typically. You never "run out" of voices but not channels (ignoring Renoise here for the moment).

I'll have to ponder how all that'd work methinks. Takeaway is that I think such a concept, while pretty new for classical tracker, could be possible. It can also be something baked into the tracker - as in there might only be 8 channels instead of 16 in the tracker, but they can use multiple voices. Compared to SID trackers, this would be a pretty new concept. For me I have no problem doing all that in pattern data and actually kinda like that level of expression since I find it easier to do (compared to say automation lanes in Ableton) but that might be a question to posit to other tracker musicians. Especially ones used to something like Renoise, which really blends the idea of a DAW with a tracker. I don't think there's quite enough horsepower on the X16 for something of that magnitude (and/or that might be beyond my skillset :P).

Effects

So here I was thinking of a few things. I was coming up with a concept of envelopes. For VeraSound, LR+vol and wave+PWM could use the same envelopes (they're the same precision) and these can be assigned to things in the instrument AND/OR changed in the pattern data by way of an Envelope Effect. I'm unaware of trackers that used this concept (other than Renoise but I think it handles it differently) but that could be neat.

Then there's pattern level "automation" by way of effects. These would be stuff like these:

https://gitlab.com/m00dawg/commander-x16-programs/-/blob/master/command_tracker/effects/index.md

Of note, different trackers handled these differently. Impulse Tracker didn't have effect persistence. SO if you wanted to do a volume slide down (C04) you would have to define that over and over in the pattern data. In FamiTracker, you "enable" it once and it continues to take affect until you turn it off. I think the latter is more efficient (probably why FT does that) but the composer has to remember a bit more.

Most of those above are common tracker effects but how they are implemented could vary wildly. So they could be using things like modwheel if under the covers, they're using things like modwheel or interact more directly with the sound engine and the composer doesn't have to be any the wiser.

Instrument Compatibility

Yes I think sharing instruments is something that is indeed possible, at least if/once figuring out the voice to channel concept. I wrote up some ideas of what an instrument would be here:

https://gitlab.com/m00dawg/commander-x16-programs/-/blob/master/command_tracker/file_formats/file_format.md

That's pretty early on and incomplete heh but my envelopes concept included the concept of looping, so if one wanted an instrument to have a default vibrato, one can either set it in the channel data (noting that, if there is effect persistence, it can be set once at the beginning of the song), or use an envelope with a loop. It also probably make sense to have a concept of instrument effect default though, like pre-setting a vibrato value like one would have in the channel data (so it works on any channel, not just the one with vibrato set). That would certainly be more efficient than using an envelope I think.

 

A lot of these concepts come from how I'm used to using FamiTracker. I think to your point of multi-timbral voices, not all of it may actually apply on the X16 though. E.g. if all instruments were 2 voice, that's still 17 total channels across 3 sound engines.

 

It's dinner time so I gotta cut this short but this is a great conversation!

Edited January 16 by m00dawg

[m00dawg 107]

I've had this in my head for a while on how multi-voice instruments could work...if there remains a concept of 1 to 1 channel/voice mappings, then I wonder if this could be implemented with channel skipping.

So let's say I have something like:

ROW| VERA01 | VERA02 | VERA03 | ...
## | -------+--------+--------+
00 | C-4 01 | --- -- | --- -- |
01 | ... .. | ... .. | ... .. |
02 | C-4 02 | C-4 03 | ... .. |
	...
	

Sort of hard to represent in text, but basically if instrument 01 allocates 3 voices, we disable channels 2 and 3 (represented by the dashes, though in a multi-color tracker, we would probably just dim or highlight them to indicate they are in use). Then when we use instrument 02 on channel 1, if it's a single voice instrument, we know channels 2 and 3 are available and can be used for other things (as in the example).

It could even be possible to override one of the voices (say by placing a note on VERA02 at row 01 in the above example where instrument 01 might have a long tail), then their new note would take precedence. And actually likewise, if someone wanted to use a multi-voice instrument on channel 01 at row 00 and then put another instrument on channel 02 on the same row, channel 02 should override the voice.

So in other words, the UI could provide cueues to channel usage when using multi-timbral voices, but the composer can still do as they please if they want to override voices.

This seems like it would be non-trivial to implement in the tracker UI but provides the ability to use complex instruments while preserving the sort of lower level nature of using trackers while still adding a lot of convenience AND saving space. It also puts the voice overriding in the hands of the composer. Such a concept provides a lot of benefits - at the cost of unknown complexity (I'm currently not sure how complicated this would be to implement in practice, particularly on the X16).

I could see this being HUGELY useful for things like delay. Handling that at the instrument level means we don't have to worry about manually wrapping the delay when we cross a pattern boundary.

I think it could be limiting in terms of effects though - maybe for multi-voice instruments we could assign effects to each voice. So in a two voice instrument, effect column 1 effects voice 1, column 2 voice 2. That's not ideal when we want to apply affects to both though (like pitch slides).

Anyways hopefully that all makes sense!

 

[m00dawg 107]

Was talking about the YM2151 situation on the Discord. It's an aside but I'm hoping the Rev 3 board will move the YM chips to a daughter board which can be swapped with an FPGA board (like the VERA) when 2151's start to either fail in large numbers or get hard to find.

But while talking about that, we started talking about how the C256 solved this, and that got me down the rabbit hole of the C256 tracker that's being worked on. It seems based on RAD tracker, so I pulled up their file format (available at the bottom of this doc) and it's actually not too far off from what I was thinking.

The one addition is that it tracks byte counts for each pattern. I'm not sure why by just looking at the file definition. Could be a very reasl reason to do that - I'm just don't know yet. It also uses a different "end of row / end of pattern" flag than I do.

 

[kliepatsch 71]

I have finally come around to publishing the CONCERTO synth engine, along with its source. (Link to GitHub in the description).

Now you can see why I think multi-voice instruments are a useful thing Try out some of the timbres and the demo loops (buttons at the bottom).

[m00dawg 107]

Indeed that sounds beautiful! I was having some trouble with using the mouse when running it on my workstation (Linux) and haven't had time to take a look at that (looks like an emulator / desktop environment issue - nothing to do with Concerto). I was able to use it via Try It Now and though it did skip some, it was enough to hear the demos and make some nice juicy squares and super-saws. Very lovely!

I definitely agree, multi-voice instruments have some clear benefits. The challenge is trying to figure out how to manage that as I'm not aware of a tracker that had dynamic channels and part of the power of a tracker is allowing the composer very precise control. I wouldn't want to take that away but having multi-voice instruments add a ton of convenience and avoids a lot of monotony and repetition (e.g. with single voiced instruments, there would have to be a lot of channel duplication and in-pattern effects to do similar things).

I like the idea of UI hints with the channel skipping thing  but it might be better to simply have some sort of UI queue as to which voices are in use. That takes away a bit of flexibility (the composer no longer would know which voices they are about to clobber if they really want to try and use all 16 channels at the same time) but that's probably ok.

The # voices thing is probably not something I'll worry about anytime soon anyway - there's a lot to do in going from zero to a full tracker and trying to do all that while learning assembly is a very tall mountain. Given some of the KickC conversations I thought about perhaps considering that but learning 6502 is one of the main things I want to get out of the X16. Point is, this will probably be a long project

Anyways fantastic results with your synth! Kudos for putting together such a rich sound engine in a very short period of time! It's quite wonderful! I took a quick look through the source code but will try to take a look in earnest later today.

 

Edited January 20 by m00dawg

[kliepatsch 71]

Thanks I am glad that you like what you have seen so far.

Maybe you should give yourself one or several smaller toy project before tackling such a big project with 6502. That's how I've done it, anyway, and it helps understand the source code of others better.

Anyway, if you are going to look at the source code, take a look at the Readme first. This will give you a rough idea where to start looking for what.

Is the problem with the mouse that you cannot reach every area of the UI? If so, simply move the mouse across over the whole window. This somehow resets the "mouse coordinate offset" or something so that you can get to everywhere.

I am not worried about the decoupling of the tracker channels with the hardware channels. Because the hardware PSG channels are identical, the user won't notice it. And in the cases where you actually want to continue using the same PSG channels, you can do that by using portamento (well, I'd have to check the details in the source myself). I like the idea of somehow monitoring how many of the PSG channels are in use at any time. Similar to how Synth1 does it (one of the most iconic VST synths).

[m00dawg 107]

Yep, that's the plan. Making a bunch of effectively useless programs that help me figure something out. I'm at the stage of assembly that it was a treat when I was able to simulate a pattern row count that updates with VSYNC Next big thing is really figuring out the textmode VERA stuff. I started messing with that a while back and need to again. Some heavyweight things I need work well for a tracker (like scrolling a pattern on playback). I took at look at your UI code just a moment ago to try to figure some things out. Some of it was beyond my understanding but I'll get there. I just learned a bit about jump tables yeterday haha.

On the note of the mouse, I'll have to play with it more. The problem is compounded by my desktop environment (Linux) where I have focus-follows-mouse set and where the X16 isn't grabbing the mouse. So I can't easily "wrap". I though the X16emu had a fullscreen mode but I didn't find it as a flag (I think it's a keyboard shortcut?)

You mentioned "I am not worried about the decoupling of the tracker channels with the hardware channels. Because the hardware PSG channels are identical, the user won't notice it." can you elaborate?

Normally in a tracker, the channels and voices are 1:1 but when supporting multi-voice instruments, things are now not strongly coupled. Which I think is worth it, but means you could "run out" of voices. So yep having a way to see which voices are in use makes sense. Many trackers have this sort of feedback, though since voices == channels in those cases, it is a bit different than here. Something as simple as the equivalent of LEDs on a real synth showing which voices are active I think would be sufficient.

Another thing to ponder is what happens when you use a 3 voice instrument followed by a 1 voice instrument on the same channel. Previously, when voices == channels, the second note immediately took over (there was no release). This might seem like a drawback but is actually a very useful feature (if you want to have long release tails you just use multiple channels, no biggie). But with decoupling voices, this changes things.

Definitely A LOT to think about there. For now, I'll focus on the fundamentals I think and worry about these more complex problems later

[kliepatsch 71]

Well, this is the first time I publish a larger coding project and I don't know ... If I compare it with e.g. @Stefan's code of X16-edit (which helped me a lot with the file stuff btw), my code looks awfully messy Anyway, I don't expect anyone to grasp what's going on after a few minutes of looking at it (especially in the gui.asm file). I tried to explain the overall concepts at the top, which are hopefully helpful.

Talking about the voice assignment is a bit too tedious for me at the phone right now, might do that later.

[m00dawg 107]

1 minute ago, kliepatsch said:

Talking about the voice assignment is a bit too tedious for me at the phone right now, might do that later.

Haha yes it's a potentially rich topic. I can't type more than a sentence or two before I get irritated at my phone

This is a solveable problem for sure, it's just a question of the pros and cons of various solutions. E.g. When interfacing with MIDI instruments, Renoise doesn't cut notes off within a channel (so playing one note after another simple engages the release rather than a cut). That's one option here - problem is when you're playing notes right after each other - there's no way to control that. But perhaps this could be controlled via an effect parameter that dictates if a channel will use note off or note release when playing the next note. That actually might be an elegant solution thinking about it. So having a "Mode" effect (Mxy). So say setting M10 uses note release and M11 uses note off.

[SlithyMatt 527]

4 hours ago, m00dawg said:

I was having some trouble with using the mouse when running it on my workstation (Linux) and haven't had time to take a look at that (looks like an emulator / desktop environment issue - nothing to do with Concerto).

That's a general issue with the emulator mouse support. Nothing you can do about it from inside the X16 code. What I find helps is to drag the mouse across the screen until it stops as the edge, then bring it back across all the way to the other side, then back to the middle. From there, do the same in the vertical directions, and the mouse should be synched with emulator.

[kliepatsch 71]

Here's what Concerto does with the voicing.

First off, terminology inside Concerto:

• Oscillators correspond to the 16 voices of the VERA PSG. Every sound can use one or more oscillators.
• Channels are the way Concerto organizes playback internally. There are 16 Channels. Each channel is monophonic, i.e. it hosts a single voice that is either active or inactive.
• Voices refer to the notes that are played. Because each channel can host only a single voice, the two terms can often be used interchangeably. If you play a note on channel X, you automatically address voice X.

Each note-on event needs to specify

• Channel
• Timbre (which of the 32 synth patches to use)
• Pitch
• Volume

If a note-on event is issued, it is first checked whether there is already an active voice in the specified channel. If it is a different timbre than the one being played, the old voice is stopped and the new voice started. If the new voice has the same timbre as the old one, the voice is updated according to the "retrigger" and "portamento" settings.

Every time a voice is started, it is checked if there are enough oscillators available. If yes, they are taken off the list of available oscillators (and returned to it once the voice is stopped). If there aren't enough oscillators available, the note simply is not played. It can run out of oscillators.

There are two types of note-off events. Hard and soft ones. Hard note-offs (in the source code "stop_note") immediately deactivate the voice and return the oscillators to the free oscillator list. Soft note-offs ("release_note" in the source) trigger the release phase of all envelopes. As soon as the master envelope (the first one of the three) hits level 0, the voice is turned off automatically and the oscillators are returned to the free oscillators list.

Note-off events are simply communicated via the channel number.

 

Now I'd like to comment on a few things.

22 hours ago, m00dawg said:

Normally in a tracker, the channels and voices are 1:1 but when supporting multi-voice instruments, things are now not strongly coupled. Which I think is worth it, but means you could "run out" of voices. So yep having a way to see which voices are in use makes sense. Many trackers have this sort of feedback, though since voices == channels in those cases, it is a bit different than here. Something as simple as the equivalent of LEDs on a real synth showing which voices are active I think would be sufficient.

Yes, Concerto can run out of oscillators. And I agree some kind of visualization of how many are used seems reasonable.
I could imagine something in the direction of how Synth1 does it. See for reference

https://youtu.be/__2AFeG4xII?t=187

In the bottom right of Synth1 you see the 32 voices of the synth and which voices are currently active. Since the sound is making use of unison, it uses several detuned voices per note.
In our case, we would instead be displaying individual oscillators instead of voices (in the sense that a voice can employ several oscillators).

But your suggestion

On 1/17/2021 at 3:25 AM, m00dawg said:

So let's say I have something like:

 

ROW| VERA01 | VERA02 | VERA03 | ...
## | -------+--------+--------+
00 | C-4 01 | --- -- | --- -- |
01 | ... .. | ... .. | ... .. |
02 | C-4 02 | C-4 03 | ... .. |
	...
	

 

Sort of hard to represent in text, but basically if instrument 01 allocates 3 voices, we disable channels 2 and 3 (represented by the dashes, though in a multi-color tracker, we would probably just dim or highlight them to indicate they are in use). Then when we use instrument 02 on channel 1, if it's a single voice instrument, we know channels 2 and 3 are available and can be used for other things (as in the example).

seems possible, too. This would suggest that the user directly chooses which PSG oscillators to use instead of letting the synth engine manage that. I'm not a big fan of it, but it would provide more control, especially if you combine it with the user being able to stop individual oscillators and reuse them, while other oscillators of a previous note are still going. This is what you said if I understood you correctly

On 1/17/2021 at 3:25 AM, m00dawg said:

It could even be possible to override one of the voices (say by placing a note on VERA02 at row 01 in the above example where instrument 01 might have a long tail), then their new note would take precedence. And actually likewise, if someone wanted to use a multi-voice instrument on channel 01 at row 00 and then put another instrument on channel 02 on the same row, channel 02 should override the voice.

So in other words, the UI could provide cueues to channel usage when using multi-timbral voices, but the composer can still do as they please if they want to override voices.

If you did this, you would need some restructuring of the current voicing system. You need to allow for the extra freedom to terminate parts of a multi-oscillator sound. Currently, all oscillators are released at the same time. This would change if you allowed for partial releases.

It's good to see that you are already on your way with 6502 machine language.

[m00dawg 107]

All great points!

Oh boy the voice vs channel thing might get confusing haha. But yeah sounds like your engine isn't far off from how a tracker would work, though you lost me a bit on your channel vs voice explanation. You mentioned:

Quote

Each note-on event needs to specify

• Channel
• Timbre (which of the 32 synth patches to use)
• Pitch
• Volume

But you mentioned each channel is monophonic? But the timbre (synth patch) could itself use multiple voices?

Your note-on looks very similar to my pattern note-on as well. For example:

	PSG NOTES [CHANNEL 0-15, VeraSound 1-16]
2-X bytes
                      | If Note Flag Set       | If Vol Flag Set | If Effect Flag Set                        |
Channel : N/V/E Flags | Note   : Octave : Inst | Pan    : Vol    | Next Effect Flag : Effect # : Effect Data |
5-bits  : 3-bits      | Nibble : Nibble : Byte | 2-bits : 6-bits | 1-bit            : 7-bits   : Byte        |
	

The NVE flags indicate what comes after the Channel/NVE byte. You can have note, pattern, or effects (or all) in pattern data so this helps keep things smaller (in the sparse format). You can see there's a "next effect" concept here where there can be, in this case, N-number of effects. In reality, the main issue here is how to represent a 'non-sparse' pattern buffer such as when the composer is actively writing a pattern. Having that in a non-sparse format makes lots of things much easier. In the sparse format, if I add a note in between two previously defined rows and channels I'll have to do a lot of shifting around of data. Or at least I haven't figured out a great way to solve this. That means, practically, the number of effects might have to be limited.

In terms of the channel allocation, instead of the example I mentioned previously (the one you commented on), it is the most powerful option but the trade-off is having to navigate through patterns more when composing a pattern. I kinda like the idea of moving allocations to the end (last channels). So instead of, in the above example, channels 2 and 3 getting flagged as used, 15 and 16 would get flagged as used instead. That way you can keep the pattern data easier to follow since there won't be a lot of empty channels in the middle a pattern to contend with - more of the data will get pushed to the lower channels this way. The downside is the composer looses control of which notes get overtaken in this scenario I think - or at least it becomes a lot less obvious Instead of pattern-UI cueues though it may be simpler and almost just as effective to have an active voice UI area (e.g. Synth1).

I'd imagine most songs, especially with multi-voice instruments, won't use 16 channels commonly within patterns. I could see it in some cases but probably won't be super common. In this case, multi-voice instruments is a nice solve because it keeps the pattern data more sparse (at least in the sparse file format).

[kliepatsch 71]

5 hours ago, m00dawg said:

Quote

Each note-on event needs to specify

• Channel
• Timbre (which of the 32 synth patches to use)
• Pitch
• Volume

But you mentioned each channel is monophonic? But the timbre (synth patch) could itself use multiple voices?

Correct. Channels are monophonic and timbres can be used multiple times. For each event it needs to be specified on which channel it is. I don't see why that should contradict each other
Edit: I think I get the point. In a tracker, the events are on a channel, so you don't have to specify which channel for each event. You simply put them on which channel you want them. But nevertheless, you have to communicate that to the synth engine.

 

5 hours ago, m00dawg said:

In reality, the main issue here is how to represent a 'non-sparse' pattern buffer such as when the composer is actively writing a pattern. Having that in a non-sparse format makes lots of things much easier. In the sparse format, if I add a note in between two previously defined rows and channels I'll have to do a lot of shifting around of data. Or at least I haven't figured out a great way to solve this. That means, practically, the number of effects might have to be limited.

I was worried about this as well, when I started out. But then I read somewhere in this forum about how Stefan had solved this problem for X16 edit. I don't recall where exactly he talked about it, but he manages memory in blocks of 256 bytes I think (memory "pages"). Each block of data contains a couple of bytes of metadata, pointing to the next and the previous block and how many bytes of the block are actually used. That way, if the composer inserts/deletes effects/notes/whatever, you never have to move more than 256 bytes of data around (which should be fast enough for human editing). If more space is needed, you simply allocate a new page of data and move into it what couldn't fit into the other pages, and update the pointers of the previous and the following page. He puts those pages into the banked RAM. Of course, you need to keep track of which pages have been used and which ones are free. (And memory could get fragmented over time, which Stefan also solved IIRC). I think a lot of these concepts could be applied to a tracker. I found it helpful to at least know that those problems can be solved.

 

Edited January 21 by kliepatsch

[m00dawg 107]

@JimmyDansbo Here's the thread! If you want to see my entire pattern explanation it's here:

https://gitlab.com/m00dawg/commander-x16-programs/-/blob/master/command_tracker/file_formats/sparse_pattern_format.md

I should preface this by saying this is a big topic so you're advise thus far has been super helpful! And if this is a bit too much to get into, no worries!

This is tracker centric so if you're not familiar with music trackers, as a quick intro to patterns, a pattern is comprised of rows. Usually it's 64 (though can be other lengths in many trackers). It's a bit like a piano roll. Each row then has multiple channels - for the X16 there is, currently, 25 (16 PSG, 8 FM, 1 DPCM). In each channel, we can have a note, volume, or 1 (or more) effects. So borrowing an example from up above:

ROW| VERA01 | VERA02 | VERA03 | ...
## | -------+--------+--------+
00 | C-4 01 | --- -- | --- -- |
01 | ... .. | ... .. | ... .. |
02 | C-4 02 | C-4 03 | ... .. |
	...

So in this simplified example, at row 0 we are playing C at octave for using instrument 1 on channel 1. Row 1 has no data, and row 2 has note data for channels 1 and 2.

In a non-sparse format, this would end up being a big matrix as one way to look at it. Where every row has data for all 25 channels, even if empty. So in terms of multi-arrays, I could perhaps see where there is 25 arrays of 64 where each array is a channel maybe?

But for sparse storage, it gets more complicated. In the above example, the sparse format might be something like:

00 01 C-4 01 30
02 01 C-4 02 02 C04 03 31

I simplified this a bit compared to the real file format definition but hopefully it shows off the problem. Row 00 would be, say, 4 bytes. row 01 isn't stored at all since it has no data, and row 02 is 8 bytes. 30 and 31 are special channels that indicate we are at the end of defined data for a row (30) and pattern (31). Again simplifying a bit since the actually channel byte includes some flags so the byte value would be different.

A full song is comprised of a series of patterns using an order list (internally an array of pointers to patterns).

Since the patterns are compact and variable data, in this scenario, I don't think I can avoid storing the data in a single sequential array of bytes?

Hopefully all that makes sense. I recognize that it's a non-trivial problem and also point out I'm still pretty green when it comes to assembly programming here. My sparse file format actually looks similar to GoatTracker's so that makes me think I'm on the right track. It seems like some of the modern trackers on PC (e.g. Deflemask) use non-sparse patterns since there's plenty of RAM to spare, but on the X16 I don't think I can get away with that easily.

If I pull back on a few features (notably multiple effects) then I can likely cram a single full pattern into 1 bank of 8k RAM. That makes things considerably simpler - each pattern gets mapped to a bank and we're done. And for the tracker, this may be ideal. It would still be too much RAM (in my opinion anyway) to use for just playing a song back, say in a game, where the sparse format saves A TON of space.

 

 

[JimmyDansbo 138]

Unfortunately I am not familiar with trackers other than I have used some to play music in DOS many years ago.

2 hours ago, m00dawg said:

00 01 C-4 01 30
02 01 C-4 02 02 C04 03 31

Where does the '30' and '31' come from in the end of those 2 lines?

I am thinking about the sparse format... there might be a way

Edited January 23 by JimmyDansbo
re-read post above and my question is no longer relevant.