Jump to content

Command Tracker Dev Log


Recommended Posts

I thought I would start a new thread as the old one was about sparse files, which I decided not to use currently. I think it can be a goal for a nice on disk format but I'm finding using hiram is working pretty well. Yes it's pretty wasteful but saves on CPU cycles and will make editing patterns, when I do start working on the UI, perhaps bearable.

Here's the latest version! With support for 6 channels, though no effects other than volume and note-off. And also no concept of instruments. Each channel is configured to play a static instrument for now:

Next steps, there are many 🙂  klieptasch and I have been sharing some ideas on how to integrate the Concerto synth engine with the tracker. This opens the door for some really interesting things, and means I can focus on the UI. In this way it also keeps the synth engine separate from the tracker engine so folks could more easily use the various parts of both programs if they wanted to do something else (though given both apps are open source it would also be nice for folks to contribute to either!)

The UI will be a big deal and I'm not actually show how complicated it will be yet haha. I have ideas on how I might do some things but that will all be new to me. But then again so was 6502 and I'm pretty happy with the tracker thus far!

I'm not sure what I will work on next - trying to interface with Concerto or start the UI. I have ideas fresh in my head for Concerto but also editing patterns by hand is quite the pain so I'll have to ponder that more.

Anyways the source code is here:

https://gitlab.com/m00dawg/command-tracker/-/tree/master

I don't have a demo up on the forums yet because I already have some file I/O interactions (pulling in Petdraw screens but this is also the basis to handle song files). But since there is no functional UI, apart from hand coding patterns in hex, not too much to do yet 😉

 

  • Like 6
Link to comment
Share on other sites

I spent part of the weekend better organizing the code into scopes and completely redoing how UI elements are drawn. Using the PetDraw save files was just a bit too much as they are over 9k (since it's the full screen buffer). So I experimented around with programmatically drawing the UI. Might be better ways to do that but it saves a significant amount of space (noting I want to use as much of hi RAM as possible for the banked patterns). I also added the ability to stop and start the song - using the F5 and F8 keys ala Impulse Tracker (F7, as I recall, plays current pattern or plays from current row position). The is the start of actual user interactions.

 

And while I don't have it working yet, I did start to work on an order list section. I decided to do this first because it requires figuring out a lot of things I'll need to do in order to edit patterns, but at a much smaller scope. So right now one can put in hex numbers into the list along with use the arrow keys to navigate around. This doesn't currently account for the editable section of the pane yet, nor does it yet scroll once you get further down the order list.

This is where things currently get a bit odd. Having to convert to/from screen-codes is something I need to improve as I'll probably need to do that quite a bit. My thought was to edit largely in VRAM and to then pull and translate the data at various points (this is where there is that "Press enter to save orders" message). For the tracker, it may have to be more fluid than that, but for the orders list, to start, it may work as a start. I think to do this easily I'll need to use the lower layer like I do with the pattern data. That requires decoupling playback from the pattern display a bit more than I currently have it setup.

But once I do that, I think such a solution could work ok. In pressing enter, it will read the patterns values in the order list in VRAM, since they are characters, it'll then have to convert back to hex numbers (noting the screencodes thing), then simply write out the order list until it sees '--' and write that out to the actual order list array.

I'll also need a way to goto patterns (as I recall in Impulse Tracker, you did that by pressing G on the current pattern the cursor was at) but I think that part would be easy compared to the above but flows right into then needing to figure out how to edit patterns. That's going to be quite a bit more complicated as I can't fit the entire width of a pattern into VRAM and, likewise, editing patterns is much more fluid than the above order list ideas. I'll need to keep track of a lot more state of where the cursor is relative to the real pattern and have a means to scroll/draw to be able to view all 25 channels.

A bit of a ramble so I guess I'll stop here. Small steps, but steps nonetheless!

 

  • Like 5
Link to comment
Share on other sites

Well no new video yet. I'm having a bit of trouble reliably reading from VRAM that I haven't figured out, but then also ran into a an issue with storing the entire order array in VRAM and using hscroll/vscroll to navigate around. The trouble there is I thought I was using a resolution of 128x256 on the second layer, but it's only 128x128 and I seemed to be clobbering some important video memory by just increasing it.

Thing is, I don't need the drawing routing for the orders list to be fast - I was using the scroll stuff just to, in theory, make it easier to navigate around since I just move the display and I wouldn't have to track as many things (such as the current order # the cursor is at). I think given this, plus having to juggle the pattern data also using the same layer, I will have to scrap these plans.

On the pro, I did write a routine to convert from characters on screen back to a number, something I will need to do regardless among a few other things.

Link to comment
Share on other sites

Whew it's taken me several days to finally get to the point where I can seemingly edit orders and they're saved to the order list. I don't have this yet integrated into the player yet so I don't have much to show. And in fact the next thing is to rework how all that actually works, though I'm going to try to at least get editing orders sort of working before I tackle that.

I ran into a weird case where user inputted letters (but not numbers) was giving me back the hi value screencodes. So instead of 'A' being $01 it was $81. That really threw me off and I'm not sure why it's doing that. Other than, perhaps, technically I typed a lower-case 'a' though in VERA it's properly rendered as uppercase. I'm sure there's an explanation there but I couldn't figure that out after significant digging (and definitely a place I hope the X16 dev docs cover).

It's taken me much longer to work on the order list but it's all been good work. I feel like I have a decent handle on how to keep track of things like the cursor. Admittedly I don't have yet scrolling working in the orders list. Pondering that a bit since I currently am allowing up to xFF orders. It looks like I can just squueze the full list on the display but that would be pretty cluttered. I like scrolling better and, either way, I'll have to solve that when editing pattern data.

So no new demo yet, but hopefully soon!

  • Like 2
Link to comment
Share on other sites

Orders are finally working!

As is a (very basic) UI implementation. I'm sure there's a better way to handle it, but it works for now and lets you play the song sort of int he background while moving between panes. You can even update the orders list while it runs, although I'll probably want to highlight which order is playing. Also need to setup scrolling on the orders list so you can go all the way through all xFF orders.

Some minor UI issues - like the horizontal bar looks pretty ugly while playing (it doesn't connect to the rest of the UI) and there are dangling connectors on the panes but that feels like a minor issue.

Next is actually being able to edit patterns...

  • Like 1
Link to comment
Share on other sites

Pretty quick update. I just wanted to share the start of pattern editing! Not much yet - there's no key to note translation yet (that's why there's just 3 of the same letter) but navigation works mostly how I want. No scrolling yet either but it's getting there! Obviously adding note translation is the next thing along with keeping track of which octave you're on.


The compiled code size is currently around 4.2k so far so I think I'm doing ok. Gives me hope I shouldn't have any issues with low RAM to add in the remaining features. I started thinking about this because the keyboard-to-note routine might have a lot of branching in it. At least for the first go 'round I wasn't going to worry about optimizing that too much. There's also some other room for optimizations I'm sure.

  • Like 1
Link to comment
Share on other sites

Pattern editing works!

It's buggy....(and perhaps messy) like if I edit volume after adding a note, something doesn't seem to go quite right. Likewise while I can delete a channel/row (with SHIFT-BACKSPACE), my code to move the cursor to the right spot doesn't work right. So if I'm editing say effects and shift-backspace, it does funky things.

It works by using VRAM and converting during the save process. This means I have to track less data but means I have to do more work when I save (hence the bugs).

Oh and scrolling doesn't work. I'm tracking the "screen" pattern/channel positions separately from the global pattern/channel in hopes it won't be too bad to implement. Of course I need to get that working (on the pattern list and order list) and, you know, not have crazy bugs, for it to be useful.

Then after that actually loading and saving songs where I can also add all the planned channels. That will perhaps be the first barely usable version - barely as there isn't any effects yet, no real synth engine to generate the sounds, no instrument editing as of yet. Hopefully that's when I can look at interfacing with Concerto to help fill in some of these gaps.

Concerto doesn't yet support FM so I'll have to look at adding that. The real goal is being able to use both FM and PSGs (e.g. you can't do this in Deflemask as it doesn't have an X16 mode - at least not yet).

But I'm getting ahead of myself! One step at a time...

 

  • Like 2
Link to comment
Share on other sites

I didn't make a video this time but I managed to fix the bugs with the pattern editor. It's now working reliably! Though I haven't yet implemented scrolling through the pattern while editing so that will be next. Part of the reason I didn't break out a video was because very quickly I can see the need to have some nice to haves (like copy paste, insert note/delete note which shifts the row, etc.) but those will likely wait until I have the basic functionality done. I can't even save songs yet which is the next thing I'll be working on once the initial work on the pattern editor is done.

Currently, the program (including buffers) is 4934 bytes (on disk). I found that pretty crazy myself! I'm not suggesting my code if compact (it likely isn't) - moreso that it's amazing how much you can do in assembly on an 8-bit computer, particularly when comparing it to the huge sizes programs are on modern machines.

 

  • Like 3
Link to comment
Share on other sites

Thanks! It's been quite the learning experience! But I'm super excited with the prospects of making tracker style music on this thing given the hybrid PSG and FM approach and I couldn't see any real alternatives (at least if/until Deflemask gets an X16 mode). The ultimate goal is for it to be a capable tool to make music that's also compact and suitable for integration into apps. That will require the sparse format (or exporting into a sort of VGM style format). That's a ways off but is an important goal. One step at a time though!

If I can make music on the X16 I'll be happy, but am hoping others may find this useful, both to use directly, but also for seeing what sort of trackers can be made on the X16. I have to think someone with more assembly experience than me can probably make a better one 😉 Or since Command Tracker is modeled loosely after ScreamTracker and ImpulseTracker, I could see someone working on a FastTracker style tracker since that UI style seems to be more dominant in some of the modern popular PC trackers (Famitracker, Deflemask, Renoise, for instance).

 

  • Like 3
Link to comment
Share on other sites

A video this time! If a short (and silent) one, but a video nonetheless!

Here I'm showing off pattern navigation whee! It's nearly complete - there's still some note save bugs, but I squashed many of them. What isn't shown is scrolling left - right now you can only scroll right and otherwise have to reload the editor to get back 🙂 I'll fix that, I just ran out of time this evening. Likewise, I'd like to have tab work for scrolling faster through the channels.

As you can see, 25 channels is a lot! I'm not sure if the final version will have 25 or if I'll cut down the number of PSGs and do some sort of dual voice solution. Purists won't like that I"m sure (wanting to have the option to fully exploit all channels) but it does add some benefits like making it easy to do "supersaws" as instruments. As I plan on trying to use the Concerto synth engine still, this may not be that big a deal to worry about (with the trade-off being decoupling of channel vs voices). I'm also not sure if DPCM will remain as I'm worried it might be too CPU intensive to really be viable in the way it would be used in the tracker. I'm not sure on that one yet. I'd love to have it for some NES style DPCM drums and things.

Fortunately, that's not something I have to think about in depth today and more on that at a later date I'm sure.

I will fully admit the code is starting to get a little, erhm, artisanal? I'm going to need to clean things up some and add comments, organize variables, etc. so I might opt to consider that before I jump into the save/load functions but those are definitely next on the list once the UI is at a minimum viable state.

  • Like 4
Link to comment
Share on other sites

It is great to read about your progress and I am looking forward to be able to really play around with the tracker.

1 hour ago, m00dawg said:

I'm going to need to clean things up some and add comments, organize variables, etc.

When my projects get to a certain size, I usually go back to the beginning and try to read my code with new/fresh eyes. This usually results in more comments in the code as I need to explain to my self what certain pieces of code does.

I also try to look at macros and functions and see if I can "lock it down", meaning that I am fairly certain that the code is correct and does not need fixing or tweaking. When I do find those, I usually move it out of the way or "encapsulate" it in "****" lines letting my self know that I don't need to look at this again unless there is a problem.

Next time I decide to read from the beginning, is is usually a lot easier because some macros and functions have already been locked away and the rest should have better comments.

It seems like a lot of work, but in my experience, it has helped me weed out bugs before they appeared and it usually also helps me see where I have inefficient code.

  • Thanks 1
Link to comment
Share on other sites

Thanks Jimmy that's great advice and yep well worth the exercise I think! The pattern editor got a little sprawling, and there's place where macros (and functions) could be used, most certainly. I use macros for some things, but didn't think about using them as a means to make things a little easier to follow. That makes a ton of sense but I hadn't though about it that way until now 😉

  • Like 1
Link to comment
Share on other sites

Finally! The pattern editor is (mostly) bug free! I have a few input bugs but they don't seem to affect pattern saving adversely (which was not previously the case). This means it's actually usable as a very basic demo. Only the first 3 channels are setup - the others might make noise depending on whatever default state the VERA is in, but since I don't have instruments or a real synth/voice engine, things are hard-coded. Really just a means to do and test basic pattern and order editing and show off some of the design concepts I decided to do:

Not the quickest demo but hopefully folks can see a bit more about what I'm thinking instead of the previous quick and voice-free demos.

At this point, the pattern editor is good enough to leave for now I think (even with the minor artifact bugs) and start work on saving and loading. At least that is my plan next as that will make it nearly almost usable. After that there's a few next steps I might do. I might make a very basic sound engine so folks can configure sounds in the pattern data to make real music (albeit without instruments) while I work on a proper sound engine. My hope is to be able to interface with Concerto as I have mentioned previously. I've put thoughts into how I might present the engine via the tracker interface, from presenting all 16 channels and letting Concerto manage the voices (noting that if you use multi-timbral instruments, you cannot effectively use all 16 channels); to reducing the channels down to 8 and making instruments dual voice or maybe even tri voice. I'm not quite sure. Actually for folks that are familiar with trackers, or for those that just might have an opinion, curious as to people's thoughts! I got back and forth on the best way to manage all that.

As I keep saying though, one thing at a time - so yep next is figuring out how to load and save files (including loading data into various parts of memory)!

  • Like 2
Link to comment
Share on other sites

Thank you very much for the video. Nice to hear you talk about your progress as well as the speedbumps. As a bonus I now have an idea on how to actually use your tracker 🙂 

I find it a lot easier to get the basics by watching a video like this than to have to read through documentation that often describes a lot of theory that I don't really understand anyway. When I get a better grasp of how to use this tracker, I will most definitely need real documentation to help me get to the next level.

- Keep up the good work

Link to comment
Share on other sites

10 hours ago, JimmyDansbo said:

Thank you very much for the video. Nice to hear you talk about your progress as well as the speedbumps. As a bonus I now have an idea on how to actually use your tracker 🙂 

I find it a lot easier to get the basics by watching a video like this than to have to read through documentation that often describes a lot of theory that I don't really understand anyway. When I get a better grasp of how to use this tracker, I will most definitely need real documentation to help me get to the next level.

- Keep up the good work

Thanks! And yep that was the idea. The documentation is thin on how to use it. I haven't added much in terms of help info to the tracker software itself. I probably will but may save that for last (I might burn a page of hi-RAM to store that for example). And if not or in addition I'll have, at the very least, a text manual but if there's enough interest to justify it, may setup a website for the project and put contextual help or a wiki there. Until then I expect I'll keep showing off features via videos so folks can learn by example as it were.

Once I start to get to effects, documentation will be a must. I already have an effect list of what I was planning, but it may get re-arranged. I have thought about allowing A-Z for the effect selector column and would need the resulting effects to be ordered in the right way to make the text conversion simpler (and smaller) in the code. For instance A, which is $01 in screencode values, would be for the arpeggiator effect so internally that would be effect 01, S which I'll probably use for global song speed, would be internally be effect $13 as that is the screencode for 'S'. stuff like that. For Macros I'll likely use M# - unless I think I need to support more than 16 macros. Or I'll say to hell with all that and just use raw hexedecimal values for effects 🙂 In both cases, having documentation for what does what is quite necessary.

  • Thanks 1
Link to comment
Share on other sites

Sure that seems like a reasonable feature request! I've logged that here. Tab, Shift-Tab, Pg-Up, and Pg-Down help A TON so hopefully we see at least support for Page Up/Down on the X16 (given the keys exist on both keyboards, I have to imagine that is indeed planned). Likewise copy/paste is a pretty huge feature. You may have noticed in the video I wanted to grab copies of things so I could more easily add variations. Also being able to insert/delete pattern rows (where the rest of the pattern moves up and down) is nice to have.

The bigger issue I see is how to navigate around say from channel $00 (PSG) to say channel $12 (YM). If composing a song that uses a healthy mix of both, that's a lot of scrolling through patterns. I can alleviate some of that by remembering where the cursor was when leaving and returning to the pattern edit screen but I think I'll also need some sort of hotkey to jump between PSG, VERA, and (if supported), PCM. Likewise being able to wrap the patter would be pretty neat (so moving left from Channel $00 wraps over to PCM and then back through the YM channels). Much like I have it working for the order list and moving up and down in the pattern.

  • Like 1
Link to comment
Share on other sites

  • 2 weeks later...

Quick update, I live in Texas and while we are thankful we didn't have any pipes burst, we had rolling blackouts and are still boiling water. Power is now stabilized so I can start working on Command Tracker again soon. The last thing I added was a save routine, which appears to do the job but is super slow since I was grabbing every single pattern in banked memory, regardless of whether it is actually being used. It took over a minute in the emulator 🙂 So that won't do.

Instead, I'm going to have to write a sorting algorithm (which also removes duplicates) so I only have to grab the patterns that are actually being used (namely those which are defined in the order list). Having a sorting routine is something I'd love to see in ROM but for now I'm just going back to my college days and remembering about all the sorting algos (given I don't need speed but want compact code, I'll likely just do a bubble sort).

This should make saving much faster for most songs. The main speed loss is likely just the size of the patterns and the speed of calling CHROUT TONS of times. My initial goal is getting a usable tracker (one that can, you know, be used to make, save, and load music 😉 ) so I just need it to be good enough for now. The sort should considerably help but a post 1.0 feature is to implement the sparse format, at least on disk. That same format wouldn't be too far off of an in-memory format for playing music outside of the tracker (e.g. in a game just using the tracker engine).

Anyways saving and loading are ultimately pretty straightforward in how it will all work, but you might not see as regular an update as in the past while I work on it, since these aren't as flashy as the GUI and sound elements. Having saving and loading will be real useful while I start to work on the synth engine (likely using Concerto) and integrating things like instruments and effects.

If anyone else is in Texas, I hope you're all doing well! I know 8-Bit Guy has been having some issues given his latest video and I know quite a few folks that have had burst pipes and things as well 😕

  • Like 2
Link to comment
Share on other sites

Another quick update. Tried to throw a video together but eh, there isn't that much to see 🙂 The save routine is now built into the UI and was able to make saving a bit faster using ROM bank 0. I still need to do a load routine and then probably redesign the load/save section since it's pretty messy. Then I may want/need to look at jump tables for the global keys (basically the F-keys) as I'm starting to duplicate more code than is ideal I think.

The save routine does indeed use a simple bubble sort so instead of saving all patterns (essentially most of hiram and regardless of if they're used or not), it only grabs the patterns actually in the order list. As a result it's much faster! Still slower than a proper sparse format, but as noted, that's going to be a ways yet. Loading and saving will allow me to much more easily test bugs and things so that's the real reason I'm doing it now.

And after the above work is done, then I'm hoping to finally start to see how I might integrate the Concerto engine as well as if I might want to consider multi-voice instruments and reduce the raw channel count in the pattern data. Jury is still out on that one.

Sorry for no fancy video! I didn't get to spend quite as much time working on it this week as I wanted, so the progress has been important, but small.

  • Like 2
Link to comment
Share on other sites

Whew! Finally, loading and saving now works!

I have no doubt there's bugs all over the place. And of note I'm not sanitizing inputs in all cases, etc., etc. but basic loading and saving does work, which will make it much easier for me to start working some of the important core features finally!

  • Like 4
Link to comment
Share on other sites

  • 2 weeks later...

I haz no video today alas as I haven't done quite enough to make it show-worthy yet, but I have managed to interface with @kliepatsch's Concerto! Basically channels 0-15 are now being handled by Concerto and playing a song plays through Concerto. While instruments are working (I think), I haven't configured anything but the default so I haven't tested that yet. Volume doesn't work. Effects don't work 🙂 But note-release works! That one is currently mapped to noteoff and noterel as I was having some issues with note-off and I'm not quite sure why.

My Bad Apple demotune plays well enough - I still have the rendering pause bug and since I haven't setup any instruments the drums don't sound so amazing 😛 So that's partly why I also have no video. Once I get some basic instruments setup to play with I'll likely show off a bit more, as well as test more things.

Though there is no video, the codebase has been updated - currently the Concerto integration lives in a branch:

https://gitlab.com/m00dawg/command-tracker/-/tree/concerto-integration

The branch also fixes some core tracker bugs as well but for now I was keeping the mainline as-is since it works and I have more work to do with Concerto before I merge it all in.

 

  • Like 2
Link to comment
Share on other sites

Up until now, it's all the PSG (programmable sound generators) in the VERA. No samples so far.

CPU, wise it depends really on what you are doing. If you are using the tracker with the UI, playback will use significantly more CPU power than if you are "just" playing back without any visual feedback. And how much CPU is taken up by playback itself also depends on how much of the sound engine's capabilities you are using. That includes, how many voices are playing and especially if the sounds are using a lot of modulation and effects.

In other words, it should be possible to compose songs that are pretty lightweight, e.g. with 3 or 4 voices and not too much modulation going on. The 8 MHz of the X16 help quite a lot with this kind of stuff.

 

Edit: I should mention that at the moment, as far as I know, the tracker exists only in conjunction with the UI. But maybe later a player-only version could be made that could be used inside other applications.

Edited by kliepatsch
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

×
×
  • Create New...

Important Information

Please review our Terms of Use