Jump to content
kliepatsch

Tracker vs. piano roll (How to compose music on the X16?)

Recommended Posts

Hi everyone,

I am currently working on a sound engine utilizing the 16 voice Programmable Sound Generator (PSG). Later I would like to add a way to compose music with it. I have thought a lot on what is the best format for such a program. Mainly, I can't decide between the piano roll vs tracker approaches, because they both have their strengths.

 

Tracker

Example:

Basically, you have a table with all the "tracks" in the columns, and every row represents either a 16th or a 32th note. When playing back the song, the player moves from row to row and executes all the commands it finds in each row. Often, the tracks represent the channels of the hardware (e.g. the original Amiga trackers had 4 tracks for the 4 hardware channels. AdLib trackers have 9 channels for the 9 FM voices etc.), but it doesn't necessarily have to be that way.

Each cell can contain commands like play a note of instrument X at pitch Y and volume Z, or stop the note currently playing in this channel.

Additionally, each cell can contain an additional command, called "effect", which is used to control volume slides, pitch slides, vibrato, tremolo and various other things.

In a track, there can only be one note playing at a time.

Pros:

  • Nice direct control over every voice, even if several voices of the same instrument are playing (like 3 violinists, but every one can have individual vibrato)
  • More hardware-friendly and easier to implement.
  • Everything that is happening in the song at a given time is there in a single view.

 

 

Piano Roll

Examples:

https://youtu.be/AfDuZ2vHkUQ?t=384

https://www.youtube.com/watch?v=_8PbvMxkIhg

Here, there is also a table. This time, time increases from left to right. And pitch increases from bottom to top. Notes are displayed as bars stretching horizontally from note-on to note-off.

Additional control like pitch bending, volume control etc. need to be done outside of this view. Often, there is an additional area where one can edit the note's velocities and other things.

Pros:

  • The ability to "see" the pitch, and to see chords.
  • The ability to adjust the timing very precisely (as opposed to a very rigid grid in the trackers).
  • The ability to drag and drop (although this would be doable in a tracker, too)

 

I was thinking of maybe combining the pros of those two options. I can live without the visual representation of pitch, but I am not willing to sacrifice freedom of timing. And I really want the freedom to modulate individual voices, because that is one of the few ways you can give expressiveness to chiptune.

So I could try to find a way to make the tracker approach more flexible with timing.

I am thinking out loud. Anyway, your thoughts, ideas, experiences etc. are welcome 🙂

Have a nice day!

  • Like 2

Share this post


Link to post
Share on other sites

To add more thoughts, I see it as two separate conversations. The preference of the musician for how (s)he likes to compose; and the capabilities of the hardware.

I find trackers make it much easier to exploit the hardware to the fullest ability and presents a lot of information in a nice and compact but easy to manipulate way, although it's less real-time for sure. I was unaware that the PSG would have 16 channels. Last I heard it was still the YM2151 + SAA1099 + DAC solution. But whatever the audio solution, I would guess the channel counts would be fixed and that, to me, tends to make a nod to a tracker interface. To me the piano roll makes more sense in modern DAWs where there are fewer limitations and you tend to manipulate synths as single entities versus manipulating individual channels like you tend to in a tracker. Consider the case where you might play a chord. On a piano roll, you put in the cord in on "track" but on a tracker, you would use 3 channels (ignoring some of the meta channel stuff like what Renoise has, given it's a fully modern tracker).

A standard chord is 3 channels but if the chords are more staccato, I could interleave other sounds in between the chords using the same channels. Doing that via a piano roll style interface seems really complicated, or I've never been able to do that level of precision on any old or new tool. The flip-side for most non-tracker musicians, a piano roll (perhaps in addition to MIDI support for realtime playing) would be a more conventional solution.

I did some of my own thought experiments on a tracker UI and a tracker format (for the FM/PSG/DAC solution) which you might be able to perhaps use, though it's not very far along: https://gitlab.com/m00dawg/commander-x16-programs/-/tree/master/command_tracker

Related aside, I'm looking forward to seeing what folks come up with for a MIDI I/O card. For me it'd mostly be for clock-sync such that, with support from a tracker or song player, I could sync up the x16 for recording should I want to combine it with other external instruments (like my Gameboy, NES, MidiBox gear, etc.).

All told, as the developer, you get to decide on the vision, so don't take my opinions as gospel or anything. They're just my opinions and they weigh about as much as Helium 😛

  • Like 1

Share this post


Link to post
Share on other sites

Thanks for your opinions! Yes, these are some additional thoughts worth considering.

I hope I will also take the courage to look at what you have done. Most of the time I'm glad when I don't need to understand someone else's work, but in this case it might be worth it 😉

I can see all the pros for a tracker. Thing is, I never used tracker up until one week ago. I spent my whole life in a piano roll, so using trackers is a big step out of my comfort zone. But finding that "Rolling town the street" song in my post at the top immediately sparked the belief in me that indeed trackers can be of good use. I'm glad, because without it I were probably stuck in cloning whatever MSSIAH seemed to have done. Now that I tried out a tracker myself, I can see all the benefits of that layout, even though it's still very awkward for me to use one.

Since the sound engine adds another layer of abstraction between the hardware and the music, the exploitation of the hardware argument is not as valid as for e.g. Amiga trackers. There are sounds that use multiple PSG voices, so the 1-on-1 equivalence of Tracks and hardware channels is broken.

Share this post


Link to post
Share on other sites

As a musician, I much prefer piano roll style editing, but there are times an event viewer is useful. I do NOT like trackers, as they tend to limit what you can do musically, often being at fixed intervals. 

My favorite program ever for composing music is the old Cakewalk for DOS, and I'd be thrilled to have something similar on the Commander

http://www.vgmpf.com/Wiki/index.php?title=Cakewalk#:~:text=Cakewalk is a MIDI sequencer,being re-branded as Sonar.

 

Edited by TomXP411
  • Like 1

Share this post


Link to post
Share on other sites

I'm with several other commenters here in that the tracker format tends to lock one in to square rhythms. 

One thing I very much value about piano roll views is their scalability: it's nice to be able to zoom in and out, both vertically and horizontally.  Not sure how difficult arbitrary zoom will be to implement on the X16.  The tracker interface generally doesn't need scalability across its tracks (that's horizontal in a tracker) but if someone could develop a tracker that's zoomable in the rhythm direction (vertical for trackers), and thus allowed for fine rhythmic granularity so you're not locked into a 16th grid, such a design might be the best of both worlds.

I mean, right now some trackers have ugly solutions to the problem of getting out of the grid like a 'delay' property; you can enter a note event on what visually  looks like beat 1 but also add a delay property that pushes is back to the middle note of a triplet.  This is awkward to use and not a great solution.

So yeah, as you design your interface, be it tracker, piano roll, or a hybrid, definately think about the ability to zoom in and out.

On the other hand... the gridlike nature of the tracker interface may feel limiting, but it is also a sensible solution for a computer with keyboard-input and character-grid graphics (as many 8bit computers had).  A zoomable piano roll system may seem nice, but may also be very awkward on a system without a mouse.

 

On 12/16/2020 at 6:33 PM, TomXP411 said:

My favorite program ever for composing music is the old Cakewalk for DOS, and I'd be thrilled to have something similar on the Commander

Me too! The flexibility of the old Cakewalk was just great.  I actually first learned to compose music as a kid by trying things out in Cakewalk Home Studio for Win 3.1 - I didn't know there was a DOS version until you mentioned it just now.  

It used to be, Cakewalk was the software, made by a company called Twelve Tone Systems.  Then for a while, the Company name became Cakewalk and the software was called Sonar (as your article covers).  About two years ago, the IP was bought by a startup called BandLab and now the software is called Cakewalk again.  This modern version is my main DAW.  Elements from the Win 3.1 version that I learned on, which were excellent interface decisions, are still present in the most recent version. 

(Their "Staff View" in particular is well designed: it allows people who are used to reading staff notation to do so in a MIDI sequencer, but it understands that its goal is sequencing and NOT notation; this has always made it better to compose in, in my opionion, than the dedicated notation programs like Finale or Sibelius.  Too bad staff views are such a bear to program, or I'd recommend that to @kliepatsch)

On 12/16/2020 at 11:21 AM, m00dawg said:

... I want to combine it with other external instruments (like my Gameboy, NES, MidiBox gear, etc.).

Not to get off track in the thread, but what strategy/hardware are you using to use the NES as an instrument?  I've been looking for a good solution for that. 

Share this post


Link to post
Share on other sites

Hey thanks for all the input! I will take it into account when thinking about what I will do. Of course depends also on how far I will get with the project overall.

Maybe this thread will also be useful for other projects, such as TeamX16, who intended to make music software, too. (But they may have already settled upon a tracker ...)

I see that a piano roll is more popular overall, which I can understand. However, I am scared about the fact that you need more infrastructure around a piano roll than for a tracker. A tracker is song structure and composing of all the parts, in a single view. For a piano roll, you cannot have the whole song in it. Also, the capabilities of a piano roll for editing modulations are near zero, which makes even more infrastructure necessary.

I like the idea of making a tracker with more flexible timing than usual.

A piano roll could still be made available by selecting one or more tracks, which makes the respective notes appear in a piano roll. So there could still be chords, but never more notes at a time than there are tracks selcted. And all the modulation could be done in the tracker. And to make the editing easier, some sort of cross-highlighting would be ideal. Hovering over a note in the piano roll could highlight the corresponding note in the tracker view and vice versa, so one would have to think less while fine tuning the notes.
This would currently be my dream 😄

@TomXP411 pretty cool, I didn't know that Cakewalk has been around so early!

@John Chow Seymour thanks also for your thoughts. Yes, Zoom will be nice. I have to think about that. Mouse will be a requirement.

@Perifractic ha, I guessed so 😉 I didn't know you are a musician before you uploaded that video, though.

  • Like 1

Share this post


Link to post
Share on other sites
15 hours ago, kliepatsch said:

Also, the capabilities of a piano roll for editing modulations are near zero, which makes even more infrastructure necessary.

You're just not being creative enough. 🙂

All of the DAWs I've used recently have a split view mode on the piano roll. The top half of the window shows the note, while the bottom half shows one or more controllers. 

I'll work up a quick mockup of what I mean and post it in a few minutes.

image.thumb.png.be4f7e978a98c8a882263231cdf7b987.png

So here's what I mean. Note events are drawn on the piano roll portion at the top and the numbers below are control values applied to each note.

Look at something like SID Wizard, with 5 data elements per note, you could stack those data elements below the roll. With 60 rows on the screen, you can afford to have 20 rows dedicated to note data, while still having a linear track on the top... and you can offer different modes, with two or three levels of detail on the data elements part of the screen. 

For detailed editing, you could color code the channels (White=1, Yellow=2, etc), then show just one channel at the bottom and let the user use the TAB key to cycle through the channels.

 

Edited by TomXP411
  • Like 3

Share this post


Link to post
Share on other sites

Come to think of it, tracker music is basically just a list of instructions for a sound playback Virtual Machine. The playback overhead for this is minimal. I don't know if the same is true for other types of music representation

  • Like 1

Share this post


Link to post
Share on other sites
6 minutes ago, desertfish said:

Come to think of it, tracker music is basically just a list of instructions for a sound playback Virtual Machine. The playback overhead for this is minimal. I don't know if the same is true for other types of music representation

The only difference between piano roll, tracker view, and an event list view is the way the data is presented. 

On a piano roll, you are simply drawing a graph where X=Time and Y=Note. Other information, like velocity (volume), modulation, pitch bend, and other parameters are usually shown as a segment or line graph below the roll. 

On a tracker view, you are listing the events on a grid where Y=Time and X=Channel or effect type.

No matter the display format, the underlying data structure is the same. Each event consists of a time index, an event type (note on, note off, patch change, controller change), and a data value (note number, controller value, patch number)

The rendering time is going to depend more on the size of the display than on the presentation method.  

  • Like 2

Share this post


Link to post
Share on other sites
1 hour ago, TomXP411 said:

The only difference between piano roll, tracker view, and an event list view is the way the data is presented. 

On a piano roll, you are simply drawing a graph where X=Time and Y=Note. Other information, like velocity (volume), modulation, pitch bend, and other parameters are usually shown as a segment or line graph below the roll. 

On a tracker view, you are listing the events on a grid where Y=Time and X=Channel or effect type.

No matter the display format, the underlying data structure is the same. Each event consists of a time index, an event type (note on, note off, patch change, controller change), and a data value (note number, controller value, patch number)

The rendering time is going to depend more on the size of the display than on the presentation method.  

It's a bit more complicated than that though. If using a piano roll, it's expected you can do chords and to me the fundamental method of handing what music is by instrument vs by channel. The software could do some trickery to be able to cleverly reuse channels I'd imagine, but that would be handled more by the software. The trackers, by contrast, hands that over to the musician.

So elaborate on my previous example, say I have something like a SID chip that only has 3 voices. But I want to do more than just chords, so I would interleave other sounds in between when the chords aren't playing. So channel1 might play the base note, but then also play a bass drum. Channel2 the middle note, but then might interleave a melody in between. Channel3 might be doing some rhythmic stabs.

In a tracker, that's a frequent tool to add more apparent channels than there are but you can also more precisely control effects while keeping channel usage in check. When I have more channels, I tend to use fewer tricks of course, but it's an interesting way to think about it and sometimes I wish I did this more with modern music. For instance, if I'm using my Deepmind, I might want to map voices to multiple patches and change them around. I *CAN* do that already but in a modern DAW it's a lot more work than just changing an instrument number in the pattern data within a tracker and I often don't think in this way. Instead I'll record the synth to audio then add the other part in after, which kinda also breaks the creative flow.

Likewise I have a MidiBox SID (actually a monster MB6582 I just finished) which has EIGHT SIDs in it. You'd think that would allow me to create some insane things, but interfacing with the chips via MIDI and a DAW, while I can make some incredibly complicated individual patches, is overall a bit more limiting than if I were to more precisely control each SID voice directly. I can do that, but it's real complicated to do in a DAW, but would be trivial in a tracker.

I'm not trying to speak ill of piano rolls mind you - actually I'm fond of them as well and I think they are much more approachable than a tracker interface, which can be quite intimidating at first. I just wanted to elaborate a bit on the power of trackers which shouldn't be overlooked.

Now ALL that said (and it was a lot, sorry!) - with a rich sound implementation like the YM2151, there's enough channels where folks can write lovely music without resorting to full on tracker trickery to the point that a more musical interface could make sense. Likewise the chip is kinda built around the concept of "instruments" which lends itself more favorably to composing music more with that frame of mind. Tracker-music often sounds different than erhm not-tracker music for lack of a better term, and I think there's plenty of opportunity for both approaches to eventually exist on the x16. I'm eager to see what the final sound design is though!

  • Like 1

Share this post


Link to post
Share on other sites

Good stuff everyone! 🙂

Yes, I am currently thinking it from the data structure side. Yesterday, I implemented a rudimentary player that works quite well so far. (Actually just to be able to test the sound engine better, which is still a WIP)

The format is dead simple: Everything is an event. Note-on, Note-off and wait are currently the only events available, but all modulation stuff will be events, too (defining points and/or slides, rather than only points as in MIDI modulation).

The player processes one event after another. It keeps processing events until a wait event is encountered. Then it waits for the specified amount of ticks until the next event. (A tick currently being something like 7 milliseconds)

The computational effort for this format while playing back is minimal. But the format will be very annoying to write an editing program for. Imagine just how much data you need to move around to insert a single note. (And the effort it takes to determine the correct position, maybe splitting existing wait events in two etc.) But I think that is just how it is. I like the simplicity and flexibility of the format.

The task is now to think of ways to present that data. Thanks, @TomXP411 for your sketch. It's really helpful to be able to see it! How did you make it? I'd like to be able to make such sketches myself. And your idea about how editing might work is definitely worth considering. (BTW that extra area at the bottom for editing velocities etc. is what I meant by "more infrastructure necessary" 😉 )

And I am also with @m00dawg. I totally want those kind of tricks to be possible.
BTW I think I am decided to make the whole program "channel" based. That means that we will have 16 channels, where each channel can play a single monophonic note. That makes note addressing (for both note-offs and modulations) waaayy easier than a polyphonic track layout. Also, instruments will be separate from channels, so that every channel can play all instruments, so that those interleaving tricks are possible.

However, that doesn't mean that it will have a tracker layout. In fact, the discrete timing of a tracker might be very impractical for a tick-based timing of al events, and I start to like TomXP411's idea.

  • Like 2

Share this post


Link to post
Share on other sites
19 hours ago, m00dawg said:

If using a piano roll, it's expected you can do chords and to me the fundamental method of handing what music is by instrument vs by channel. The software could do some trickery to be able to cleverly reuse channels I'd imagine, but that would be handled more by the software. The trackers, by contrast, hands that over to the musician.

Yeah, I strongly agree with all the points in this post (and not just the part I quoted).  With later, fully developed MIDI systems which can have polyphony on a single channel, the piano roll does make more sense. But we're not looking at that here; and as @m00dawg describes, trackers let you more directly control what goes to which channel of which chip.

You could still make it work on a piano roll; if you understood it would show one monophonic channel at a time, and there were an easy way to switch between which channel is being displayed.

On 12/18/2020 at 6:04 AM, kliepatsch said:

Yes, Zoom will be nice. I have to think about that. Mouse will be a requirement.

Is anyone even working on a mouse for the X16?  Hey, we get SNES controller ports, right?  Maybe you can make your music app compatible with the SNES mouse. (This is a joke, please don't actually do that: SNES mice are hard to find and awful to use.)  I guess there's always the strategy where the D-pad (or the arrow keys for that matter) move the cursor like a mouse, instead of moving from one discrete screen character to the next.

As much as I dislike the tracker-style interface, as much as I would rather see @kliepatsch make a piano roll, the more I think about it the more the tracker is starting to make sense.  If you can find a good way to break the tracker out of its tendency toward (at least visually) locking the user in to a rhythmic grid, then maybe that's the best way after all. 

In any case, I wouldn't really get started until we know which sound chips we're getting.  The FAQ is still vague about it.  I mean, you couldn't even plan how many columns a tracker would have, or how many data points you'll need under the piano roll, until we know which chips we're even programming for.

EDIT:

@TomXP411's sketch is really good, I like the look of it already.  I think you'll want to sacrifice 1-2 more rows in order to add a label to the X-axis (that'll be especially important if you manage to implement zoom).

Edited by John Chow Seymour
Thought of one last thing

Share this post


Link to post
Share on other sites
On 12/20/2020 at 12:28 AM, John Chow Seymour said:

In any case, I wouldn't really get started until we know which sound chips we're getting.  The FAQ is still vague about it.  I mean, you couldn't even plan how many columns a tracker would have, or how many data points you'll need under the piano roll, until we know which chips we're even programming for.

Don't worry. I am doing this mainly for fun, and for that, the emulator is equally as good as the real hardware for me 😉

If the PSG is cancelled, well, then my whole sound engine can't be used on the real hardware. But the concepts of song composing can still be applied to whatever sound system will be found in the real thing.

  • Like 1

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...

Important Information

Please review our Terms of Use