Jump to content
rje

IEC Mass-Storage device

Recommended Posts

Posted (edited)

Since the X16 is not a C64, an IEC mass-storage device need not be cycle-exact: in short, it can patch/monitor the I/O on the device side to deliver data with less fuss than things which require really painful cycle-exactness due to C64 disk copy protection schemes.

Right?

Or perhaps I'm wrong... the X16 is a physical device speaking IEC with the same connector as the later machines...

Or, perhaps it depends on the device being emulated. 

* * *

Maybe I'm overthinking this, and I should just expect to use Pi1541.

Edited by rje

Share this post


Link to post
Share on other sites
Posted (edited)

Yes, you're overthinking it.  Use an SD2IEC (not a Pi 1541). 

Since you don't need cycle exact timing or copy protection, the SD2IEC is actually faster and simpler to use. It also has the advantage of being a real mass storage device (not a floppy emulator) and handling 1541, 1571, and 1581 images. 

 

Edited by TomXP411
  • Like 3

Share this post


Link to post
Share on other sites
Posted (edited)
16 hours ago, rje said:

Since the X16 is not a C64, an IEC mass-storage device need not be cycle-exact: in short, it can patch/monitor the I/O on the device side to deliver data with less fuss than things which require really painful cycle-exactness due to C64 disk copy protection schemes.

Right?

Right!

The only reason that there is a benefit for being a cycle exact replica of a 1541 is because of C64 disk based games that run copy protection that requires the disk to be accessed by a 1541.

I believe a version of Pi1541 can be developed for the CX16 which will be much faster, due to not bothering with emulating the 1541. Indeed, even cheaper, it may be that something could be developed to run on a RPi Pico which could be used with a USB Flash Drive (since the RPI Pico can host USB On The Go).

Edited by BruceMcF

Share this post


Link to post
Share on other sites
Posted (edited)
10 hours ago, BruceMcF said:

I believe a version of Pi1541 can be developed for the CX16 which will be much faster, due to not bothering with emulating the 1541.

I was reading the designer's notes, and I came to the same conclusion:

Quote

Currently when Pi1541 boots it will be in browser mode (or SD2IEC mode)
In this mode Pi1541 will support very minimal SD2IEC commands. Basicaly you can load PRG files and browse folders and images. (Multiple channels and Saving are also supported for sequentual files only)
Once a valid disk image or images have been selected Pi1541 will drop down into emulation mode.
In full emulation mode reading and writing are supported.

If r/w support is added at the SD2IEC level, then running in "SD2IEC mode" would be fine on a Pi Zero without overclocking, I suspect.

Edited by rje
  • Like 1

Share this post


Link to post
Share on other sites
Posted (edited)

Would it really be faster?  Isn't throughput limited by the IEC-Serial bus itself?

Your answer might be "it depends on the device being emulated", which is true and opens up another set of questions.

 

So maybe the correct next question is "which IEC device should be handled in non-cycle-exact form?".   I see no reason to emulate a D64 when the D81 is superior in every way, for example.

Edited by rje

Share this post


Link to post
Share on other sites
18 hours ago, rje said:

Would it really be faster?  Isn't throughput limited by the IEC-Serial bus itself?

Your answer might be "it depends on the device being emulated", which is true and opens up another set of questions.

 

So maybe the correct next question is "which IEC device should be handled in non-cycle-exact form?".   I see no reason to emulate a D64 when the D81 is superior in every way, for example.

It might not be massively faster, but it will be faster. EG, reading the directory takes more time if emulating a disk spinning in a 1541.

As far as a flash USB drive reader, an ability to work with .d64 files, .d81 files, and to treat the mass storage hierarchy or a part of it anchored on a main root directory as a CMD hard drive would seem to cover it.

A lot of microcontroller based solutions would have the GPIO to be able to work with either the IEC port or parallel access over the User Port, where the IEC would provide plug and play convenience while the device could also ship with the program and device driver files to support faster access on the User Port.

Share this post


Link to post
Share on other sites

Regardless of the speed increase, my bet is that it would take the load off of the Pi, allowing a Pi Zero to run without overclocking.

 

Share this post


Link to post
Share on other sites
Posted (edited)

I'm still a bit confused, though... what's the point of all this? To make a faster SD2IEC? 

The SD card slot in the Commander is faster than an IEC drive can hope to be, since the IEC drive will always have to use software bit banging, and the SD card slot can deliver data to the computer in parallel. So the SD slot is already 8x faster than any IEC drive can ever hope to be. 

(see below) You can get up to around 64KB/sec using IEC, but that seems like a lot of work for something few people will need or use. 

Edited by TomXP411

Share this post


Link to post
Share on other sites
Posted (edited)
3 hours ago, BruceMcF said:

It might not be massively faster, but it will be faster. EG, reading the directory takes more time if emulating a disk spinning in a 1541.

The SD2IEC running JiffyDOS is roughly 2x as fast as a 1541 running JiffyDOS. As far as I have been able to test, the SD2IEC is the fastest serial storage device available.  So yeah, there can be a pretty big speed boost from not dealing with floppy media. 

In fact, based on the 300 RPM rotation speed, the 1541 floppy drive is capable of a maximum of about 26KB/s, but the 10:1 interleave drastically reduces the speed, allowing only two sectors to be read per rotation. If my math is right, this limits 1541 media to around 2.4K per second. Also, if memory serves, this is about what I see when using JiffyDOS. Interestingly, even when using the standard Commodore protocol, the SD2IEC is slightly faster, at 650 bytes/sec, compared to 400 bytes/sec from the 1541. 

On the SD2IEC, this is more like 8KB/sec, and this cap is due to the 1MHz speed of the CPU. 

So using 8KB/sec as our baseline, we can expect 64KB/sec on the Commander X16 using the JiffyDOS protocol. 

The good news there is that most of this work has already been done. There's already an Arduino version of SD2IEC (which is largely compatible with CMD hard drive commands). So it's just a matter of adjusting the SD2IEC code to transfer at 8MHz, instead of 1MHz, and write or adapt any of the existing compatible fastloaders. 

Again, I'm not sure why @rje seems to be stuck on using the Pi for this... it's not the ideal solution. An Arduino type device is more appropriate, as most of the features of the Raspberry are simply not needed, and Arduino software is simpler to work with than bare metal Raspberry Pi code. 

Edited by TomXP411

Share this post


Link to post
Share on other sites
Posted (edited)

Here's an ATMega solution: https://github.com/fachat/XD2031

Or, perhaps what Tom was alluding to: https://github.com/Larswad/sd2iec_mega2560

I have a Mega 2560 and a Nano as well, so that's fine, they're just not as familiar to me.

 

The Arduino IS probably better; for one thing, the voltage levels are more compatible.

 

There's also https://www.insentricity.com/a.cl/208/C64PiVideo.

Oh, I'm stuck on the Pi because 

(1) I "know" it better than other little platforms and
(2) I have some lying around but also
(3) The Pi Zero is just so cheap and
(4) I get stuck on things.

 


 

Edited by rje

Share this post


Link to post
Share on other sites
Posted (edited)
53 minutes ago, rje said:

Here's an ATMega solution: https://github.com/fachat/XD2031

Or, perhaps what Tom was alluding to: https://github.com/Larswad/sd2iec_mega2560

I have a Mega 2560 and a Nano as well, so that's fine, they're just not as familiar to me.

 

The Arduino IS probably better; for one thing, the voltage levels are more compatible.

 

There's also https://www.insentricity.com/a.cl/208/C64PiVideo.

Oh, I'm stuck on the Pi because 

(1) I "know" it better than other little platforms and
(2) I have some lying around but also
(3) The Pi Zero is just so cheap and
(4) I get stuck on things.

Yes, I was thinking of the sd2iec_mega2560 version - although I would not implement it on a Mega. I would use a Teensy 3.5, due to the built in SD reader and smaller footprint. 

Actually, you should be able to do it on a Pi, using a bare metal coding solution like Circle. Just don't expect to do this in Raspbian. Pi 1541 was developed as a bare metal program specifically because Linux doesn't have the timing resolution needed to work in this situation.

Most of the code should also work, although you may need to deal with the timer - I don't know what the Pi uses for a timer, but you need some pretty tight precision if you're going to use a serial connection. A parallel connection would actually be simpler to code,  since you just need to set the data pins and strobe the clock pin. 

I also have RunCPM running on a Grand Central - which would actually be ideal. It has a lot of pins, so you can stick it on the User port and also use it as a serial and WiFI breakout, with the addition of an ESP8266 or ESP32. 

 

Edited by TomXP411

Share this post


Link to post
Share on other sites

 

7 hours ago, TomXP411 said:

what's the point of all this? To make a faster SD2IEC? 

The SD card slot in the Commander is faster than an IEC drive can hope to be [...]

(...) You can get up to around 64KB/sec using IEC, but that seems like a lot of work for something few people will need or use. 

My use case is: complex programs.

I'll define "complex programs" as ones which use files as resources, loaded on demand.

 

Perhaps that's what the SD card is for, but I am not sure about that.  For one thing, you don't want to swap an SD card out like you would a floppy diskette; it's not intended for that kind of use -- although perhaps it's OK.  But if that puts an unreasonable amount of wear-and-tear on the connector, then that could be a problem.  I've tried googling for the MTBF on SD card connectors, but obviously I don't know what I'm looking for.

Now if the SD card is the target, then programs will be written with that device driver.  Is that what we want?  Does that cause a portability issue?  Maybe not -- I mean large programs which load in data files are by definition not very portable.  So strike portability.

If the IEC bus is the target, then programs will use the IEC protocol.  Maybe that doesn't matter (portability is not the issue).

 

I was also thinking about an RS-232 port; I was thinking how interesting it would be to write a shell-like transfer program for it, and that's when I realized that complex programs will never be written for that protocol on the X16, exactly because it's not a supported standard, and "no standard" will never be able to compete with two existing-and-supported standards.

Share this post


Link to post
Share on other sites
6 hours ago, rje said:

 

My use case is: complex programs.

I'll define "complex programs" as ones which use files as resources, loaded on demand.

 

Perhaps that's what the SD card is for, but I am not sure about that.  For one thing, you don't want to swap an SD card out like you would a floppy diskette; it's not intended for that kind of use -- although perhaps it's OK.  But if that puts an unreasonable amount of wear-and-tear on the connector, then that could be a problem.  I've tried googling for the MTBF on SD card connectors, but obviously I don't know what I'm looking for.

Now if the SD card is the target, then programs will be written with that device driver.  Is that what we want?  Does that cause a portability issue?  Maybe not -- I mean large programs which load in data files are by definition not very portable.  So strike portability.

If the IEC bus is the target, then programs will use the IEC protocol.  Maybe that doesn't matter (portability is not the issue).

 

I was also thinking about an RS-232 port; I was thinking how interesting it would be to write a shell-like transfer program for it, and that's when I realized that complex programs will never be written for that protocol on the X16, exactly because it's not a supported standard, and "no standard" will never be able to compete with two existing-and-supported standards.

Actually,  the SD card is made for exactly that sort of use. Remember that the SD card was originally made for digital cameras and MP3 players, and the intent was for users to pull the card out of their device and plug it into a computer to transfer data. 

However... it does seem sort of silly to sneakernet the card to a PC when the X16 should come with some sort of network device built in. We've all been saying it since the beginning, and it's still true: The Commander should have an Ethernet port, WiFi interface, or at least a hardware driven serial port. 

Anyway... the IEC port is intended to support Commodore peripherals. That means a 1MHz clock speed. THAT means 600 characters per second. So you won't be using that for complex programs using the default on-board software. 

 

Share this post


Link to post
Share on other sites
Posted (edited)

Yes, yes, and yes.  I don't disagree on any particular point up there.

I wonder what Dave's been thinking re I/O. 

With the IEC port at crawly 1541 speeds, there may be an effort (post release?) for fastloader support, or a KERNAL update to implement the 1581-era burst mode (1500 characters per second, perhaps).

 

 

Edited by rje

Share this post


Link to post
Share on other sites

Let me see if I can restate the premise.

You want to be able to transfer information to and from this machine, but you're worried that repeated insert/eject cycles are going to wear out the SD card slot, plus the slot is inconveniently placed. You would use something like an SD2IEC but the IEC bus is too slow for the kind of file quantities that the SD card makes possible and that the 2MB version of the machine makes useful. You also don't think connectivity via serial would be a popular option, given the existence of the SD and IEC standards on the system.

Do I have that about right?

Share this post


Link to post
Share on other sites
Posted (edited)

The thing about load time for a "complex program" if the SD stays in the system is that if you are loading a complex program to the SD card, then it's a one-off. I am more than a little skeptical about there being a lot of 4MB programs for the CX16, but the issue is that if, optimistically, the bit banged serial port is 19200Kbps, that's 2,400KBps, and a 4MB program is like half an hour. Even for a one time process, that's a bit more than a "make a cup of coffee" wait time. An EPP transfer is more like 24KBps (with a routine with a timeout countdown on /ACK), so a 4MB program would be more like 3 minutes.

If complex programs are more in the range of 128K-256K, then if the bit banged serial is (optimistically) 19200kbps, that's more like 1-2 minutes to load a file onto the SD card, and more like a sip of coffee wait time for an EPP transfer.

If you are going to sneakernet the SD card, an SD card extension cable totally eliminates the worry about the SD card wearing out, though I'd not so much worry about that as want to have the SD card more easily accessible.

On 3/19/2021 at 9:20 AM, rje said:

I was also thinking about an RS-232 port; I was thinking how interesting it would be to write a shell-like transfer program for it, and that's when I realized that complex programs will never be written for that protocol on the X16, exactly because it's not a supported standard, and "no standard" will never be able to compete with two existing-and-supported standards.

Except if (1) the bit-banged serial on the User port is a Kernel device, like bit-banged serial on the C64 User port, and (2) an RS-232C card has a Kernel API driver written for it that works the same way, then it would be an existing standard.

 

Edited by BruceMcF

Share this post


Link to post
Share on other sites
On 3/18/2021 at 7:20 PM, rje said:

 

My use case is: complex programs.

I'll define "complex programs" as ones which use files as resources, loaded on demand.

 

My video demo used 15947 files. Every 1/20 second, it loads a PCM audio clip, tile data, and a palette from the file system to VERA (the two extra files are a tile map and the default palette). It's all loaded on demand, triggered by VSYNC.

The whole thing is over 67 Mb. The only way to avoid sneakernet with something like that is if the SD card has an on-board wifi. Luckily such cards exist, if one wants to write the protocol. 

 

Share this post


Link to post
Share on other sites
5 hours ago, Ed Minchau said:

My video demo used 15947 files. Every 1/20 second, it loads a PCM audio clip, tile data, and a palette from the file system to VERA (the two extra files are a tile map and the default palette). It's all loaded on demand, triggered by VSYNC.

The whole thing is over 67 Mb. The only way to avoid sneakernet with something like that is if the SD card has an on-board wifi.

20-50 minutes is indeed something only a certain segment would have the patience and ability to plan ahead and do something else while it is finishing. But I do reckon that if I sneakernet everything above 10MB that I have an interested in getting loaded up into the CX16, doing that doesn't seem like it would be a very regular occurrance.

Share this post


Link to post
Share on other sites
Posted (edited)
12 hours ago, BruceMcF said:

I am more than a little skeptical about there being a lot of 4MB programs for the CX16 [...]

If you are going to sneakernet the SD card, an SD card extension cable totally eliminates the worry about the SD card wearing out, though I'd not so much worry about that as want to have the SD card more easily accessible.

Except if (1) the bit-banged serial on the User port is a Kernel device, like bit-banged serial on the C64 User port, and (2) an RS-232C card has a Kernel API driver written for it that works the same way, then it would be an existing standard.

You made three good points.

(1) I don't know about 4MB either.  However, games that use a map will seek to fill available space within reason.  I have 5mb of data I *could* put in map files.  I chose to limit it to 260K.  I might pare that down further, or go the (1a) route.

(1a) Slow performance OR extraordinary file sizes would suggest breaking a map up into multiple files, or doing seek()s on an open stream.

(2) Yes to the extension cable.

(3) Yes to a KERNAL driver for serial on the user port / RS-232C.

Edited by rje

Share this post


Link to post
Share on other sites
12 hours ago, rje said:

(1a) Slow performance OR extraordinary file sizes would suggest breaking a map up into multiple files, or doing seek()s on an open stream.

I would expect that the SD drive will be used by programs that rely on accessing files, and the SD drive will be fast enough that speed of access will rarely be the bottleneck ... the speed issue is how fast can a larger file or set of files be loaded onto an SD drive one time.

It would likely be convenient enough to give people an option on which device number they use, but if they elect to use their old 1541 on the IEC port and it makes for long pauses in game, that is really on them.

That's not where I see any advantage from an IEC mass storage device.

Where there is an advantage would be, for example, if there is a microcontroller that hosts USB being used to support a USB flash drive, or USB to WIFi bridge, or whatever ... and it talks to the CX16 over a bespoke User Port parallel port A interface, using the VIA parallel port handshake modes to transfer a page of data at time time ... IOW, you have the microcontroller, you have the read and write routines so they are fast enough to keep up with the VIA using the hardware handshake on the VIA side, you do your "RDY ACK" handshakes a page at a time and rely on the hardware handshake to just pump data through 256 bytes at a time.

That's where you get your really nice User Port data transfer rates ... READPG1: LDA portA : STA (target),Y : INY : BNE READPG1 ... as an inner loop, with the ACK / RDY overhead happening once every 256 bytes, so you might hope for around 16-20 clocks per byte, which is getting up into the 400KB/s-500KB/s, or 3 minutes or less for that 67MB demo.

Now, while I can see EPP parallel port routines getting more than "single project lifetime" support, because of the retro appeal of accessing existing old dusty PC parallel port devices and bringing them up on the CX16, something like this benefits a lot from being an all-in-one solution.

So also having an IEC port to plug into the CX16 IEC plug (or terminal of the IEC daisychain) as "Device #14" would allow the microcontroller to store the utilities and/or drivers to install the support for User port parallel port access to the board and serve them over the IEC port ... making it LOAD"*",14 plug and play.

  • Like 1

Share this post


Link to post
Share on other sites

+1 on overloading the IEC port with a new, magical Device 14.

 

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...

Important Information

Please review our Terms of Use