Jump to content

Internet interface


msx
 Share

Recommended Posts

Hello there, as discussed in this post, i'd like to address the issue of how to connect the x16 to the internet. It is something probably everyone wants to do so maybe we could prepare some ground (i'm already looking forward to BBS, simple multiplayer games, etc :)).

So, i think the general idea is to have some external device which will handle the network stack, and relays data to the X16 via uart, spi or any other channel. It would be great if said hardware could be made inexpensively with off the shelf parts and minimum extra, like an ESP32 or some other network-enabled arduino.

Now, one thing that i consider very important is to have a common, abstract interface to do simple network access, so that every hardware developer can build their api against it and offer programmers an uniform, vendor indifferent way to access the internet. Let's not relive the errors of the past era where every program must include code to deal with all possible hardwares 🙂 Remember when you had to choose which soudn card you had and the game developer had to code for all of them?

So, if anyone has some ideas, we could perhaps come to a consensus.

My first idea is to have a specific device that handle tcp sockets, i'll paste my previous post:

Quote

So i was wondering if it would be possible to create a "tcp socket" device, where the filename is an IP address. It would need to be managed by the microcontroller, but it shouldn't be hard at all. It would work like this:

OPEN 1, 20, 0, "142.250.184.100:80"

OPEN 2, 20, 0, "api.server.com:555"

The microcontroller will open the socket connection and associate it with the file number. Subsequent read or write will be routed to the socket, and errors can be mapped to the usual FILE_NOT_FOUND, DEVICE NOT PRESENT etc.

The only problem is that filenames are 16 character long AFAIK, so they can hold any ip address (15 chars top) but not the port. Using the secondary address as port is a problem becouse i think it's at most 128. Now an address+port is 6 bytes so it CAN be fitted in 16 char, for example using hex (C345A100:00C1), but it's pretty unelegant. It would require a conversion routine which is annoying. Perhaps someone has some suggestion (if the whole proposal makes any sense :P). 

There's a problem with the size of filenames that's a bit unfortunate.

Another idea could be to reserve some space for a jump table that is to be populated by a "driver" the user can load before the program, if it makes any sense? The API can be as simple as an open/read/write/close set.

I think TCP sockets will cover most of our needs, but beside it, UDP might also be interesting and could be similarly developed. Higher level protocols, i'm not sure. Perhaps an HTTP interafce could be implemented but i think it will be pushing things too far. (A simple HTTP protocol can be directly written on the x16 side, in case.)

A flag could be reserved to create a SSL connection instead of a regular one (obviously the cryptography will be on the microcontroller).

Link to comment
Share on other sites

Posted (edited)

One simple approach is to use the secondary address as a generic index to a protocol string, and send the protocol string along with the secondary address it is associated with on the command channel. Supposing the device is device #14:

OPEN15,14,15
PRINT#15,"SA2:P80"
CLOSE15

 

Edited by BruceMcF
  • Like 2
Link to comment
Share on other sites

I think tcp is overkill for most applications. I might be wrong, but I suspect most of what people will want to do is talk directly between two x16s, or between an x16 and a server that expects to be talking to an x16 (not to, for example, an imap mail server or an http server). What if instead of tcp, we had some kind of virtual phone exchange? (which would itself run on top of either tcp or udp)

I haven't thought through all the details, but here's a sketch of how it might work: 

- the x16 is connected to a "modem" over whatever kind of serial is available

- at poweron, the modem connects to the virtual telephone exchange over tcp or udp (or over whatever kind of network the modem wants to). the modem contains some kind of secret data that authenticates it with the exchange server as having a certain identity (a "telephone number"--but it could just as easily be a username)

- the x16 will send a command, asking the modem to "dial a number". the modem relays this to the exchange server, which will cause the modem belonging to that phone number to "ring" if it's connected. software on that end can accept or reject the call.

- if the call is accepted, then the server will proxy packets between the two clients

this set up requires a centralized exchange server to act as a proxy, but it would allow anyone to write machine-to-machine multiplayer pretty easily. the proxy server can be protected from abuse by limiting throughput to a few kilobytes per second, and a single active connection per person.

Link to comment
Share on other sites

6 hours ago, lamb-duh said:

I think tcp is overkill for most applications. I might be wrong, but I suspect most of what people will want to do is talk directly between two x16s, or between an x16 and a server that expects to be talking to an x16 (not to, for example, an imap mail server or an http server). What if instead of tcp, we had some kind of virtual phone exchange? (which would itself run on top of either tcp or udp)

{snipped}

I think TCP and / or UDP accessibility via the X16 is fine, or via the smart add on card / interface, or via a proxy server. Whatever people want to do.

Given that the C= 64 and C= 128 can handle TCP, I don't think it would be an issue for the X16 to handle it if people were interested.

My biggest concern with a virtual exchange type of setup is the fact that it is one more piece that has to be working for two machines to communicate. If it should disappear at any point, no one can connect to anything. Which is fine if you want to use it for some sort of MMO (where the first M could be massively or moderately) but some of us like the good old BBS style direct connection (over telnet in this case). And it could work for people who wanted to scratch an itch to do IMAP or POP3 or SMTP or HTTP or ... whatever their imagination demands!

Link to comment
Share on other sites

The most common way people will likely hook to the Internet is probably via a serial device hooked to the User port, with some sort of Hayes AT emulation. Of course, that will only work for one connection at a time, with either raw TCP or Telnet mode communication. 

For interfacing with multiple endpoints at the same time, or for UDP,  I'd suggest looking at the Espressif AT command set. https://docs.espressif.com/projects/esp-at/en/latest/AT_Command_Set/TCP-IP_AT_Commands.html

 

  • Like 1
Link to comment
Share on other sites

10 hours ago, Scott Robison said:

My biggest concern with a virtual exchange type of setup is the fact that it is one more piece that has to be working for two machines to communicate. If it should disappear at any point, no one can connect to anything. Which is fine if you want to use it for some sort of MMO (where the first M could be massively or moderately) but some of us like the good old BBS style direct connection (over telnet in this case). And it could work for people who wanted to scratch an itch to do IMAP or POP3 or SMTP or HTTP or ... whatever their imagination demands!

This is always going to be a problem with tcp though, unnless you're directly connecting one x16 to another over tcp, which will have its problems since most of them will be behind at least one nat router. The exchange would be running very simple software (which would probably be either public domain or MIT if i wrote it), which could be run by anyone themselves anywhere. tcp is just really bad for making direct connections between two clients because of changing addresses and various nat routers that try to stop incoming connections. It does has a centralized server, but you really need a centralized server to do peer to peer tcp.

I don't think there's anything wrong with writing IMAP, POP, SMTP or HTTP clients/servers for the commander x16 if people want to communicate with the internet. I just think that most of what people will want to do is use the internet to communicate directly with another x16.

Link to comment
Share on other sites

6 hours ago, lamb-duh said:

This is always going to be a problem with tcp though, unnless you're directly connecting one x16 to another over tcp, which will have its problems since most of them will be behind at least one nat router. The exchange would be running very simple software (which would probably be either public domain or MIT if i wrote it), which could be run by anyone themselves anywhere. tcp is just really bad for making direct connections between two clients because of changing addresses and various nat routers that try to stop incoming connections. It does has a centralized server, but you really need a centralized server to do peer to peer tcp.

I don't think there's anything wrong with writing IMAP, POP, SMTP or HTTP clients/servers for the commander x16 if people want to communicate with the internet. I just think that most of what people will want to do is use the internet to communicate directly with another x16.

TCP could be the protocol used to access a "proxy" (game server) ... IP works just fine making outbound connections. As you allude, listening for connections behind a NAT is trickier, but not impossible. https://stackoverflow.com/questions/1511562/how-do-i-make-a-tcp-server-work-behind-a-router-nat-without-any-redirection-co ...

Link to comment
Share on other sites

UDP and TCP are the same as far as making NAT pinholes is concerned. The one and only thing that makes UDP "easier" to do this with is that both ends can start sending packets immediately, which will simultaneously open a dynamic NAT pinhole on both ends, whereas TCP must do the handshake thing, so the "listening" side can't send anything until it's received a SYN packet first.

Even so, the UDP version of this depends on a few things being "just so." - i.e. this whole "full cone NAT" stuff that you see mentioned in XBox Live, etc. Namely, the NAT box should use the same source port as the packet used. This isn't always possible, and many boxes only map the local port for the specific remote host:port.

In general, as a network engineer, I always recommend customers who want inbound connections to use a static IP on the inside host, and either map the specific port(s) or a range of ports to that inside IP.

We could always just allow IPv6 and then it's not even a problem anymore. 😉

 

 

Link to comment
Share on other sites

3 hours ago, ZeroByte said:

UDP and TCP are the same as far as making NAT pinholes is concerned. The one and only thing that makes UDP "easier" to do this with is that both ends can start sending packets immediately, which will simultaneously open a dynamic NAT pinhole on both ends, whereas TCP must do the handshake thing, so the "listening" side can't send anything until it's received a SYN packet first.

Even so, the UDP version of this depends on a few things being "just so." - i.e. this whole "full cone NAT" stuff that you see mentioned in XBox Live, etc. Namely, the NAT box should use the same source port as the packet used. This isn't always possible, and many boxes only map the local port for the specific remote host:port.

In general, as a network engineer, I always recommend customers who want inbound connections to use a static IP on the inside host, and either map the specific port(s) or a range of ports to that inside IP.

We could always just allow IPv6 and then it's not even a problem anymore. 😉

Exactly, I think the target audience for X16 is tech savvy enough to configure their home routers to allow inbound connections. I have a dynamic yet mostly static IP address that I use to access stuff at home.

More to the point, lest I be misunderstood. I'm not necessarily suggesting that the X16 itself needs to support IP protocols. A smart add on card could do that. I was just trying to suggest that there is no reason why TCP (or other IP related protocols) couldn't be used between the two remote systems, whether or not a proxy was involved.

Link to comment
Share on other sites

TCP is ready-made to be completely handled by some device and presented to the X16 as if it were some kind of file stream.

Personally, I'd like to see an expansion card that has several data ports like the VERA ports, but something like 8 or 16 of them that you can ask the card to map onto any active network socket. Then you can just read from them as long as there's data available, and write to them for as much as you need. The card can handle all of the packetization, windowing, fragment reassembly, etc. UDP would actually be more challenging to use as an application because the onus of these tasks is placed on the application itself for everything except per-packet checksum validation. TCP will just look like an input stream as if it were from a serial connection. That's the whole reason it was made. Going to UDP just because "NAT is hard" will end up surprising you at just how much work you may need to end up doing on the back end that would have been taken care of for you "automatically" by TCP.

Link to comment
Share on other sites

I would think that if someone wanted to use UDP based packets that the way to expose them to the X16 from a smart device would be to treat them exactly like a TCP stream, but each datagram would be prefixed with a length. At least for reading inbound datagrams. Writing datagrams could work the same way, it just doesn't feel right going the other direction based on first thoughts.

Link to comment
Share on other sites

On 7/10/2021 at 10:15 PM, lamb-duh said:

I think tcp is overkill for most applications. I might be wrong, but I suspect most of what people will want to do is talk directly between two x16s, or between an x16 and a server that expects to be talking to an x16 (not to, for example, an imap mail server or an http server). What if instead of tcp, we had some kind of virtual phone exchange? (which would itself run on top of either tcp or udp)

I haven't thought through all the details, but here's a sketch of how it might work: 

 

IDK, this seems way overengineered. You're inventing a new network over the internet and with a SPOF. And i don't think TCP is overkill, it's just the simplest of stream connection. It's the safest bet to connect to anything. The server doesn't have to be made for x16, you could connect to any telnet, mud, or bbs already available.

 

Now i think we can all agree that x16 will NOT be handling IP and TCP itself, the hardware device will. It will present a simple end to end connection to the x16 that can send and read data as with any serial connection. Obviously configuring the router is outside the scope of this thread 🙂 as @Scott Robison said we should all be able to do it, or ask for help, anyway it's not business of the x16 or the expansion network device.

So it looks like we have 3 possible implementation:

  • An expansion card, in this case the device can interact with the bus and so a memory mapped solution could be feasible, as @Zerobyte mentioned.
  • User port device. As i understand, this is an end to end connection, so at the most basic setup, this will enable only one connection to be open at a time (which could be enought actually). So in some way the x16 have to tell the device where to connect to. AT commands are a possible solution.
  • IEC device. This offer a more standardized way to access the connection as it could be mapped to already existing x16 API. Data transfer speed will be pretty slow

This is about outgoing connections. For incoming one, some server socket logic needs to be implemented. If there's support for more than one socket, it should be doable. The server logic will reside on the hardware device and the x16 can query wether there are new connections waiting.

Link to comment
Share on other sites

1 hour ago, msx said:

... So it looks like we have 3 possible implementation:

  • An expansion card, in this case the device can interact with the bus and so a memory mapped solution could be feasible, as @Zerobyte mentioned.
  • User port device. As i understand, this is an end to end connection, so at the most basic setup, this will enable only one connection to be open at a time (which could be enought actually). So in some way the x16 have to tell the device where to connect to. AT commands are a possible solution.
  • IEC device. This offer a more standardized way to access the connection as it could be mapped to already existing x16 API. Data transfer speed will be pretty slow

This is about outgoing connections. For incoming one, some server socket logic needs to be implemented. If there's support for more than one socket, it should be doable. The server logic will reside on the hardware device and the x16 can query wether there are new connections waiting.

It seems like given that bit-banged serial will be available, it would be possible to just use a serial to WiFi converter, either something like this (Amazon listing), or like this (PiModem project page).

If going for a bespoke solution, perhaps it would be because it offers some features or some price/performance point that already existing solutions do not offer.

 

Link to comment
Share on other sites

7 hours ago, msx said:

This is about outgoing connections. For incoming one, some server socket logic needs to be implemented. If there's support for more than one socket, it should be doable. The server logic will reside on the hardware device and the x16 can query wether there are new connections waiting.

I think the server process should reside on the X16 itself, or at least be possible to do such a thing. Opening an inbound socket is essentially the same as opening an outbound one, except that you tell the network stack to listen, and on which interface and port, and there would need to be an IRQ for whenever a new socket opens.

So basically this device should cover OSI layers 1-4.

Link to comment
Share on other sites

On 7/13/2021 at 3:02 AM, BruceMcF said:

It seems like given that bit-banged serial will be available, it would be possible to just use a serial to WiFi converter, either something like this (Amazon listing), or like this (PiModem project page).

If going for a bespoke solution, perhaps it would be because it offers some features or some price/performance point that already existing solutions do not offer.

 

Here's an example of what I think you meant: https://www.amazon.com/Ethernet-Converter-Adapter-Support-USR-TCP232-410S/dp/B07FM5WQKD

This an RS-232 to Ethernet Terminal Adapter. (These devices are not modems, nor do they pretend to be, though. For one, they do not have the Hayes-friendly ATD command. Instead, you have to issue a more complicated AT+ command string to make a TCP or Telnet connection.)

What surprises me is how so many retro enthusiasts don't realize this is already a thing. These terminal adapters can already do everything mentioned on this thread, and all we need to do is provide software on the Commander side to interact with the TA. 

It's worth noting that TCPSer is actually impersonating a modem, rather than a terminal adapter, so it is not a multi-channel device and can't handle multiple TCP streams, nor can it do UDP at all (that I'm aware of.) The AT+ firmware on terminal adapters is a little different in that it CAN do those things... but it's not as simple as just "Connect to server and send data." You have to request to read data from the TA's buffer and request to send using AT+ commands. 

It's actually pretty straightforward to write something like this for the Raspberry Pi... I might go ahead and write something myself, since I've put the parallel interface stuff on the shelf for a bit. 

Link to comment
Share on other sites

8 hours ago, TomXP411 said:

That's the exact listing I had in the tab that I failed to copy the link from, $50, though this one is RS232C only and $23.

Not to be confused with one of these, which is just a plug adapter to let you extend an RS-232C interface using cheaper network cable.

Edited by BruceMcF
Link to comment
Share on other sites

9 hours ago, BruceMcF said:

That's the exact listing I had in the tab that I failed to copy the link from, $50, though this one is RS232C only and $23.

Not to be confused with one of these, which is just a plug adapter to let you extend an RS-232C interface using cheaper network cable.

That’s actually a perfect example. I have two of those with WiFi radios that I purchased to work with my Altairduino… only to find that the level converter chips in the Altairduino doesn’t work, due to a design defect on the adapter board. Since then, I have found that RunCPM is a lot more effective for running CP/M software, so the only thing the AD does now is blink for me.

Anyway, at $23, this is more cost effective than a Raspberry Pi, and while you still need a level converter, a TTL to RS-232 level converter is actually easier to get and use than the 5v - 3.3v we’d need for a Pi. (I’m pretty sure the User port will be running at 5V.)

 

  • Like 1
Link to comment
Share on other sites

5 minutes ago, TomXP411 said:

That’s actually a perfect example. I have two of those with WiFi radios that I purchased to work with my Altairduino… only to find that the level converter chips in the Altairduino doesn’t work, due to a design defect on the adapter board. Since then, I have found that RunCPM is a lot more effective for running CP/M software, so the only thing the AD does now is blink for me.

Anyway, at $23, this is more cost effective than a Raspberry Pi, and while you still need a level converter, a TTL to RS-232 level converter is actually easier to get and use than the 5v - 3.3v we’d need for a Pi. (I’m pretty sure the User port will be running at 5V.)

 

If it's industrial equipment, it might be 5v tolerant, since voltage drops with longer cable runs ... it would be fortunate if that was true, so you only have to convert voltage on the input lines.

Edited by BruceMcF
Link to comment
Share on other sites

14 hours ago, BruceMcF said:

If it's industrial equipment, it might be 5v tolerant, since voltage drops with longer cable runs ... it would be fortunate if that was true, so you only have to convert voltage on the input lines.

I'm not sure what you mean there... the Altairduino runs on an Arduino and requires a MAX232 chip in order to convert TTL to RS-232. Likewise, the Commander X16 will require level converters when connecting to basically any 32-bit microcontroller, since most are not 5v tolerant. The Raspberry Pi definitely is not. 

 

Link to comment
Share on other sites

1 hour ago, TomXP411 said:

I'm not sure what you mean there... the Altairduino runs on an Arduino and requires a MAX232 chip in order to convert TTL to RS-232. Likewise, the Commander X16 will require level converters when connecting to basically any 32-bit microcontroller, since most are not 5v tolerant. The Raspberry Pi definitely is not. 

 

I wasn't talking about the Altairduino, I was talking about the RS-232C to Ethernet box. The spec range of RS-232C is +3V<Logic0<+15V and -15V<Logic1<-3V, so if it's a direct connection, it would tolerate a very simple level conversion circuit.

Link to comment
Share on other sites

On 7/16/2021 at 1:17 AM, BruceMcF said:

I wasn't talking about the Altairduino, I was talking about the RS-232C to Ethernet box. The spec range of RS-232C is +3V<Logic0<+15V and -15V<Logic1<-3V, so if it's a direct connection, it would tolerate a very simple level conversion circuit.

A max232 based shifter with pre-soldered DE9 is like $3 on Amazon...

 

  • Like 1
Link to comment
Share on other sites

27 minutes ago, TomXP411 said:

A max232 based shifter with pre-soldered DE9 is like $3 on Amazon...

 

Now that's my price point!

 

 

https://en.wikipedia.org/wiki/MAX232:

Quote

The MAX232 is an integrated circuit first created in 1987 by Maxim Integrated Products that converts signals from a TIA-232 (RS-232) serial port to signals suitable for use in TTL-compatible digital logic circuits. The MAX232 is a dual transmitter / dual receiver that typically is used to convert the RX, TX, CTS, RTS signals.

The drivers provide TIA-232 voltage level outputs (about ±7.5 volts) from a single 5-volt supply by on-chip charge pumps and external capacitors. This makes it useful for implementing TIA-232 in devices that otherwise do not need any other voltages.

The receivers reduce TIA-232 inputs, which may be as high as ±25 volts, to standard 5 volt TTL levels. These receivers have a typical threshold of 1.3 volts and a typical hysteresis of 0.5 volts.

 

Edited by rje
  • Haha 1
Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

×
×
  • Create New...

Important Information

Please review our Terms of Use