Suggestion(?): File transfers to/from SD card?

[MarkJelic]

NOTE: I have only rudimentary coding knowledge, and have next to zero knowledge of how the Pi works. So if this Suggestion is dumb - I apologise in advance.

So I'm having a discussion with some people on the RC2014 Facebook group and someone brought up that the reason they don't use PiGFX is because it doesn't have any file transfer functionality. (I'm taking that as being truthful. Please correct me if it can indeed do them.) Which I agreed with him and got me thinking:

The Pi Zero has a SD card. The PiGFX part of that software takes up less than 1gig of space leaving heaps left over. Why can't we make use of it? Could we make some extended escape codes that

1. List the files that are on the SD card and assign a numeric tag to each.
2. Initiate a file download using the numeric tags from above (of course, your Monitor program would then have to be able to accept and do something with those files.)
3. With a third new escape code, it saves to a file that you can name. (And again, the Monitor would need to be engaged with the file transfer.)

Thoughts? Put it on the wishlist or is this an unworkable idea?

[chregu82]

Ok, here are my thoughts.
At the moment there's no such feature in PiGFX. It's able to read data from the SD card, write access is not implemented. This would have to be done.
The next thing would be finding a useful way to interact with the RC2014 or any old computer with a serial interface. What would be expected? Sending escape codes to PiGFX which then responds with a binary data stream of the file? Or should we somehow emulate a harddrive which would be accessible within CP/M? Or would it be useful to just load a file from the SD card into RAM and run it?
Next problem probably is, that the Pi would probably overload the serial input buffer if file data is sent.

[MarkJelic]

Good questions and thank for the reply, because you bring up some good points. I'll be honest and say I have not used the RC2014 much as I only pretty much built it a week or so ago. And tbh I don't have a lot of experience with serial terminals, either... But I get all the concepts.

I watched this video today... I don't own the z80ctril board but seems like it would be a good one. Here is an overview video:
https://www.youtube.com/watch?v=sVV1PYWX18A&t=637s

At about the 3:20 mark, he starts talking about file interactions. Could PiGFX do similar?

[chregu82]

I think I understand how this file transfer works. You put the RC2014 into some kind of listening mode and initiate the file transfer on the sending system. The problem is, how would you tell the Pi what to send, after you put the RC2014 into this listening mode?

[BleuLlama]

I started to work on a "vampire-tap-like" drive interface bit of hardware for the RC2014, thinking that I would eventually add the functionality into PIGFX... basically it would do (x|y|z)-modem like transfers, to an sd card over the same serial connection as the console. I had initially designed it to plug right into the FTDI header on the RC2014, but the functionality could also easily be implemented in PIGFX, or a terminal application on a host system... I had started with an extension/modification to miniterm.py. Like many of my other projects, I wandered off to another shinyshiny before finishing it... perhaps it's time to revisit it? I know it would be slow as hell, but it would never be as slow as a Commodore 64, and hey, it'd be free, since it wouldn't specifically require additional hardware to implement. The base feature set i was working towards: - Escape-sequence based interface that wouldn't interfere with standard VT100/ANSI escape sequences - Capture text to a file - for logging or saving stock-RC2014 BASIC programs - "type" text from a file - perfect for "loading" programs into stock RC2014 BASIC, or typing in hex files to the loader. - random file io. (like the CH376 interface chip's command set, but sane.) Then: - subdirectory support Eventually; - commands for CP/M to do normal file transfer/utility stuff. Copy files to/from SD card. - extensions to add additional virtual files/protocols that can be remote devices... ie http:127.0.0.1/readme.txt or telnet://129.21.60.4:1701 - CP/M BIOS to natively use this interface instead of the expensive CF interface... perfect to try out CP/M without paying anything extra. ;) so... should I return to this?

…

-s

On Sun, Nov 22, 2020 at 2:27 AM Christian Lehner ***@***.***> wrote: I think I understand how this file transfer works. You put the RC2014 into some kind of listening mode and initiate the file transfer on the sending system. The problem is, how would you tell the Pi what to send, after you put the RC2014 into this listening mode? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#49 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAQ7GGP5MOCHV47TBBNBN3SRC4MTANCNFSM4T3H2QDA> .

-- Scott Lawrence yorgle@gmail.com

[chregu82]

I think this sounds absolutely cool and would give PiGFX a great new set of features.

[MarkJelic]

You put the RC2014 into some kind of listening mode and initiate the file transfer on the sending system.

As I said I've only just put the RC2014 system together and today I got to play with serial transfers. The Monitor it comes with (the Small Computer Monitor) seems to be looking at the input stream and if the input text is in an Intel Hex format, it does the usual thing of loading it into the defined memory location. So from the Terminal software, depending on what you run, you either initiate a "Send File" command, or in the case of TinyTerm, you copy the Intel Hex file to the clipboard and then Right Click > Paste it onto the window ans the SCM does its thing.

The problem is, how would you tell the Pi what to send, after you put the RC2014 into this listening mode?

Well, that could be done a couple ways; As per my OP, one of the commands could be doing a DIR command that gets streamed back to the computer as if it was typed by the user.
Also, in terms of telling the Pi what to send, sending escape codes that also have the file name I think would cut it, no?

[MarkJelic]

Like many of my other projects, I wandered off to another shinyshiny before finishing it... perhaps it's time to revisit it?

Yeah, I think we all suffer from that. :) I think it would be great if you could revisit, even just starting with a very simple "Write to a text file", and "read in a text file".

I know it would be slow as hell, but it would never be as slow as a Commodore 64, and hey, it'd be free, since it wouldn't specifically require additional hardware to implement.

Exactly. And I'm sure it wouldn't be that slow to make anyone care.

The base feature set i was working towards: - Escape-sequence based interface that wouldn't interfere with standard VT100/ANSI escape sequences -

Perfect!

Capture text to a file - for logging or saving stock-RC2014 BASIC programs

I think this is going to be the trickiest part, as to output onto screen being "caught" by the Pi and sending instead to a file... That I think will require some modification to the SCM. For example, the command to Disassemble would ideally have an option of a PIPE "|" that has the escape codes required to start the Pi saving to a file. For example

• D 8000 | Example.asm
  This would print to screen the normal disassembler text, but before doing so it would send the escape codes for Pi to grad and create a text file called Example.asm
  And then we could extend SCM to output the contents of RAM as an Intel file, again, starting with the hidden escape codes for the Pi.

"type" text from a file - perfect for "loading" programs into stock RC2014 BASIC, or typing in hex files to the loader.

Yep, perfect. I honestly think this is the easiest part to implement, as the SCM already knows how to handle incoming Intel Hex files. See if you can quickly implement this to see if this concept is viable.

random file io. (like the CH376 interface chip's command set, but sane.)

Not familiar with this, but sure, put it on the wishlist. :)

Then: - subdirectory support

As in, creating and deleting folders, or do you mean just allowing subdirectory names to be used as part of the filename?

Eventually; - commands for CP/M to do normal file transfer/utility stuff. Copy files to/from SD card.

Nice.

CP/M BIOS to natively use this interface instead of the expensive CF interface... perfect to try out CP/M without paying anything extra. ;)

Totally agree, because who the hell uses CF cards any more?? It's not like they were around in the 80s, and they certainly aren't in wide use today... So it matches no era and we may as well get with the times. :)

so... should I return to this?

Ummm... YESSS! lol

So when can we expect a beta? ;)

Thanks for your input and look forward to anything you want tested. :)

[BleuLlama]

Heh. Thanks. (fwiw. i responded to this thinking it was an email thread, since i'm responding via mail... it didn't click until just now that this was a thread on one of pigfx's github issues. Apologies for the hijack. I'll post more to the RC2014 email list about these bits shortly. :)

…

On Mon, Nov 23, 2020 at 6:37 AM Mark Jelic ***@***.***> wrote: Like many of my other projects, I wandered off to another shinyshiny before finishing it... perhaps it's time to revisit it? Yeah, I think we all suffer from that. :) I think it would be great if you could revisit, even just starting with a very simple "Write to a text file", and "read in a text file". I know it would be slow as hell, but it would never be as slow as a Commodore 64, and hey, it'd be free, since it wouldn't specifically require additional hardware to implement. Exactly. And I'm sure it wouldn't be that slow to make anyone care. The base feature set i was working towards: - Escape-sequence based interface that wouldn't interfere with standard VT100/ANSI escape sequences - Perfect! Capture text to a file - for logging or saving stock-RC2014 BASIC programs I think this is going to be the trickiest part, as to output onto screen being "caught" by the Pi and sending instead to a file... That I think will require some modification to the SCM. For example, the command to Disassemble would ideally have an option of a PIPE "|" that has the escape codes required to start the Pi saving to a file. For example - D 8000 | Example.asm This would print to screen the normal disassembler text, but before doing so it would send the escape codes for Pi to grad and create a text file called Example.asm And then we could extend SCM to output the contents of RAM as an Intel file, again, starting with the hidden escape codes for the Pi. "type" text from a file - perfect for "loading" programs into stock RC2014 BASIC, or typing in hex files to the loader. *Yep, perfect.* I honestly think this is the easiest part to implement, as the SCM already knows how to handle incoming Intel Hex files. See if you can quickly implement this to see if this concept is viable. random file io. (like the CH376 interface chip's command set, but sane.) Not familiar with this, but sure, put it on the wishlist. :) Then: - subdirectory support As in, creating and deleting folders, or do you mean just allowing subdirectory names to be used as part of the filename? Eventually; - commands for CP/M to do normal file transfer/utility stuff. Copy files to/from SD card. Nice. CP/M BIOS to natively use this interface instead of the expensive CF interface... perfect to try out CP/M without paying anything extra. ;) Totally agree, because who the hell uses CF cards any more?? It's not like they were around in the 80s, and they certainly aren't in wide use today... So it matches no era and we may as well get with the times. :) so... should I return to this? Ummm... YESSS! lol So when can we expect a beta? ;) Thanks for your input and look forward to anything you want tested. :) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#49 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAQ7GHQY5CSVEYTBC5X23TSRJCOHANCNFSM4T3H2QDA> .

-- Scott Lawrence yorgle@gmail.com

[rkincaid]

I would probably use the file transfer functionality if it were available. I'm interested in connecting to a small 6502 SBC. It already has support for invoking client-side XMODEM transfers via BASIC SAVE and LOAD commands. This seems to be somewhat prevalent on the really low end 8-bit schemes folks have been coming up with. If the basic pgfx firmware had the ability to invoke and XMODEM send/receive from the SD card, it would be pretty handy (at least for me grin). Both ExtraPutty (windows) and minicom (Mac/Linux) have UI's for doing exactly this with a serial connection. So those would be good models for how to implement the UI/functionality. For extra credit, a full ZMODEM implementation would be cool, but XMODEM alone is probably simpler and sufficient. FWIW...

[chregu82]

I could probably quite easily implement something to send a file with XMODEM to the host. But I really don't know a good way for selecting the file. Any suggestions about that?
As long as FAT write access is not implemented saving something to the SD card is not yet possible.

[rkincaid]

Yeah... I see now. I saw that uspi was recognizing the SD card as a device, but I realize now it only supports bulk transfers. It probably isn't too hard to hack some kind of very rudimentary storage structure to store and retrieve a few BASIC files on a dedicated partition at known locations, but it wouldn't be readable on a Mac/PC as a real file system. I'll think some more about this to see if there's an approach that might work or if there's any existing code that could be applied to the problem.

[chregu82]

Ok, I had some time to implement something. It's in the filetransfer branch. It's no much yet, but maybe worth a try.
You'll have to create a directory "transfer" on the SD card. There are 2 new commands:
ESC[!!d List files in "transfer" folder on the SD card
ESC[!filename.ext!t Type file named "filename.ext" to serial port (ASCII)

[rkincaid]

Thanks. Great start. I'm waiting on some new SD cards, but will try this soon (I've already successfully compiled the branch). The only issue for my needs are that the sending the text to the serial port needs to have delays (ideally user specified in a config file somewhere) that has both an inter-character delay as well as an end-of-line delay. If you stream this into a home-brew SBC, chances are it won't have flow control and won't interpret the input stream fast enough to avoid dropping characters and getting errors. I've successfully worked around this with a number of terminal apps like TeraTerm and minimum using such delays (which they support). Don't feel the need to jump in and do this right away (unless you feel like it). I see enough in the source code to hard-code a few things for now and probably get it to work. Just as an example, I use a 10ms character delay and and 100ms line delay for my SBC and it works great at 9600 baud (the speed my SBC is hardcoded at). Also, to explain a bit, the inter-character delay is just to avoid overflowing the input buffer with no flow control. The line delay is to give the interpreter time to parse a line of basic and do its internal management of the tokenized BASIC code. This will probably vary a lot depending on the SBC, hence the need to be user-specified.

One other thought, but maybe breaks your design goals... minicom and screen (Linux/Mac terminal programs) invoke their non-display functions via a user-settable control character (usually ctrl-a). Using the Esc function makes sense to get started, but again is problematic for me as Esc from the keyboard is also used to stop any transfers. Not really a problem at the moment, but if we ever wanted to invoke xmodem, I would need to start a transfer on the SBC and then invoke the corresponding end in pigfx, which would probably cancel the SBC end. Again, not something to deal with yet, but maybe something to think about from a usability perspective.

Again, thanks for implementing this great beginning.

[chregu82]

I already had the feeling, that speed would be an issue. So I implemented 2 parameters in the config file for setting a character and a line delay, more or less like TerraTerm does.
Xmodem is definitely one of my goals, so I'm going to have a look at your ctrl-a suggestion. This character could probably be used the same way as the escape character.

[chregu82]

I tried the type file feature with a RC2014 mini in BASIC. It seems to work. I also uploaded 2 samples into bin/transfer.
In BASIC just type:
10 print chr$(27) + "[!quest.bas!t"
or
10 print chr$(27) + "[!masterm.bas!t"
and
RUN

There will be no display output while data is transfered. It takes some time...
After loading you should be able to list the program or run it.
For starting a transfer you should choose a line number which gets overwritten by the data you are loading.