Hey Joseph?

[mgederberg]

Hey Joseph, I hope you are well? So now that I pretty much have my Apple //e's built out the way I want, I have begun the process of configuring this A2SERVER stuff. Ivan X's site is down but before it went bye bye, I managed to grab the RasappleII image as a Wheezy Distro and yeah I got it running. However, Tight VNC server will not install as expected anymore due to broken links and it would be a nice to have since for the most part my pi will be headless unless using the Apple II pi stuff. Now I really wanna get a build running as expected with Buster for all the stuff that is working on the Wheezy distro I have as an image 'cause Buster is now the "official" Raspbian legacy OS. image it and be done so I always have a backup that I can restore without the headaches of broken links. I will eventually be procuring a Pi card for my //e and if all is cool with the testing, I can use my super serial card for other stuff. Hell, if it works I will try and scare up another 2 cards for my other earlier Apple II and my IIGS. Is there a buster image out there I can work with quicker than having to piece this all together as you've suggested here? Having the LinApple, GS emu and the server stuff all working would be MORE than handy for my museum I am building here in my lab. Many thanks brother!

[knghtbrd]

Hi! Sorry for the delay getting back to you—I've been a bit under the weather the past couple of days and your issues got buried in the stack.

This project (or rather I should say these projects) fell off the rails a bit because of a number of reasons. Some of them are totally my fault. Others are a little complicated. The TL;DR version is that at this point I've kind of got to rebuild a lot of things from scratch. Which is fine, I'm interested in doing that. The question is … what to build? I don't really use social media at all anymore, so I'm pretty far out of touch with what users are looking for.

The first reason I've gotta build it from scratch is that Ivan's build script for the whole kit and kaboodle was a bash "history" file. The idea was he'd install Raspbian Wheezy or a Debian Wheezy virtual machine in VirtualBox, and then he'd load this file in containing like 47 command lines (they were numbered) and you'd run each one in turn. Sometimes you had to reboot between steps (not really documented, but I have an email somewhere that tells me which ones.) These command lines were all one line code blocks with if-then-else statements, etc., and by the time I was trying to use it Raspbian had changed things so it didn't work right anymore. Like specific Broadcom firmware and partition map and bootloader changes. Hoo boy. So I didn't have a single baseline. 😬 I could probably have made sense of it if I knew then some of what I know now.

Plus the scripts spat out tons of errors that just flew by unchecked, and at least some of those were important, leading to broken installations. Much effort was made trying to clean that up, sort it out, maybe migrate to using Debian packages for things, do it "the right way" so other people could step in and help with things beyond my skill, etc. These hit walls a few times, and I kind of felt like there wasn't really any interest in it—certainly it didn't seem like there was any from people who were in a position to help make stuff work that I was struggling with.

If that sounds bleak … it isn't necessarily. There's multiple projects involved as I said before, and some of them are more approachable today than others, but they're all somewhere within the realm of possibility. Here's the components that existed in the big image you downloaded:

• Customized Raspbian
• A combination of Apple II compatible netatalk and samba
• An ADTPro server
• Virtual serial disk drives using ADTPro
• A couple of Apple II emulators
• Dave Schmenk's Apple II Pi software
• A serial terminal hopefully set up for Apple II use
• A utility disk for ProDOS systems to interface all of the above

Dave Schmenk kinda wanted to do his own thing with the Apple II Pi software that basically turned an Apple IIc into a 20MHz "IIgs", let you access your Apple II disk drives from Linux, etc. It was only ever lumped in with our stuff because at the time Ivan's thing became the catch-all for Apple II and Raspberry Pi projects. That removed the hard requirement for Apple II emulators on Linux (since that's why they were there…) I didn't want to remove them because, um, I use Linux and I occasionally like having an emulator on hand! 😃

That leaves a bit customized Raspberry Pi installer (which I know how to do now), some custom stuff for a serial terminal and maybe a friendlier Linux shell environment, the ADTPro stuff, and the AppleTalk + SMB file server. That's a lot, but most of it should not be too crazy difficult, especially if I start with stuff people actually want to use.

And actually, going beyond what the old a2server or a2cloud did with those things is possible if there's enough interest. Some of it's going to require code, though, and I don't yet know how to write some of it particularly for the netatalk side of things. Since I'm replying to you in the a2server project (which is netatalk + samba), I'll describe that … in a separate comment so your eyes don't bleed.

[mgederberg]

Right on Joseph... Yeah, I got the Wheezy image to run on my raspberry pi 2B+ here but for the life of me I cannot figure out what the heck is going on with ADTPRO and the serial ports that I have tried with the Pi and then it came to me, what if this build was intended for use with the pi card directly connected to the GPIO's? Is that what's happening here? I wanted to test drive it all before committing to purchase the card from Henry at ReactiveMicro so I have a null cable, my super serial card and then I tried a USB serial, a double serial hat to the GPIO and then a GPIO serial dongle too. No comm ports in ADTPRO for any of them. I know I have communications when I tested with a windows machine and the USB serial with ADTPRO.

[knghtbrd]

The challenges to building a new a2server (as a Linux distribution for the Pi:

• Custom Raspberry Pi OS (I'm just going to keep calling this Raspbian, sorry.)
• Case sensitive filesystem problem
• DDP kernel driver set up for macipgw usage
• netatalk 2.2.5 is a disaster, security-wise, totally unsupported, etc.
• samba
• forks
• macipgw
• The EtherTalk bridge problem

A few of these are basically "solved problems", but I'm laying the issues out there so anyone who comes across this understands the pieces needed.

Custom distribution

This is done using a set of build scripts that I understand and have actually contributed to on the Pi. The RPF went kind of corporate on us a few years ago now and the whole thing is more isolated/closed off now … but I know how to do what we need to do. If everything else is done "the right way", this is easy.

Case-insensitive filesystem

At least one GS/OS version (Apple's IIRC) doesn't install right if you don't have this. For AppleTalk generally though, it's an optional thing for convenience. Still, we used to have a solution found/suggested by me and implemented by Ivan while he was still involved. It's no longer needed in Bullseye. Just create a directory (it has to be empty), chattr +F it, and anything you stick in there is case-insensitive but case-retentive like every non-UNIX OS seems to be. Hurray!

Android devices use a flash-friendly filesystem called F2FS. I find mention on Phoronix that it was slated to get casefold in 5.11, but bullseye ships with 5.10—if Raspbian switched to that for new installs (unlikely but possible), we'll need to revert that feature in the custom build.

DDP kernel driver for macipgw

There's a thing called dkms for wrangling kernel modules from outside the tree. I built a dkms package for the Linux kernel's DDP driver (AppleTalk's equivalent to TCP/IP) using the compile-time-only option we need to allow macipgw to work. It works. Is it done properly? No idea. Need someone who understands this to check it. But it does work in that it builds the driver we need, and it will move an existing DDP driver out of the way if your kernel comes with one. (Debian kernels do, Raspbian kernels I think do not.)

Problem: If you load the wrong DDP module before ours gets installed, you really need to reboot to get the correct driver loaded or macipgw will do nothing for you. Only really a problem for people running whatever a2server installer we have on existing Raspbian and Debian installations.

Netatalk 2.2.5 needs to go away already

a2server previously used netatalk 2.2.5 because something something better than 2.2.6. Nobody has ever actually explained how, except maybe that a default or two we change anyway is different on 2.2.6. Of course 2.2.6 was the last netatalk 2 version in Debian, it was removed. The packaging on it kinda needs redoing and the package is a little complicated. But I basically know how to package things for Debian still, so it'll take a bit of doing, mostly to catch up on current practices.

There isn't a 2.2.7, but there are security patches to 2.2.6. More might be needed, see below in the discussion about forks for what a solution might look like.

samba

Installing samba is easy. This provides modern filesharing of your AppleTalk share to Windows/Mac computers. Two issues exist with samba. The one I'll bring up here is that samba has a monolithic config file without a standard include directory. Files you can include! So if the requisite line doesn't already exist in your smb.conf, we can append the following:

# DO NOT REMOVE the following line! It's needed for a2server
    include = etc/smb/a2server.conf

If it's there at removal time, we can remove those lines. Removal should be done carefully though. If the comment and include line are there, like that, they can be removed. Up to two blank lines before them too. But if the comment and include line are not both there as we wrote them, we should only comment the line. Or are missing files simply ignored? Must investigate, they might be.

Forks and the proper way to do them

Prior to OS X, there was no standard way to represent forked files on a filesystem that didn't support them. The proper way after that was for file.ext, create ._file.ext for the resource fork. In zip files there was a MACOSX folder containing the resource forks that Apple's archive handler knew how to handle. Netatalk has had a few ways of doing this and none of them match the current way Apple and samba work. Netatalk 3 can do this the way samba does, but fixing this fits into the "going beyond what a2server used to do" category. The code to do it in netatalk 3 could be backported, though.

The other option is to actually just use netatalk 3. Wait, what? Netatalk 3 has no support for DDP! It's "AppleTalk" without the AppleTalk! TCP/IP only! Yes, except ultimately the way netatalk 3 works over TCP/IP is to open a TCP socket and send the same stuff over it that would be sent over DDP. To the point that if you have a IIgs with TCP/IP, you can connect to a TCP-based AppleTalk share, assuming it's been configured so you can authenticate with it! This solves most of the real security problems with netatalk 2 as well.

Of course the IIgs isn't going to always have a TCP/IP stack and the IIe isn't going to have it at all. It shouldn't be a major thing to write a bridge for netatalk 3 that allows you to run netatalk 2's atalkd and use that to talk to the more modern and somewhat simplified netatalk 3. It's possibly easier than trying to port features from netatalk 3 back to 2.2.6 + patches. It also means the netatalk people can retire their 2.x codebase (they already have, but I mean officially) while having vintage Mac and Apple II people not miss it. Maintaining atalkd and friends is a lot more reasonable without the afpd to worry about.

macipgw

It's not complex to compile this … I've just never really used it, so it's hard for me to test.

The Ethertalk bridge problem

As time passes, more and more of this stuff becomes rarer and rarer. Ethertalk bridges were never really common, but they are often buggy. I got a couple for $20 way back when, but you rarely see them even offered for sale anymore. And I can test the AsaneTalks I have, but that's about it.

LocalTalk out of your IIgs, vintage Mac, or IIe with a Workstation Card is serial. Yeah, 230.4kbps serial, but it's serial. I was last year years old when I realized that underneath all of the fancy cabling and adapters, the Zilog chip was really just doing some encapsulation and error checking in hardware, but otherwise these are just fast RS-232 serial ports, not using any of the fancy features beyond the physical link layer signaling protocol. Which we could do in software without a whole lot of fuss, probably.

This means it should totally be possible to implement a program that'd listen to the serial port, handle the SDLC packet layer, and pass the encapsulated DDP traffic on.

This is beyond my current technical abilities because I don't know these protocols and haven't got the setup I'd want to be able to snoop the serial traffic for testing/developing the interface. But wouldn't it be cool to be able to interface any LocalTalk device with your Pi using a quality USB-serial device and a PC to mac serial cable? Because that's what we're talking about here.

Wrapping up this novel (sorry)

Whew, that's a fair bit! But the pieces are there. The problems have solutions. The nice-to-haves can be planned for if they ever materialize, And really a fair bit of it involves just people testing hardware I don't have.

Things are similar on the a2cloud side: The distribution stuff is easy. I think I have sorted out how to handle which serial port does what, but it's a little complicated and needs testing. Previously we just put a getty on a serial port, but maybe it'd be cool if there was a modem emulator in there for local logins vs. telnet/telnet-ssl/ssh BBS access? (This might already exist to a large degree.) Shell tools are easy, save for email and email's only hard if you let it become hard.

The ADTPro/disk server stuff is a little complex because Dave Schmidt has AFAICT no interest in supporting Linux for it. Little enough for Mac, actually. I did manage to sort of start work on a Python ADTPro server, but I ran into some emulator bugs related to serial that slowed down my development Need to fix some hardware for working on that more and it's just been a lower priority due to the seemingly small level of interest given the many alternatives these days.

It's all doable if there's folks to do it for.

[mgederberg]

Oh yeah, there's plenty of folks to do it for in the Apple //e, IIGS and even the the Mac 68K communities Joseph. With all that you've put out there, my feeling is the best course of action would be to get the distro running flawlessly on the STABLE Legacy version of Buster and go from there. Like I said, I have the Wheezy version running but it doesn't like to use the serial to super serial set up out of the gate, I think it was built specifically for the pi card because it kivetchs about the original configuration of the serial port not being there. Provided all the packages are updated to run with a stable and more robustly patched Buster, the VNC server works cool with it all and one can work easily to modify the serial connections in use for the pi with ADTPRO, it would be a winner across the community as a start. Does this make sense?

[mgederberg]

Hey Joseph... Check this out?

https://youtu.be/IE5MOajykL0

[knghtbrd]

These days, working on buster's like working on windows 8. Technically it's getting security patches but the only people using it are the people who haven't upgraded. It won't even run on new hardware because the device drivers aren't there. The stuff that works also works on bullseye, and the stuff that doesn't work on bullseye yet didn't work on buster either. If I am smart, I'd be making stuff work on Raspbian bullseye (since that's current) and Debian bookworm/sid, since that future-proofs things.

Quite conveniently, what have I got here but a Debian bookworm/sid machine and a Pi 4 with bullseye on it. I just need to build a custom version of that if I want a smaller download. I don't have the current version of the scripts installed for doing it, but that's a git pull away and can be done whenever I'm ready.

The biggest pain in the rear right now is going to be ADTPro. I'm not immediately in a position to test that right this minute without emulators that work due to vintage keyboard issues. Need to repair one or test on emulator. If I can get ADTPro to connect with applewin on my laptop, packaging ADTPro will be pretty easy since it no longer uses the dreaded rxtx library we had so much trouble with before.

Actually, if applewin works on my laptop, I can resume the thing I was doing earlier this year—I started a proof of concept Python ADTPro server. When I say that would make things 10,000,000 times easier, I'm not joking, because we can literally forego setting up a GUI for anything on the Pi. We just won't need it!

Do you have a way to connect your Pi to your Apple II using a super serial card right now? Even better if you have two such things so I can whip up a test of the "find the right serial port" stuff in RasppleII/a2cloud#27. I'm almost positive that works as suggested, but nobody else was willing to try it on a random setup I hadn't set up so that it absolutely would work. 😅

Step 1, set up serial ports. Step 2, set up ADTPro on one of them. Step 3, set up a terminal.

Curious, what model of Pi are you using?

[mgederberg]

I have two raspberry pi 2b+’s for this project. I do have some 3’s and 4’s too but having chatted with Ivan before I knew the system requirement for the old build was the 2b+. What with the price of pi’s these days it was cheaper to source used 2b+’s that met the system requirements. I also have two //e’s with super serial cards and floppy emu’s for testing. Shortly I will also have a pi card for the //e from reactive micro.

[knghtbrd]

2B+ should work! It's the Zero 2 (which I haven't got) that I'd be concerned about.

[knghtbrd]

See #64 for a current look at this stuff. Anything with ✦ or ✦✦ is pretty easy to do. ✦✦✦ stuff is going to take a little more effort for a number of reasons.

The GS/OS installer … technically it's kind of like ✦✦ or so because the script to do it pretty much exists and pretty much works. We just need a couple dependencies and it'd be ready to go! But it's ✦✦✦✦✦ because there is/was some dumb feud going on where a bunch of people don't want another person to have the ability to insert himself into (and control and probably monetize) it, and so they've taken the take their ball and going home attitude and … ugh.

[mgederberg]

There's more than one way to skin a cat Joseph... Email me directly at mgederberg@gmail.com. Issues shared are issues split in half...

[IvanExpert]

Hey all I just want you to know that you got me inspired to see what it takes to get my A2SERVER 1.5.2 tag on my own branch up to date. If does actually run on Stretch after I fixed a bunch of now broken links within it. But that still excludes Pi 4 and later so I’m going to see if I can coax it into Buster or newer. Something to do in a Sunday.

…

On Aug 21, 2022, at 9:43 AM, mgederberg ***@***.***> wrote:  There's more than one way to skin a cat Joseph... Email me directly at ***@***.*** Issues shared are issues split in half... — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.

[mgederberg]

That's awesome news Ivan! I will certainly be testing it with my pi 2B+ and the pi card that Henry from ReactiveMicro has for me. Was the wheezy build intended for use with that card? Is this why I cannot seem to figure out the serial port configuration issues with the various other serial connections I'd tried with the pi?

[knghtbrd]

Dug up last night the correct way to create a system user in a Debian postinst:

case "$1" in
    configure)
        adduser --system --quiet --disabled-password --gecos "a2server share owner" \
                --no-create-home --home /srv/a2server --group a2server
    ;;
esac

It might be necessary to do that in preinst, though. If it's done in preinst, it meas the user is guaranteed to exist before the postinst of any package that depends on the package that creates it. Nothing on my system does it that way, but the option exists. (Obviously, you do it on install instead of configure…) The things that populate directories with Apple II goodies otherwise need to pre-depend on the package that creates the user and directory, since those things need to exist with proper ownership.

Oh, casefold … ext4 has had it since Linux 5.2, so definitely not before bullseye. However the way to test for it is to see if the file /sys/fs/ext4/features/casefold exists and if it does, does it contain the string "supported". If not, ProDOS and netatalk are both dumb. That's why we used ciopfs before. For the time being we can detect this and stop on an error explaining what's wrong under the get it working principle … but ciopfs still exists, is packaged for Ubuntu, and I did update the package at some point to the latest upstream release so I could implement that as a fallback if needed.

If casefold is supported, logic looks like…

if dir does not exist:
    create dir
    try to chattr +F it
    if error:
        fail here

if dir hasn't got +F:
    create a new dir
    move file contents to it
    chattr +F the empty dir
    if error:
        put files back
        fail here
    put files back

Room to refactor that is obvious, but I didn't do it here because premature optimization leads to bugs. Musical chairs with files is required because directories have to be empty or chattr +F will fail. Also, if the files are moved and chattr does fail, you're well-positioned to modify the error condition to attempt to fall back to ciopfs later on.

Also, have a penguin.

(edit to improve adduser command)

[mgederberg]

And I figured out that I have been trying to connect with a bad super serial card and that ADTPRO is now updated with RXTX being replaced now for JSSC. Gonna retest everything again shortly here...

https://www.youtube.com/watch?v=TzKumG1PcQo

[mgederberg]

Here's the test on the pi... FAIL with serial connectivity, but not with Windows, why?

https://www.youtube.com/watch?v=Qgwjq_RziNk