Using a subset as blog

[sureyeaah]

So I've been using neuron for a few days now and it seems perfect for note taking. I want to use a subset of my notes as a static blog. Can neuron do that currently? I would be happy to make any PRs necessary to support this.

[srid]

This is exactly what I do in srid.ca (source). Basically tag the subset of notes with something like timeline and then pull it in the blog index page using the link query <z:zettels?tag=timeline&timeline&cf>.

One thing that's missing in blog support is #73

[sureyeaah]

Yeah that's what I was trying to do as well but how do you publish only those notes that have that tag?

[srid]

Okay, you are asking about generating static site from a subset of your zettelkasten set. What defines a subset, though? A zettelkasten is densely linked; if you say that [A, B, C] is the subset, what happens if note B links to note D? B will include the title of the note D, thereby leaking private note data into the subset.

[srid]

Also consider the case of B doing a link query using, say, <z:zettels?tag=foo>. If D, E, F are tagged "foo", then B would include a link to the zettels outside the subset. Those links would be broken, in addition to their titles leaking into the published site.

I thought about this problem briefly before (but did not brainstorm in detail), and ended up keeping my private zettelkasten separate from the public one.

[sureyeaah]

What if we let each note be marked as public/private and during generation, if we specify that we only want the public ones, we only generate the public ones and in case a private note is linked from a public one - throw a warning/error or replace the link with a filler text or link to a 404 page?

In case of a query - we skip private results during public generation.

[srid]

We can use tags (say tag blog) to mark notes, and then perhaps:

neuron rib -o ~/www/blog.myself.com --only-tag=blog

Basically, only include zettels tagged blog during graph building and skip everything else. That might work.^[1]

We don't need 404 page, because errors will be inlined like this.

---

^{[1] though I personally wouldn't use it, so as to not leak private stuff by mistake!}.

[srid]

Might also have to do #141 in the scenario of untagging a published zettel.

[leira]

We can use tags (say tag blog) to mark notes, and then perhaps:

neuron rib -o ~/www/blog.myself.com --only-tag=blog

Basically, only include zettels tagged blog during graph building and skip everything else. That might work.[1]

We don't need 404 page, because errors will be inlined like this.

[1] though I personally wouldn't use it, so as to not leak private stuff by mistake!.

Any specific example of leaking private stuff?

BTW, an unrelated question: what is your advice on how to run a private neuron site? Maybe use a init script to run neuron rib -wS? What is your experience?

[srid]

@leira Run neuron rib -w as a systemd service, and then serve yournotesdir/.neuron/output via nginx as a virtualhost (protected with htpasswd).

You can use home-manager to automate some of this: https://neuron.zettel.page/6479cd5e.html or all of this if you use NixOS, see: https://nixos.wiki/wiki/Nginx

[Nadrieril]

I have that plus a cron job that runs git pull.
I would quite like that only-tag option

[leira]

home-manager systemd service worked great! Thanks guys.

@srid, Any specific example of leaking private stuff?

I'm thinking about a --public argument, it only generate the notes with public tag into a public location, and any links which points to a note without public tag, just leave it as plain text. Then we don't have to worry about leaking or 404. We can also use a blacklist pattern, generate all but the private tagged notes. This seems to be reasonably simple to do. What do you guys think?

[srid]

@leira By accidentally leaking private stuff, I was referring to say accidentally tagging a private zettel with the foo tag (--only-tag foo), and forgetting about it. Then a private zettel will appear in the z-index of the public site, until I catch the mistake. Or, say you are refactoring the tag names, and forget to rename some of the 'public' tags. There are several ways this can happen.

To be clear, I think this feature is a good idea and worth adding to neuron (as long as it is done with #141). It is just that I personally wouldn't use it.

What benefit does --public have over --only-tag $tag?

[srid]

We actually want to dedicate a separate neuron.dhall for the public site, as that can obviously have different title, theme, etc. And in neuron.dhall we can add a "only-tag" field that filters the zettels.

To do this, we will first need the "path" field from #161 (comment) (and might have to reevalute #161's proposal in terms of this feature request).

[leira]

@leira By accidentally leaking private stuff, I was referring to say accidentally tagging a private zettel with the foo tag (--only-tag foo), and forgetting about it. Then a private zettel will appear in the z-index of the public site, until I catch the mistake. Or, say you are refactoring the tag names, and forget to rename some of the 'public' tags. There are several ways this can happen.

Well, this concern exists, but this is human error, it can be mitigated but can never to eliminated.

To be clear, I think this feature is a good idea and worth adding to neuron (as long as it is done with #141). It is just that I personally wouldn't use it.

What benefit does --public have over --only-tag $tag?

Originally, what I thought was to mark the unreachable links as "grayed out" when generating, so private links won't cause 404, but still give enough hint of something interesting here. But then I realize, it should be the case for --only-tag $tag too. Then --public doesn't really add any value to it.

So question, does --only-tag $tag "gray out" unreachable links?

[srid]

So question, does --only-tag $tag "gray out" unreachable links?

Unreachable links will be no different to 404 links. Try it out by adding <this-does-not-exist> in your zettel.

[maralorn]

I was wondering about a different solution.

Could we maybe have neuron also look for zettels in subdirs? Then separating out a Subzettelkasten would be as easy as only calling neuron on the subfolder.

If you would be open to that request I would open a different issue. For me having the ability to sort zettels into different subdirectories seems like a very useful feature anyways.

[srid]

For me having the ability to sort zettels into different subdirectories seems like a very useful feature anyways.

What are the use cases for it (aside from defining a subset for a public blog)?

[maralorn]

I have e.g. filled my diary entries of the last eight years into my zettelkasten the result are a lot of entries, that make directory listings cumbersome and could be cleanly seperated into on subdir.

Also I think it could be used to provide inline documentation for something in the folder where the thing to document lies?

But yeah, the biggest reason is separation of concerns. I want to separate work stuff from private stuff, I‘d like to have a shared zettelkasten with some other people as a subkasten of mine, etc. …

[fiatjaf]

Is neuron rib -o ~/www/blog.myself.com --only-tag=blog implemented? I guess no.

I like it as a solution for this issue, but if it isn't asking much, how about also supporting --exclude-tag=private or something? I was going to create an issue for that but I'll leave this request here.

[srid]

This was discussed at length at Matrix. We came up with #355 (namespaced zettels) - though I can't say I'm 100% convinced yet.

[srid]

Meanwhile you may just write a script that uses a query command like neuron query --tag=public | jq -r '.result' | jq -r 'map(.zettelID)', and remove or move the individual .html files based on the IDs returned by that command.

[fiatjaf]

This is actually very good. If I can do it with that script everything is solved and now my opinion is that nothing else should be implemented. The script is the most flexible solution and it's likely that people with crazy requirements like me will also be comfortable using such scripts.

[srid]

Here's another way to achieve this, and something I'd consider using myself due to its explicit whitelist nature.

1. Create a new folder outside your zettelkasten, mkdir ../pub
2. Go to that folder (cd ../pub) and make it a neuron site (touch neuron.dhall)
3. Symlink the zettels you want public manually, eg: ln -s ../zettelkasten/some-note.md .
4. Do this for every zettel you want in the public site. This will be your explicit whitelist

And voila - you have a subset, that can be published separately.

Even though this explicitly whitelist approach would seem laborious at first thought, I actually prefer this approach to tagging - because what goes in public site is now explicitly tracked (I can put the dir with symlinks in a git repo), whereas with tagging it is easy to accidentally leak some private stuff (say because I added a tag by mistake to some private repo).

[srid]

Or, you can flip it, and symlink zettels from blog zettelkasten in your private zettelkasten, which is even more safer.

[sureyeaah]

You could make a file with the names of all notes you want - this isn't as laborious but you still have to be explicit.

[maralorn]

Or, you can flip it, and symlink zettels from blog zettelkasten in your private zettelkasten, which is even more safer.

That's the solution I decided to go for. (Once I can get around to setup my public server.)

[srid]

There has nearly been 3 months since this was discussed, but ... I'm coming around to needing this feature for myself.

In particular, I plan to merge my public (srid.ca) and private Zettelkasten, while still enabling publishing of the public "subset" of the merged Zettelkasten.

At this point I'm not so inclined to go with any complex proposols bordering on perfectionism (eg: namespaces). We should probably go with the simplest one. Some thoughts on that:

• How to "select" this subset? One way is to use tags; other way is to use the recently introduced "slug" metadata (i.e., only zettels with an explicit slug are to be published; which also solves the problem of file renames, as that will continue to preserve the URLs)
• How to deal with now-broken links to private zettels? One approach is to ... just "unlinkify" them; i.e., replace [[Some title]] by the inline text Some title (perhaps with a subtle visual indicator).

[bew]

One way is to use tags; other way is to use the recently introduced "slug" metadata (i.e., only zettels with an explicit slug are to be published; [...])

Only relying on the slug is brittle I think, because I may want to have a public blog, and a private one (and almost every notes would have a slug).

What about both? This way you take all notes tagged with some tag, that have a slug (to unsure stable URLs)

[srid]

FTR, here's my workflow to publish a "subset" of my private Zk to the public one, leveraging the new dirtree neuron plugin.

1. Add slug metadata to zettels marking them for public publishing (one may as well use tags here)
2. Use the following script to import those files, preserving the directory structure for dirtree operate as expected

set -e

ZK=$HOME/zettelkasten

rm -rf ./{*.md,**/}
(cd $ZK && nix-shell -p ripgrep --run "rg -lU \"^---(\r)*\nslug: \" .") > /tmp/zkimport
rsync -zarv --files-from=/tmp/zkimport  $ZK ./
git add .

Then use git diff --staged to review the imported diff before the final push.

[srid]

@bew You can do the filtering however you want, including filtering by tag, using neuron query. For example:

neuron query -t mytag | jq ".result" | jq -r "map(.zettelPath)"

This will print the list of files tagged with the given tag.