( git://dome.circumlunar.space/~hb9kns/plog/README.md )
( commit 9352d9adbe233628b55788d03de022af843f1742 )
# plog
## Overview
Plog is a suite of some scripts to be run on a UNIX server for handling
Markdown formatted blogs, glogs, and email newsletters with Git.
An example of the generated output can be found at my personal
[blog]( http://yargo.andropov.org/blog/list.html ) and
[glog]( gopher://sdf.org/1/users/yargo ) sites.
This is describing version 3.2 of the suite.
### Notes about the Gopher protocol
Please note that _most browsers_ are unfortunately _incapable_ of displaying
[Gopher]( http://gopherproject.org ) links!
There is a fine
[Public Gopher Proxy]( http://gopher.floodgap.com/gopher/ )
on [Floodgap]( http://www.floodgap.com ) for those crippled browsers.
Therefore, if the direct gopher link above does not work,
you may try to access it
[via proxy]( http://gopher.floodgap.com/gopher/gw.lite?sdf.org/1/users/yargo ).
If you are using Seamonkey or older versions of Firefox, you may try the "OverbiteFF"
add-on for Gopher functionality. Overbite also is available for Android,
via gopherproject.org or Floodgap.
---
## Installation and Usage
### Prerequisites
The plog suite mainly consists of the (sh/bash) shell scripts `pubnext.sh`
and `allpub.sh`, a configuration file, and a suitable directory structure.
If you make use of character conversion (UTF8 or the like), also the
script `convchars.sh` will be needed.
It makes use of the standard command line tools like `grep` and `sed`
available on almost all Unix systems.
In addition, it needs `perl` (version 5.6) for Markdown-HTML conversion
(and of course the corresponding script `mrkdwn.pl` included in the plog
suite), and `lynx` for HTML-text conversion.
`git` is used to `git mv` the processed files to an archive directory;
however, in case of failure, the scripts should gracefully switch to a
simple `mv`.
The main scripts `pubnext.sh, mrkdwn.pl, allpub.sh, convchars.sh`
should be installed in the same directory.
There is an additional bonus script `poorkyll.sh` which uses
`mrkdwn.pl` and `convchars.sh` to convert all `*.md` files in the
current directory into `*.html` files. It can be given the name of a
CSS style file which then will be referred to as style file.
_This can be used as a simple static website generator._
### Customisation of scripts with `.plog.rc`
The two shell scripts have some variables set at the beginning, which
should be configured to your needs.
They can be used for processing several collections of postings through
the run-time argument of the working directory (see below). In case you
need very different configuration, you can simply copy the scripts and
use different instances for different collections.
The scripts share a common configuration file `.plog.rc`, which has to
be in the *working* directory. It is basically an
additional shell script which is executed at the beginning of each of the
other scripts, but after setting of the variables. This way, values set
in the configuration file will override the defaults hardcoded in scripts.
In the plog suite, there is a `plogrc.template` file, which can be copied
into `.plog.rc`, and modified according to your needs.
Here is a copy of the current template file, followed
by a detailed description of all used variables.
## Template file for plog configuration
## Please copy as .plog.rc and modify as needed!
## Note: uses "$wdir" and "$mydir" as defined by calling script!
# set to additional existing file,
# if you want to include additional local config
# which will be processed at the end
localrc="$wdir/.plog.rc.local"
# list of e-mail addresses
adds='ad.txt'
# directory to save text files for publication
pubtext=''
# number of text files to be included in the index
lentext=24
# name of index file for text
indtext='gophermap'
# header for text index file
indtexthead='most recent entries in reverse chronological order'
# directory to save HTML files for publication
pubhtml=''
# number of HTML files to be included in the index
lenhtml=12
# name of index file for HTML/blog
indhtml='list.html'
# title in index file for HTML/blog
indhtmltitle='blog title'
# name of RSS file (to be saved in $pubhtml):
rsshtml='rss.xml'
# title in RSS file
rsstitle='blog feed'
# description in RSS file
rssdesc='blog description'
# base name of HTML/blog directory (from outside):
baselink='http://www.example.com/blog'
# prefix for mail subject
subject='[newsletter]'
# prefix for text file names
fprefix='t'
# mark for publication-ready texts (must be at beginning of one line)
# all up to and including this line will be removed before publication!
pubready=':publish'
# archive directory for processed texts
arch='Arch/'
htmlhead='<HTML><BODY>'
htmlfoot="</BODY><!-- generated by $myself --></HTML>"
logfile='_pubnext.log'
# special chars to HTML converter
# set to 'cat' if unused
convert0="$mydir/convchars.sh"
# markdown to HTML converter
convert1="$mydir/mrkdwn.pl"
# HTML tag marker for textual output
convert2="$mydir/convtags.sh"
# HTML to text converter
convert3='lynx -display_charset=US-ASCII -force-html -dump -stdin'
# e-mail transmission program
mailer=mailx
# mailer="logit ::" # test dummy
# temporary file for saving HTML file
tmpf1='_pubnext.html'
# temporary file for saving text file
tmpf2='_pubnext.txt'
# lockfile
lockf='.pubnext.lock'
if test -r "$localrc"
then . "$localrc"
fi
- `localrc` points to a config file, which will be read at the end, allowing
to override settings (see below under "Git hook" for explanation) -- if not
needed, set to a non-existent or empty file, e.g `/dev/null` , or completely
remove it together with the final `if-then-fi` block
- `adds` is a string with the name of the address file of the e-mail recipients
- `pubtext` is a directory, where pure text versions and (by `allpub.sh`)
an index file suitable for a gopher server will be saved; if empty, no
saving will happen (but `allpub.sh` will report errors, so you should
set up dummy directories `pubtext` and `pubhtml` if you want to make use
of `allpub.sh` just for sending e-mail newsletters)
- `lentext` is the maximum number of text version posts that will be indexed
- `indtext` is the name of the index file for pure text version; if you are
planning to use it as gopher index, it probably should be called `gophermap`
- `indtexthead` is inserted at the beginning of the index file, as description
- `pubhtml`, `lenhtml`, and `indhtml` are parameters analogous to their
text counterparts, but for the HTML files and index
- `rsshtml` is the filename for an RSS feed; in most cases `rss.xml` is best
- `rsstitle` and `rssdesc` are title and description inserted in the RSS file
- `baselink` is the basename of the blog directory as accessible from outside
(i.e, not the possibly different name for accessing it from your shell)
- `subject` is a string prepended to e-mail subjects,
and could serve as a flag for the recipients
- `fprefix` is a string with the beginning of filenames to be
considered as texts to be processed (e.g 'text', 'post',
whatever) -- but make sure it does not match other files
in the working directory, like the archive directory,
address files, or log and temporary files!
- `pubready` contains the string that should mark texts
ready for publication; it is used as a
`grep` and `sed` pattern, so be careful with punctuation;
*no lines up to and including this string will be published*
i.e you may put private notes before this string
- `arch` denotes a directory (must be writable of course)
where postings (input text/Markdown files) are moved,
after being processed and published
- `htmlhead` defines the string prepended to the files
resulting from Markdown-HTML conversion
- `htmlfoot` is the analogue for the ending of these files
- `logfile` is the file logging all activity by `pubnext.sh`
- `convert0` is the script for UTF8-HTML character conversion
- `convert1` is the script for Markdown-HTML conversion
- `convert2` is the script for marking HTML tags for textual output
(like surrounding `<em>` with explicit `*` characters)
- `convert3` is the script for HTML-text conversion
- `mailer` is the program for transmitting e-mail
- `tmpf1` and `tmpf2` are temporary files
- `lockf` is the lockfile used to control execution of `pubnext.sh`
*Please note:*
- arguments for `fprefix`, `adds`, `pubhtml`, `pubtext` can be given to
`pubnext.sh` on the command line, overriding the settings in the script
or the configuration file `.plog.rc`
- `.plog.rc` is local to any *working* directory, so you have to add it
to any directory containing posts (source texts), where you want to run
`pubnext.sh`
- `.plog.rc` is read as a *shell script,* therefore you have to honour
shell script syntax; be especially careful about closing all strings
opened with `'` and to escape `"` with `\` inside `"..."` strings!
- a single argument can be given to `allpub.sh`, indicating the working
directory where the source (Markdown formatted) files are stored; it will
be passed on to `pubnext.sh`, and if empty, the current directory is used
- if working with "Git hook" technique (see below), only one
(in general the top-most) working directory can be used
- all directories are *relative to the working directory*
given as first (and mandatory) argument to the scripts
- to have only plaintext functionality without Markdown and HTML conversion,
set all `convertN` to `cat`
### Usage considerations
#### Directories
In any case, you need a directory (the "working directory"), where the
source texts (Markdown formatted) are stored. Typically, this would be
inside of a directory which is version controlled by Git, but plog should
still work without that. In this directory, as well as its subdirectories, and
additional directories needed for publication, the plog scripts (either
launched manually by you or by some automatic process like a cron job)
in principal need complete access permission (read, write, and execute).
As a subdirectory, you should set up the archive directory, and you have
to accordingly set the variable `arch` in `pubnext.sh`.
In the working directory, you have to set up the configuration script
`.plog.rc`; you can use `plogrc.template` as a starting point.
#### File names for "posts", or source texts
Next, you have to choose how to name your source files: they all need
a common prefix, so that plog knows what to look for. *Please make sure
not to use a prefix that would match any of the additional files or
directories residing in the working directory, or the scripts may crash or
even destroy necessary files or get into an infinite loop!* Good choices
might be simply `t`, or `text`, or `post`, or any letter combination
not including the address file; don't use any punctuation, except for
`-` or `_` after at least one letter.
Set this prefix as a pattern in the configuration variable `tprefix`.
(The file suffix is irrelevant: They will all be treated as text files.)
`pubnext.sh` will generate the list of all files matching the prefix
pattern, and then process the first file in that list which does bear
the "publication-ready pattern" as defined by the `pubready` variable.
All lines up to and including this pattern will be ignored, i.e remain
unpublished; you can use this to keep private notes before this pattern.
If you want to have your files processed in a certain order, you should
therefore name them in such a way that the lexical order of their names
corresponds to the desired processing order. An example would be naming
them as `pYYMMDDI`, where `YYMMDD` is indicating year, month, and day of
writing, with a possible additional index letter `I`, or `text_NNN.md`
with a numerical index `NNN`.
#### E-mail addresses
If you want to send an e-mail newsletter, you have to list the recipient
addresses in a file whose name needs to be set in the `adds` variables of
the scripts.
This file should be a simple text file with one recipient address per
line. Lines beginning with `#` are ignored and can be used for comments.
However, never use `#` after an address: it would be part of the latter!
If you do not need the newsletter functionality, you still *should have
an address file!* It can be empty (or only contain comment lines),
but it *should be present and readable.*
If the file is missing, some scripts may issue error messages.
*Remember: you need a working command line mailer for this.*
Please set the `mailer` configuration variable accordingly,
but not all systems providing shell access may allow you to use it.
#### Script execution
Both `allpub.sh` and `pubnext.sh` can of course be executed manually.
However, in most cases, it might be more useful to launch them automatically,
e.g from a daily or weekly cron job.
`pubnext.sh` processes just the next available (non-draft) post, and send it
in processed (HTML and pure text) form to all e-mail recipients, and save the
processed forms in the appropriate directories for later use (blog/glog
publication).
`allpub.sh` will call `pubnext.sh` repeatedly, until the latter cannot find
any available post, and then generate index files in HTML and pure text
version for publication as blog and glog.
##### Git hook for script execution
It is possible to run `allpub.sh` automatically and remotely, if its directory
is under git version control. For this, you must run `installhook.sh` once on
the publishing server; it installs a git "post-update" hook which is run
whenever you push the git repo's content from a remote system. The hook in
turn changes to the repo's working directory and runs `allpub.sh` in there.
(This of course won't work in the situation described below, where publication
and script execution should run as different users.)
Example:
If you had the working directory on the server at `/some/where/` and
the (bare) git repo (which you use to push to from a remote location)
at `/home/yourself/bare.git` then you could issue the command
`./installhook.sh /home/yourself/bare.git /some/where/.git/` on the
server to have this set up. Please note: the script expects a directory
`./hooks/` to be in `/some/where/.git/` and will also verify this.
If you want to mirror a repo somewhere else, you might have the
config file `.plog.rc` tracked by git, but at the same time require
different settings on the mirror host. To solve this, you may use
a `localrc` variable as demonstrated in the sample config file
`plogrc.template` : this variable can point to a local config which
is processed *after* the standard `.plog.rc` file, and therefore
can override the latter.
#### Publication (blog/glog)
*(This section may be ignored, if publication happens via git hooks.)*
As it might not be desirable to publish the generated HTML and pure text
files (together with their respective index files) as the same user who
launched the plog scripts, a copying step for the final publication may
be necessary.
A possible solution is given in the script `allsync.sh`. It simply uses
`rsync` to pull the files from a "remote" directory (which of course may
be on the same machine) to a "local" directory, and make all of them world
readable. Also a simple `cp` could of course be used, but `rsync` only
copies files which are not yet present in the target ("local") directory.
If you want to make use of `allsync.sh`, please set its variables
according to your working and publication directories! You should
understand how `rsync` is working before doing so, though.
### Example with Git hook setup for testing
To test the system locally, you can do as follows; we assume `$src`
is the directory containing the source code (plog installation).
1. choose a directory `$loc` for testing; `/tmp` is fine for this
2. `cd $loc`
3. create a bare directory (the "repo"): `git init --bare repo.git`
4. create the working directory: `git init workdir`
5. create target directories: `mkdir text html`
5. copy the config template: `cp $loc/plogrc.template workdir/.plog.rc`
6. edit the template: `cd workdir ; $EDITOR .plog.rc`
at least setting `pubtext=/tmp/text` and `pubhtml=/tmp/html`
(replace `/tmp` by whatever you've chosen for `$loc`)
7. add the config to the repo (safer, but not required): `git add .plog.rc`
8. do an initial commit: `git commit -m initial`
9. set the "remote" repo: `git remote add origin ../repo.git` (or whatever)
10. push to the repo: `git push -u origin master`
11. initialize the hook: `$src/installhook.sh $loc/repo.git $loc/workdir`
12. add some text: `$EDITOR test.txt; git add test.txt; git commit -m 'first post'`
while making sure the name of the text file begins with the `prefix`
set in the config file, which by default is `t`
13. if you want to "publish" the text, make sure to add a line
`:publish` (or whatever has been chosen for the `pubready` config variable)
at the beginning of the text -- remember *everything before* that line
will be ignored for publication
14. to see what has been published, do `git pull` in the working directory
after pushing/publishing, as this will show the published files in the archive;
the remote will also report while pushing
15. repeat from step 12 for more text
Please note that the steps 1 to 10 are a common workflow for setting up
a local git repo with a working directory on the same machine.
When working with a (true) remote server, only the `workdir` would
be on a local machine, but all the other files and directories would
have to be installed on the server, and the remote in step 9
(on the local machine) set accordingly;
for this to work, obviously you need shell access on the server.
Remember to set `.plog.rc.local` on different remote servers, if
you do mirroring of the same content, as the config most probably
will have to be different, and this way you can still keep the
`.plog.rc` for the main publication server in the repo.
---
*2019-Dec-21 / HB9KNS*
# Copyright 2015,2019 Yargo Bonetti / HB9KNS
#
# This file is part of plog.
#
# plog is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# plog is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with plog. If not, see <http://www.gnu.org/licenses/>.