This is an ancient WordPress blog which I at one point hoped would be a more comprehensive representation of my activity online. I’m increasingly frustrated with WordPress, though, and while there are excellent tools to work in the framework from e.g. Emacs, I’d still rather maintain a simple plain-text static site. However… self-promotion never seems to bubble to the top of my to-do list. This notice is just to say: SOME DAY you will see a Hugo or Jekyll site here, and when that day comes ,perhaps I’ll start blogging actively again. Thanks for dropping by!
Now that I’ve moved my course websites to Hugo via ox-hugo, life is definitely easier than when I used WordPress, and my teaching is unquestionably crawling a little bit closer to actual reproducibility. It’s been non-trivial to figure out a good deployment process, though. My course materials live in a slightly broader context than my websites: I have my readings, my private notes, my grades, and a few other documents like teaching contracts, etc., in the main course directory. Some but not all of these materials can be committed to Git, so it makes intuitive sense to me to have a course repository which exists independently of the website. If I didn’t use ox-hugo to manage the org-to-hugo process – if, say, I used the somewhat more limited googrgeous support which is built into Hugo – I might try to build the website directly from the course materials. Instead, I’ve come up with a somewhat more convoluted process. It’s by no means perfect, and there are probably better solutions , but for now this process is working pretty well for me. Documenting here in case it’s useful to anyone else, and also in case I forget what I’ve done!
For this post I’ll take my Digital History course as an example, but I’m hoping to move all my courses to the same system, semester by semester.
Each course has two repositories:
- a main repo, mostly for my own use, but potentially of use to colleagues. These sites are small, with one file per content type – usually one each for syllabus, assignments, lectures, and some kind of “other” category.
- a website repo, containing one file per webpage – so instead of a single
Assignments.org, an entry for each individual assignment. These are markdown files created by
The website repo has two relevant branches: a
master branch that contains the uncompiled source files, and a
gh-pages branch that holds the Hugo-generated files that are actually served on the server. The gh-pages branch is checked out as a git worktree as described in the Hugo docs. It took me a little while to figure this out, but once you understand the concept of git worktrees it’s kind of fun to do things this way.
I do all my editing inside Emacs, in the original org-mode files that live in the “main” repo. Usually changes belong to one of two categories:
- “features” – planned updates to assignments, course materials, due dates, etc.
- “hotfixes” – corrections to errors, omissions, duplications in the course materials. I have to do these more often than I’d like to admit!
In either case, the workflow should look like this:
- make the change in source (.org) text
- export change to Hugo
- inspect changes on local server
- if changes look good, commit them to master in the main site
- commit those changes to the web master
hugo -sin the website master dir; commit changes to
There are a lot of steps there! Fortunately we can automate or semi-automate them.
I’ve left the first two as manual stages, because it’s smart to check changes before committing. However, I now make it a little easier by following Kaushal’s advice to run hugo server with the
--navigateToChanged switches. I now keep a pinned tab up in Firefox at
localhost:xxxx (choose your own port with
-p), and as soon as I export a page, that tab will update to the appropriate page. Fantastic! Makes checking my work much quicker. As for the rest, those steps are all done via 2 git
post-commit hooks, one for each repo:
- in the main repo, I have a post-commit hook that checks if the commit was made on the master branch, and if so, then grabs the commit message & commit hash, then cd’s to the website directory & creates a new commit on master with references to the original commit.
- in the web repo, I have a second post-commit hook, which again checks if we’re on master, and if so, generates the website in the
publicdirectory (where we have the gh-pages site checked out as a git subtree), cd’s there, and commits to the gh-pages brnach. Then both branches are pushed to gh-pages
- I’m in a transitional period where I’m moving from self-hosted sites to gh-pages where possible. So I actually regenerate the site in another directory using
hugo server -s -d myserver; I also have to update
config.tomlbefore doing this to fix a couple of paths. This is all pretty easy to do.
Here’s the first, somewhat simpler script from the main repo:
push_branch=master cur_branch=$(git rev-parse --abbrev-ref HEAD) cur_hash=$(git log -1 --pretty=%h) cur_msg=$(git log -1 --pretty=%B) web_dir=dh-website cur_dir=$PWD # only run if commit is to master if [ "$cur_branch" == "$push_branch" ]; then # if [ "$(git status -uno --porcelain)" ]; then # echo "uncommitted changes, can't push"; # else # cd to website cd $web_dir # commit hcanges echo "committing changes to website master branch\n\n" git commit -a -m "push changes to website from titaniumbones/Digital-History#$cur_hash $cur_msg" cd $cur_dir echo "pushing master to origin" git push -f origin master fi
And the slightly more baroque version from the website repo, which pushes to both gh-pages and to hackinghistory (
shimano is just an alias for hackinghistory.ca).
Note that I had to set up the hackinghistory repo as a non-bare repo with
receive.denyCurrentBranch set to
updateInstead. Thank you, stackoverflow!.
push_branch=master cur_branch=$(git rev-parse --abbrev-ref HEAD) cur_hash=$(git log -1 --pretty=%h) cur_msg=$(git log -1 --pretty=%B) # only run if commit is to master if [ "$cur_branch" == "$push_branch" ]; then if [ "$(git status -uno --porcelain)" ]; then echo "uncommitted changes, can't push"; else echo "pushing master to origin" git push -f origin master echo "Rebuilding Site" hugo -s ./ cd public && echo "changing env variables" && GIT_WORK_TREE="../.git/worktrees/public" && GIT_INDEX_FILE="../.git/worktrees/public/index"&& echo "adding all files\n\n" && git add --all echo "commitchanges to gh-pages\n\n" git commit -a -m "push changes to gh-pages from $cur_hash $cur_msg" echo "pushing gh-pages to origin" git push -f origin gh-pages && echo "successfully pushed gh-pages" cd .. sed -i.gh 's/baseURL = \"https:\/\/digitalhistory.github.io\/dh-website\/\"/baseURL = \"https:\/\/digital.hackinghistory.ca\/\"/' config.toml hugo -s ~/DH/dh-website/ -d ~/DH/dh-website/shimano/ # now shimano dir, shimano-deploy branch, shimano remote cd shimano && echo "changing env variables" && GIT_WORK_TREE="../.git/worktrees/shimano" && GIT_INDEX_FILE="../.git/worktrees/shimano/index"&& echo "adding all files\n\n" && git add --all echo "commit changes to shimano\n\n" git commit -a -m "push changes to shimano from $cur_hash $cur_msg" echo "pushing shimano-deploy to shimano" git push -f shimano shimano-deploy && echo "successfully pushed shimano" cd .. # rsync -azvbP ~/DH/dh-website/shimano/ shimano:/var/www/digital.hackinghistory.ca/ mv config.toml.gh config.toml fi fi
I’ve just learned that the being I thought was a cute dancing penguin is in fact an evil alien creature bent on destruction and domination. This creature has invaded a number of projects that I’m a part of, including Civic Tech, EDGI, and my classes “Technologies for Democracy” and “Digital History”. I can only assume that these projects have a hitherto-unrealized importance to the survival of the planet.
We have known for a long time that the American intelligence agencies don’t flinch at violating our privacy. Edward Snowden’s revelations have forced some small reforms at the NSA and elsewhere; but those small improvements will doubtless be rolled back, and XKeyscore, PRISM and, ECHELON will be replaced by even more invasive and dangerous technologies. They will be controlled by a government with strong ties to White Nationalists, militias, and various hate groups. In other times and places, governments have circulated death lists of dissenters to paramilitary organizations. I don’t think that’s likely in the USA; but I also think that as citizens, we need to act to protect ourselves from the abuse of power.
The only effective tool against government surveillance is end-to-end encryption of communication. Even encryption is likely to be inadequate against a targeted attack by the NSA’s supercomputers and tactical operations; but when it comes to the mass collection of data, properly-practiced encryption protects you from being targeted for more detailed analysis. As a by-product, encrypting your communications makes it a little bit harder (but not much!) for corporations to track your preferences, and also adds a little bit of protection from other malicious agents.
The basic building blocks of encrypted communication have been available for a long time, but until recently encryption was clumsy and error-prone. Encryption of email is still somewhat challenging. In the last year, though, two excellent solutions, and one adequate one, have emerged for messaging. I’ll talk about all these in a minute, but first let’s quickly review how encryption works.
Almost all contemporary encryption is “dual-key” – each person in a conversation has both a private and a public “key”. If I want to send you a message, I encrypt it with your public key. After that, even I can’t read my message – only you can, with your private key, which you don’t show to anyone.
This sounds simple, but it is actually quite difficult to implement encrypted communication in a simple, easy-to-understand interface. Software developers have struggled with this for a long time. It’s worth reading a little more about this:
- The EFF’s Surveillance Self-Defense Guide is a comprehensive introduction to encryption practice. You should read it.
- The FSF also has an Email Self-Defense Guide. Specific to email, but still helpful, and quite a bit shorter.
- The EFF used to have a Secure Messaging Scorecard. The first version has been deprecated, but a new one will be launched soon; read it when it comes out. It will likely supersede much of this information.
The messaging landscape has been changing rapidly. After a long delay, there is now really good encryption available, but some of the interfaces are a little confusing or misleading.
- Signal allows you to send text messages and make phone calls with full, end-to-end, dual-key encryption. You should switch to it immediately. You won’t be able to make encrypted phone calls over the regular phone network – you will need to use data instead.
- As of April, WhatsApp now offers dual-key encryption by default, including a key-verification interface. Among the mainstram apps, WhatsApp is probably the best option, but it’s not perfect, and Signal is better if you can use it.
- In June, Facebook Messenger enabled end-to-end encryption. Encryption is turned off by default and has to be turned on for every conversation, so this is not as good as WhatsApp, but if your social media life mostly moves through Facebook, you can enable encryption, and it’s worth doing.
- Google’s new Allo messenging app also permits end-to-end encryption, but it’s not on by default, and it’s a bit confusing to use. Signal is still ldefinitely preferable.
Email is tricky. For most people, Thunderbird plus EnigMail is probably the best option, but it is confusing to use and it also “leaks” information (as will always be the case for email sent through traditional channels). Various projects to fix email were announced in the wake of the Snowden revelations, but the onse I know about have mostly stopped development. It’s a hard nut to crack.
Still: you should aim to encrypt a relatively high percentage of your emails if you can. It’s not good enough to just encrypt a few; the important ones need to be masked by the trivial ones. And encrypting your emails also helps to protect the journalists, activists, ando thers who need to encrypt their communication for their own protection. We encrypt not just for ourselves, but to preserve essential freedoms for others.
Over the next few months I hope to transition to encryption in most of my messaging; that means that, if you want to be in touch with me online, you’ll need to start figuring this stuff out too. Let me know what I can do to help!
Table of Contents
OK, we’re getting down to the wire. This post is for the Sikkim trip students….
I know most of you will keep in touch with friends and family via Facebook & other social media. However, I’m hoping that we will also use the group website as a way to keep in touch while we’re gone. As you know, I’ve set the site up as a major part of our intellectual output for this project, and yo all have accounts that should let you write and publish blog posts & webpages.
Here are just a few short tips on how to use the site. I’m hoping those of you who are familiar with WordPress can help the others.
Use your username and password to get into the site at the admin/login page. To create a new journal entry, go to Posts/New (or tap the
+ button in the top menu. Compose your message in the editor box, add pictures using the “add media’ button in the editor toolbar, press ”Publish“, and you’re done.
Since we’re thinking of this as “place-based” education, location data is pretty important to us. You can add three kinds of location information to your post (underneath the editor box):
- GPX Tracks
- These are recordings of where you’ve been (discussed in my last post). If you attach one of these in the “GPX File” upload box, it will display in your post (but not in the list of all the posts…). We’ll use these a lot when we’re in Sikkim.
- KML FIles
- There’s a lot of stuff these can do, but the main reason I included them is that in some cases they can be easier to export than gpx tracks.
- You can use this to indicate where you are in the moment that you’re writing your post. This is what will show up in the “archive” and “our journey” pages (yo can look at the test pages for examples of how it will look).
I strongly encourage you to use the geodata options here! It’ll help our site out a lot.
You’ll notice that each post has one special image associated with it, that is used as a packground image in a couple of places. This is the “featured image”, which you set in a box on the lower right of the writing interface page (or, if you’re using your phone, just really far down in the interface). I’d really like it if every post had a featured image, so try to choose a good one!
Showing a post on the “Our Journey” Page
“Our Journey” is, as it were, the “official” record of our trip. Each day one or two of us will write a special post about the day’s events; it’s expected to be perhaps a little more substantive or public-facing than your regular travel diary. You can move a post into the “Our Journey” feed by adding the “Our Journey” category – in the “Category” box.
Youu may also notice the “Tags” box – we an talk later about how to make use of tags on the site.
See yo all in India!
For our trip to Sikkim, I would like students to have a way to map our progress on their phones so that it can be displayed on our website. I’ve spent a bit of time looking at various Android apps (I don’t have an iPhone) and have some thoughts about how to do this; here are my notes, intended mostly for the students but also potentially for other people looking to do something similar.
Offline Map Display
Most mapping applications rely on a continuous data feed, which will not be available to us in Sikkim. So, what applications offer offline viewing?
OSMAnd is an interesting alternative. It is mostly designed for offline use. If you buy the countour Lines Plugin for $1.99, you can add topographic map contour lines to your map, which is pretty cool. I kind of love it myself, and it’s what I’ll be using. The interface is perhaps a little less intuitive than the Google Maps interface, but I’m getting used to it. You will need to download the world base map, as well as the detailed map for all of India (if you have enough space on your phone) or the information for Sikkim only (if you don’t have enough room). The contour map is a separate download, and to be able to use it, you will need to purchase the plugin linked to above.
The free version of the app allows only 7 downloads in total, but we shouldn’t need more than that for our trip. I myself bought the full paid version but that shouldn’t be necessary IIUC.
If your phone supports the use of a removable microSD card (most do, mine doesn’t), I would highly recommend buying one. You can get an 8gb card for $4.99 at Canada Computers on College St, and that will give you plenty of space for everything you want to put on it. Buy it before you install the app, so that you can choose to store your data on the card rather than the phone’s internal memory.
It’s nice to have a map on your phone, but most of the time you won’t be using the map at all. The real reason for this post is that we’d like to place at least one of you on “backup tracking” duty every day of the trip. For this you need some kind of tracking software, which can generate files in either .gpx or .klm format; and you also need a way to get the data out of the phone and into one of our tablets for writing.
There are a number of apps that let you do this. OSMAnd+ (see above) lets you create gpx tracks, though it doesn’t let you rename or share them direcly, which is a bit of a pain. Here are some alternatives:
GPS Logger is a very simple, lightweight app that only records gpx/kml tracks for you. It doesn’t display a map or anything. Its main advantage is that it doesn’t use much energy compared to other apps.
Geo tracker is a pretty user-friendly app. Unfortunately it can’t make use of OSMAnd’s downloaded maps so I’m not sur what the display will look like when you don’t have any data.
There are http://wiki.openstreetmap.org/wiki/Android#Track-making_features]][zillions of other apps]]. Hopefully one of the above works for you but if not the Open Stree Map wiki is likely your best bet.
Getting your data out REVISED
Unfortunately our tablet doesn’t support NFC!! So you will need to transfer your files manually onto a USB thumb drive. To do this you will need a file manager. There are lots of them; you can find lists here and here. I don’t like any of them all that much but they should work, at least a little. If you have a brand-new version of Android you can also use the built-in file manager. In general it’s something of a pain, hopefully we will only be doing that for a few days when we’re completely off the grid.
NEW: I had a lot of trouble with most of the file managers, and ended up installing ES File Explorer. Now I’m able to move files easily onto an external thumb drive attached to my phone with an On The Go cable (I’m bringing 2 so we should be fine on the trip). Note that some users have reported concerns about personal data being stored on Baidu, the Chinese Google. If you have anything on your phone that you don’t want the Chinese government to know about, you may want to consider another option.
However: ES File Explorer is MUCH less painful than other options. With “ES Save To”, you can save files directly to an external USB drive attached to your phone, which can then be transferred easily to a tablet or compter.
It’s still a pain to locate your .gpx files.