From Blogsphere to a Static Site (Part 0) - Requirements

Readers of this blog might have noticed, that blog layout and blog URLs have changed (a while ago). This blog now serves static HTML pages using a nginx web server (get used to nginx, it's coming in Connections Pink too). I will document the steps and code I used to get there. Step 0 is: define the requirements and evaluate the resulting constraints:

• Export of all Blogsphere content to HTML, including the conversion of entries written in RichText
• No export of configuration or layout required
• New site structure that shows articles in year and month folders
• Modular templating system with includes for repeated pieces (e.g. header, footer, sidebar)
• Summary pages for year, month and categories
• Summary page for article series
• Existing comments to become part of the html page
• Repeatability: export must be able to repeat, but not overwrite a page that hasn't actually changed
• Storage of exported pages in a file structure as JSON files
• Rendering of static site from Notes or from JSON directory
• Redirection file, so old links get a proper redirection to the new URL
• Have a comment database for new comments
• No pagination for any of the summary pages (I might change my mind on that one)
• Cleanup messy HTML automatically, fix syntax and URLs to posts and images
• Optimized HTML, CSS and JS for speedy delivery

I had a look at Jekyll, which is the templating engine GitHub is using. I would have allowed me to just commit a new file and Github would render for me. Unfortunately Jekyll fell short of the article series and category overview pages.Finally I settled on the following toolset:

• Some customizations in Blogsphere:
  • a view listing all objects, with images first, stories later
  • a view with documents containing richtext and a form overwrite to isolate the rendered html
  • The new URL as entry field for an article, and an agent to prepopulate it
  • A dropdown list, extensible, with the name of the article series
• a redirection mapping file for nginx (generated by my Java application)
• A custom Java command line application that does export and rendering. Using a set of modules:
  • Apache Maven: Dependency management for Java projects
  • Google gson: Processing of JSON from and to Java objects
  • Joda time: Time handling for Java
  • Mustache for Java: logic less template system. Available for tons of languages, so if I change the hard part - rendering - will continue to work
  • Google Guava: Core Java library from people way smarter than me. Especially the collection and cache stuff (Guava is also inside VoP)
  • Apache Commons: More core Java, mostly needed for the HTTP client and dependency of mustache
  • Apache HTTP components: using Domino's RichText to HTML capabilities to get some form of html
  • Jsoup: Parser for "real-world HTML" (ready: messy)
• Some command line-fu to update the entries and commit to git (there's a backup of my blog at GitHub.io)

Stay tuned for more details