Jim Lehmer
2014-12-21 18:55:56 UTC
[Ongoing static blogging generator saga.]
I have a first pass at generating an Atom feed working (Feeds are dead!
Long live feeds! :) ). I don't have it uploaded to my public site yet, but
thought I'd share what I've done so far, since it is working in my local
test environment. I based everything on looking at how the *JsonFile*
exporter worked. I figured an XML file exporter would be similar (and it
was).
*Step 1.*
Created an *XmlFile *tiddler, based on the *JsonFile *tiddler. Here are the
contents of the tid file. Note the *extension *field and the *$:/tags/Exporter
*tag. Also note the call to an *xmltiddlers *macro.
created: 20141221144856475
creator: Jim
extension: .xml
modified: 20141221173406629
modifier: Jim
tags: static $:/tags/Exporter
title: XmlFile
type: text/vnd.tiddlywiki
\define renderContent()
<$text text=<<xmltiddlers filter:"""$(exportFilter)$""">>/>
\end
<<renderContent>>
*Step 2.*
Created an *xmltiddlers.js *macro tiddler using the *jsontiddlers.js *macro
as a starting point. There is lots of stuff going on here. I have placed ///
comments below where I think it needs exposition beyond the normal TW macro
code conventions, which are pretty well documented already.
created: 20141221145616722
creator: Jim
modified: 20141221181223993
modifier: Jim
module-type: macro
tags: static
title: xmltiddlers.js
type: application/javascript
/*\
Macro to output tiddlers matching a filter to XML (ATOM, specifically)
\*/
(function(){
/*jslint node: true, browser: true */
/*global $tw: false */
"use strict";
/*
Information about this macro
*/
exports.name = "xmltiddlers";
exports.params = [
{name: "filter"}
];
/// We're going to use Node's crypto lib to generate MD5 hashes of tiddler
titles because we can then use
/// those to generate GUIDs for Atom's id elements, which are required for
each entry.
var crypto = require("crypto");
/// The following should probably actually be a separate exported module if
it is to be more "utilitarian" in nature.
var XML = {};
/// Make sure embedded <, >, &, " and ' characters are turned into their
proper XML entities when embedding TW
/// content into an XML element.
XML.escapify = function(input) {
return input.replace(/</gm, "<").replace(/>/gm,
">").replace(/&/gm, "&").replace(/"/gm, """).replace(/'/gm,
"'");
};
/// There is probably a better way to handle TW dates, but by the time this
macro gets them they are stringified,
/// so this takes the TW date string and makes it into an ISO 8601 date.
Ugly.
XML.twDateToISO8601 = function(twDate) {
return twDate.substring(0, 4) + "-" + twDate.substring(4, 6) + "-" +
twDate.substring(6, 8) + "T" + twDate.substring(8, 10) + ":" +
twDate.substring(10, 12) + ":" + twDate.substring(12, 14);
}
/// Here is the heavy lifting for generating each Atom entry. Comments only
where needed.
XML.stringify = function(data) {
var x = "";
data.forEach(function(element, index, array) {
x += "\t\t<entry>\n";
x += "\t\t\t<title>" + element.title + "</title>\n";
/// Generate 128-bit MD5 hash from tiddler title, which we then
reformat into GUID for Atom's id element.
/// If tiddler's title changes, the hash will change, but we
probably want that anyway.
var md5 = crypto.createHash('md5');
var md5sum = md5.update(element.title);
var md5digest = md5.digest("hex");
x += "\t\t\t<id>urn:uuid:" + md5digest.substring(0, 8) + "-" +
md5digest.substring(8, 12) + "-" + md5digest.substring(12, 16) + "-" +
md5digest.substring(16, 20) + "-" + md5digest.substring(20) + "</id>\n";
x += "\t\t\t<link href='http://dullroar.com/" +
element.title.replace(/ /g, "%20") + ".html' />\n";
x += "\t\t\t<updated>" + XML.twDateToISO8601(element.modified) +
"</updated>\n";
/// This is a kludge, trying to get the first 20 "words" of the
tiddler as a summary. I think there could be
/// better ways, like looking for a summary field in the tiddler
and if it exists use that instead.
var words = element.text.split(/\s/g, 20);
x += "\t\t\t<summary>" + words.join(" ") + "</summary>\n";
x += "\t\t</entry>\n";
});
return x;
}
/// Most of this is very similar to the run function in the jsontiddlers.js
macro, other than trying to catch
/// any native JS Date objects and format them as ISO 8601 (I was hopeful
that was how I could handle the created
/// and modified tiddler fields, but alas, no).
/*
Run the macro
*/
exports.run = function(filter) {
var tiddlers = this.wiki.filterTiddlers(filter),
data = [];
for(var t=0;t<tiddlers.length; t++) {
var tiddler = this.wiki.getTiddler(tiddlers[t]);
if(tiddler) {
var fields = new Object();
for(var field in tiddler.fields) {
if (field.toISOString !== undefined) {
fields[field] = field.toISOString();
} else {
fields[field] =
XML.escapify(tiddler.getFieldString(field));
}
}
data.push(fields);
}
}
return XML.stringify(data);
};
})();
*Step 3.*
Create an *atomfeed *template tiddler that is used when calling
*rendertiddler*:
--rendertiddler atomfeed atom.xml text/plain
Here is the *atomfeed *tiddler. My starting point was the *alltiddlers *template
tiddler, although it quickly morphed away from that. Note that for now I
have hard-coded my name, site address and the overall feed's *id *GUID,
although it would obviously be better to pick those from other tiddlers or
generate them. The updated element picks up the latest
"non-static/non-system" tiddler's modifed date for the overall feed. It
transcludes the *XmlFile *tiddler, which then calls the *xmltiddlers.js*
macro. Also note the back ticks, which escape out XML elements that would
otherwise be interpreted/dropped by TW.
created: 20141220215103148
creator: Jim
modified: 20141221180306843
modifier: Jim
tags: static
title: atomfeed
type: text/vnd.tiddlywiki
\define tv-wikilink-template() #$uri_encoded$
\define tv-config-toolbar-icons() no
\define tv-config-toolbar-text() no
\define tv-config-toolbar-class() tc-btn-invisible
`<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title>`{{$:/SiteTitle}}`</title>
<subtitle>`{{$:/SiteSubtitle}}`</subtitle>
<link href="http://dullroar.com/atom.xml" rel="self" />
<link href="http://dullroar.com/" />
<author>
<name>Jim Lehmer</name>
</author>
<id>urn:uuid:2a90a542-f00c-e5c2-85d6-cd9417dc59a8</id>
<updated>`<$list
filter="[!is[system]!has[draft.of]!untagged[]!tag[static]!is[tag]!sort[modified]limit[1]]"><$link><$view
field="modified" format="date"
template="YYYY-MM-DDT0hh:0mm:0ss"/></$link></$list>`</updated>`
<$set name="exportFilter"
value="[!is[system]!untagged[]!tag[static]!title[Table of
Contents]!sort[modifed]]">
{{XmlFile}}
</$set>
`</feed>`
*Step 4.*Run the *rendertiddler *command. Here is sample output for my
local site (not on the web yet).
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title>DULL ROAR</title>
<subtitle>A heaping plate of rambled eggs</subtitle>
<link href="http://dullroar.com/atom.xml" rel="self" />
<link href="http://dullroar.com/" />
<author>
<name>Jim Lehmer</name>
</author>
<id>urn:uuid:2a90a542-f00c-e5c2-85d6-cd9417dc59a8</id>
<updated>2014-12-20T13:00:57</updated>
<entry>
<title>Categories</title>
<id>urn:uuid:af1b98ad-f7f6-86b8-4cd0-b443e022b7a0</id>
<link href='http://dullroar.com/Categories.html' />
<updated>2014-12-20T19:00:57</updated>
<summary>&lt;&lt;list-links
filter:"[is[tag]]"&gt;&gt;</summary>
</entry>
<entry>
<title>Home</title>
<id>urn:uuid:8cf04a97-3413-2302-f96d-a8e113e80ce5</id>
<link href='http://dullroar.com/Home.html' />
<updated>2014-11-27T17:48:41</updated>
<summary>Welcome. As you can see, the site has changed. Feel
free to [[browse around|Table of Contents]]. If you are</summary>
</entry>
<entry>
<title>Static Pages How-To</title>
<id>urn:uuid:18860dd9-2510-1134-568e-dfea82c82ab7</id>
<link href='http://dullroar.com/Static%20Pages%20How-To.html' />
<updated>2014-12-08T01:18:56</updated>
<summary>There is a decent write-up on [[generating static
pages|http://tiddlywiki.com/static/Generating%2520Static%2520Sites%2520with%2520TiddlyWiki.html]]
using Tiddlywiki on the main TW site. However, there are a</summary>
</entry>
</feed>
*To Do's:*
1. Wire up the Atom feed into each page's HTML header.
2. Figure out why the feed is not in sorted order, even though the filter
should be passing it in sorted in reverse order on the *modified *field.
3. Do something better in terms of generating the *summary *elements, since
as you can see it doesn't like Markdown much, nor does it handle purely
"functional" tiddlers (e.g., the *Categories *tiddler, which I need to just
filter out anyway).
4. ???
5. Profit!
Hopefully this has been interesting. Comments appreciated.
I have a first pass at generating an Atom feed working (Feeds are dead!
Long live feeds! :) ). I don't have it uploaded to my public site yet, but
thought I'd share what I've done so far, since it is working in my local
test environment. I based everything on looking at how the *JsonFile*
exporter worked. I figured an XML file exporter would be similar (and it
was).
*Step 1.*
Created an *XmlFile *tiddler, based on the *JsonFile *tiddler. Here are the
contents of the tid file. Note the *extension *field and the *$:/tags/Exporter
*tag. Also note the call to an *xmltiddlers *macro.
created: 20141221144856475
creator: Jim
extension: .xml
modified: 20141221173406629
modifier: Jim
tags: static $:/tags/Exporter
title: XmlFile
type: text/vnd.tiddlywiki
\define renderContent()
<$text text=<<xmltiddlers filter:"""$(exportFilter)$""">>/>
\end
<<renderContent>>
*Step 2.*
Created an *xmltiddlers.js *macro tiddler using the *jsontiddlers.js *macro
as a starting point. There is lots of stuff going on here. I have placed ///
comments below where I think it needs exposition beyond the normal TW macro
code conventions, which are pretty well documented already.
created: 20141221145616722
creator: Jim
modified: 20141221181223993
modifier: Jim
module-type: macro
tags: static
title: xmltiddlers.js
type: application/javascript
/*\
Macro to output tiddlers matching a filter to XML (ATOM, specifically)
\*/
(function(){
/*jslint node: true, browser: true */
/*global $tw: false */
"use strict";
/*
Information about this macro
*/
exports.name = "xmltiddlers";
exports.params = [
{name: "filter"}
];
/// We're going to use Node's crypto lib to generate MD5 hashes of tiddler
titles because we can then use
/// those to generate GUIDs for Atom's id elements, which are required for
each entry.
var crypto = require("crypto");
/// The following should probably actually be a separate exported module if
it is to be more "utilitarian" in nature.
var XML = {};
/// Make sure embedded <, >, &, " and ' characters are turned into their
proper XML entities when embedding TW
/// content into an XML element.
XML.escapify = function(input) {
return input.replace(/</gm, "<").replace(/>/gm,
">").replace(/&/gm, "&").replace(/"/gm, """).replace(/'/gm,
"'");
};
/// There is probably a better way to handle TW dates, but by the time this
macro gets them they are stringified,
/// so this takes the TW date string and makes it into an ISO 8601 date.
Ugly.
XML.twDateToISO8601 = function(twDate) {
return twDate.substring(0, 4) + "-" + twDate.substring(4, 6) + "-" +
twDate.substring(6, 8) + "T" + twDate.substring(8, 10) + ":" +
twDate.substring(10, 12) + ":" + twDate.substring(12, 14);
}
/// Here is the heavy lifting for generating each Atom entry. Comments only
where needed.
XML.stringify = function(data) {
var x = "";
data.forEach(function(element, index, array) {
x += "\t\t<entry>\n";
x += "\t\t\t<title>" + element.title + "</title>\n";
/// Generate 128-bit MD5 hash from tiddler title, which we then
reformat into GUID for Atom's id element.
/// If tiddler's title changes, the hash will change, but we
probably want that anyway.
var md5 = crypto.createHash('md5');
var md5sum = md5.update(element.title);
var md5digest = md5.digest("hex");
x += "\t\t\t<id>urn:uuid:" + md5digest.substring(0, 8) + "-" +
md5digest.substring(8, 12) + "-" + md5digest.substring(12, 16) + "-" +
md5digest.substring(16, 20) + "-" + md5digest.substring(20) + "</id>\n";
x += "\t\t\t<link href='http://dullroar.com/" +
element.title.replace(/ /g, "%20") + ".html' />\n";
x += "\t\t\t<updated>" + XML.twDateToISO8601(element.modified) +
"</updated>\n";
/// This is a kludge, trying to get the first 20 "words" of the
tiddler as a summary. I think there could be
/// better ways, like looking for a summary field in the tiddler
and if it exists use that instead.
var words = element.text.split(/\s/g, 20);
x += "\t\t\t<summary>" + words.join(" ") + "</summary>\n";
x += "\t\t</entry>\n";
});
return x;
}
/// Most of this is very similar to the run function in the jsontiddlers.js
macro, other than trying to catch
/// any native JS Date objects and format them as ISO 8601 (I was hopeful
that was how I could handle the created
/// and modified tiddler fields, but alas, no).
/*
Run the macro
*/
exports.run = function(filter) {
var tiddlers = this.wiki.filterTiddlers(filter),
data = [];
for(var t=0;t<tiddlers.length; t++) {
var tiddler = this.wiki.getTiddler(tiddlers[t]);
if(tiddler) {
var fields = new Object();
for(var field in tiddler.fields) {
if (field.toISOString !== undefined) {
fields[field] = field.toISOString();
} else {
fields[field] =
XML.escapify(tiddler.getFieldString(field));
}
}
data.push(fields);
}
}
return XML.stringify(data);
};
})();
*Step 3.*
Create an *atomfeed *template tiddler that is used when calling
*rendertiddler*:
--rendertiddler atomfeed atom.xml text/plain
Here is the *atomfeed *tiddler. My starting point was the *alltiddlers *template
tiddler, although it quickly morphed away from that. Note that for now I
have hard-coded my name, site address and the overall feed's *id *GUID,
although it would obviously be better to pick those from other tiddlers or
generate them. The updated element picks up the latest
"non-static/non-system" tiddler's modifed date for the overall feed. It
transcludes the *XmlFile *tiddler, which then calls the *xmltiddlers.js*
macro. Also note the back ticks, which escape out XML elements that would
otherwise be interpreted/dropped by TW.
created: 20141220215103148
creator: Jim
modified: 20141221180306843
modifier: Jim
tags: static
title: atomfeed
type: text/vnd.tiddlywiki
\define tv-wikilink-template() #$uri_encoded$
\define tv-config-toolbar-icons() no
\define tv-config-toolbar-text() no
\define tv-config-toolbar-class() tc-btn-invisible
`<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title>`{{$:/SiteTitle}}`</title>
<subtitle>`{{$:/SiteSubtitle}}`</subtitle>
<link href="http://dullroar.com/atom.xml" rel="self" />
<link href="http://dullroar.com/" />
<author>
<name>Jim Lehmer</name>
</author>
<id>urn:uuid:2a90a542-f00c-e5c2-85d6-cd9417dc59a8</id>
<updated>`<$list
filter="[!is[system]!has[draft.of]!untagged[]!tag[static]!is[tag]!sort[modified]limit[1]]"><$link><$view
field="modified" format="date"
template="YYYY-MM-DDT0hh:0mm:0ss"/></$link></$list>`</updated>`
<$set name="exportFilter"
value="[!is[system]!untagged[]!tag[static]!title[Table of
Contents]!sort[modifed]]">
{{XmlFile}}
</$set>
`</feed>`
*Step 4.*Run the *rendertiddler *command. Here is sample output for my
local site (not on the web yet).
<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
<title>DULL ROAR</title>
<subtitle>A heaping plate of rambled eggs</subtitle>
<link href="http://dullroar.com/atom.xml" rel="self" />
<link href="http://dullroar.com/" />
<author>
<name>Jim Lehmer</name>
</author>
<id>urn:uuid:2a90a542-f00c-e5c2-85d6-cd9417dc59a8</id>
<updated>2014-12-20T13:00:57</updated>
<entry>
<title>Categories</title>
<id>urn:uuid:af1b98ad-f7f6-86b8-4cd0-b443e022b7a0</id>
<link href='http://dullroar.com/Categories.html' />
<updated>2014-12-20T19:00:57</updated>
<summary>&lt;&lt;list-links
filter:"[is[tag]]"&gt;&gt;</summary>
</entry>
<entry>
<title>Home</title>
<id>urn:uuid:8cf04a97-3413-2302-f96d-a8e113e80ce5</id>
<link href='http://dullroar.com/Home.html' />
<updated>2014-11-27T17:48:41</updated>
<summary>Welcome. As you can see, the site has changed. Feel
free to [[browse around|Table of Contents]]. If you are</summary>
</entry>
<entry>
<title>Static Pages How-To</title>
<id>urn:uuid:18860dd9-2510-1134-568e-dfea82c82ab7</id>
<link href='http://dullroar.com/Static%20Pages%20How-To.html' />
<updated>2014-12-08T01:18:56</updated>
<summary>There is a decent write-up on [[generating static
pages|http://tiddlywiki.com/static/Generating%2520Static%2520Sites%2520with%2520TiddlyWiki.html]]
using Tiddlywiki on the main TW site. However, there are a</summary>
</entry>
</feed>
*To Do's:*
1. Wire up the Atom feed into each page's HTML header.
2. Figure out why the feed is not in sorted order, even though the filter
should be passing it in sorted in reverse order on the *modified *field.
3. Do something better in terms of generating the *summary *elements, since
as you can see it doesn't like Markdown much, nor does it handle purely
"functional" tiddlers (e.g., the *Categories *tiddler, which I need to just
filter out anyway).
4. ???
5. Profit!
Hopefully this has been interesting. Comments appreciated.
--
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+***@googlegroups.com.
To post to this group, send email to ***@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.
You received this message because you are subscribed to the Google Groups "TiddlyWiki" group.
To unsubscribe from this group and stop receiving emails from it, send an email to tiddlywiki+***@googlegroups.com.
To post to this group, send email to ***@googlegroups.com.
Visit this group at http://groups.google.com/group/tiddlywiki.
For more options, visit https://groups.google.com/d/optout.