Now prints JSON versions of feeds when given --json.
Also handles flags correctly by using @local_argv inside main() rather
than the unmodified @ARGV - a bug lingering from copypasta of wrt-display
code.
Uses JSON::Feed, for the moment, although it might be worth just rolling
my own to avoid the dependency.
Adds:
- feed_print_json_recent()
- feed_print_json()
Also puts a decode() call around feed_print() output and adds open()
pragmas to a couple of utilities.
This could still use some tweaking, probably, but it removes a special
case that's bugged me for years and lets the feed code be used for more
general problems.
Adds a module for wrapping up the sorting in order to use it for more than
just dates. Specifically, adds sorting to the entries-by-property list.
Generalizes sorting to something that kinda works for entries paths which
contain more than dates.
This entails a bunch of stuff, and I'm not sure how good an idea a lot of
it is.
- Adds a new tagging format for properties:
tag.foo.bar.prop
Where `foo` and `bar` are treated as elements of a path. Thus if
`archives/2019/10/5/tag.topics.technical.prop` exists, then
the entry at `/topics/technical` will link back to `/2019/10/5`.
- Adds "virtual" entries - if tags link to entries that don't exist,
they're implicitly created.
- Adds a concept of metadata distinct from (but about) entry text, and
caches this for every entry.
- Adds caching of child entries to EntryStore.
- Migrates `get_sub_entries()` and `subentry_expr` into EntryStore.
There are XXX comments littered all over the place at this point. Stuff
will definitely change before a release.
This stashes the HTML version of every entry in memory and uses
Mojo::DOM to extract headers from the markup for use as titles.
Titles are displayed in $self->{page_navigation}, now available
inside templates as ${page_navigation}.
In order to keep Mojo::DOM from choking on other input, it uses the open
pragma to open everything as UTF-8 by default, which also eliminates a
whole class of character encoding bugs and removes some fiddling with
Encode::decode() from feed_print().
This is all obviously a more memory-intensive, but caching the markup
turns out to have the side effect of making it much faster to render
even a large site, probably as much as anything because the HTML in
question is only getting generated once per entry instead of
(potentially) 2-3 times.
This commit isn't very atomic. In the process of roughing it out and
testing it, I made a small pile of minor but potentially breaking
changes:
- Removed entry_map from settings and hardcoded handling of various
types of entry as some if-statements instead.
- Removed embedded_perl flag in settings - was always turned on in
practice, and wasn't very coherent since templating would have
broken without it.
- bin/wrt-display - now handles the "feed" alias correctly
- EntryStore: now supports retrieving values for properties with
prop_value() - this isn't currently used, but it seems like a
reasonable extension of the property idea.
- Added `wrt ls --with-titles`.
- Added dependency versions to Build.PL.
- Refactored Markup's line_parse() a little.
- Refactored some tests to give cleaner / more useful output.
- Renamed default template file to "default".
- move get_date() from App::WRT::Util to App::WRT::Date
- make App::WRT::FileIO and ::Util less redundant
- add App::WRT::Mock::FileIO for faking writes in tests
- make bin/wrt-render-all testable, add t/bin-wrt-render-all.t
Including:
- children() - get immediate children of entry
- parent($entry) - get immediate parent of entry
- days_for($container) - get days in a month or year
- months_for($year) - get months contained in a year
- basename($entry) - get basename of an entry
These are used in App::WRT::year() and App::WRT::month(), replacing
some dir_list() invocations and at least somewhat simplifying the code.
These display helptext and current version of the application,
respectively.
`wrt -h` and `wrt --help` still work, as do a new `wrt --version` and
`wrt -v`. I think the subcommand forms are useful (and echo a growing
norm for this kind of interface), but the others should be discoverable
for users with traditional expectations. Both basic help and version
info should be readily available without much thought, which I think
justifies the repetition.
This commit is something of a hairball, the result of evenings-and-weekends
hacking building up a set of changes that got out of hand in parallel.
If I had the energy to spare, I would break it apart into
semantically-related changes, but I don't - and I suppose all this crap
being rolled together is at least reflective of how the code was
written.
These changes are really half-finished, at best. Eventual goals:
- App::WRT shouldn't directly touch the filesystem
- App::WRT::EntryStore should model the entry archive completely
- App::WRT::Renderer should say what to write to the publication
directory
- This one's a maybe: Filesystem interaction should pass through
App::WRT::FileIO or something like it so that EntryStore and Renderer
can be more usefully tested, with mocked writes (maybe)
I do think this represents an inflection point in the long, silly life of
this program: It includes a handful of new tests, and a number of the
code changes were in turn easy to make because the test suite begins to
model the code in a useful way. It's less and less necessary to run wrt
against the p1k3.com archives to be sure that I haven't trashed something.
Breaking changes to note:
- Will no longer render HTML for nonexistent entries
- Months and years which are flatfiles or contain an index are handled
differently, albeit less brokenly
- EntryStore includes index files in its overall list of entries
(this seems to break less than I thought), which trickles out to
bin/wrt-ls
Overall changes herein:
- App::WRT::Date
- Move month_name() in here from App::WRT, add tests.
- App::WRT::EntryStore
- Hash file types for entries (directory or flatfile)
- Use keys of file type hash for complete list of entries.
- has_prop($entry, $property)
- is_dir($entry), is_file($entry), is_extant($entry)
- parent_of($entry)
- has_index($entry)
- Make EntryStore cache whether a file is a flatfile or a directory, as
well as its existence, in a single hash.
- Include index flatfiles in @source_files for use by has_index()
- Various tests.
- App::WRT::FileIO
- Still duplicates a bunch of shit from Util, so that needs sorted.
- App::WRT::Renderer
- Convert to a proper class.
- Add experimental FileIO class to use in Renderer (imperfect,
tricky, still thinking about this). The idea is to separate out the
concerns of reading and writing the filesystem.
- App::WRT
- Refactor display() and improve tests
- Use "@entries" instead of "@options" for clarity
- Handle entry names that might evaluate as false
- Test running display() without any params
- Rename expand_option() -> expand_alias(), refactor
- Use EntryStore::has_prop() to detect wrt-noexpand.prop
- year(), month(), entry() partially rewritten to use EntryStore
- year() should handle months which are a flatfile
- Refactor icon_markup() to use is_file() / is_dir() / is_extant(),
add tests.
- Add subtitle to feeds
- bin/wrt-ls is now a "modulino" with tests
- bin/display errors on non-existent entries
- Build.PL
- Remove bogus XML::Feed dependency
wrt-ls is a provisional interface to the tools in WRT::EntryStore. It
should do more, but work is needed that I'm not going to put in before
releasing v5.0.0.
This seems like a good point to call things stable for this major release.
I have a bunch of ideas that I'd like to implement for a v6, which might
well break the API and underlying data structures.
This is all pretty academic since no one else actually uses this thing.
The CGI implementation starts to run into a problem: Cache invalidation.
The state of all the cached filesystem stuff is going to get out of sync
with the underlying files.
Adds a couple of hashes in the App::WRT::EntryStore::new() constructor
which stash arrays of properties for each entry, and arrays of entries
for each property, as File::Find works its way through the $entry_dir.
Uses props_for() in App::WRT::entry_topic_list() instead of dir_list().
This is maybe a tiny performance gain, but also probably more hassle
than it's worth unless other features are going to be built on these
methods (or other code that calls dir_list() simplified).
This also moves a chunk of logic out of EntryStore::all() and into the
constructor, and directly accesses $self->{source_files} in a couple of
methods instead of tossing around copies of the array.
I'm kind of iffy on whether any of this is worth doing.
I never could quite figure out why non-ASCII characters were getting
badly munged in XML::Atom::SimpleFeed output. Using Encode::decode() on
the content of entries turns out to fix this, although it's probably the
tip of an iceberg.
- uses file_get_contents() for include processing
- adds cache_includes setting
- uses include_cache on wrt instance to stash file contents in memory
I didn't expect that this would be substantially faster than just letting
the kernel worry about it, but on a large set of files with includes in
the template, it shaves about 100ms off a render.
I'm not actually sure this is worth it.
Brennen Bearnes
Brennen Bearnes