Almost-minimal filesystem based blog.
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

157 lines
2.2 KiB

v6.0.0: expand EntryStore, test more, cache harder This commit is something of a hairball, the result of evenings-and-weekends hacking building up a set of changes that got out of hand in parallel. If I had the energy to spare, I would break it apart into semantically-related changes, but I don't - and I suppose all this crap being rolled together is at least reflective of how the code was written. These changes are really half-finished, at best. Eventual goals: - App::WRT shouldn't directly touch the filesystem - App::WRT::EntryStore should model the entry archive completely - App::WRT::Renderer should say what to write to the publication directory - This one's a maybe: Filesystem interaction should pass through App::WRT::FileIO or something like it so that EntryStore and Renderer can be more usefully tested, with mocked writes (maybe) I do think this represents an inflection point in the long, silly life of this program: It includes a handful of new tests, and a number of the code changes were in turn easy to make because the test suite begins to model the code in a useful way. It's less and less necessary to run wrt against the p1k3.com archives to be sure that I haven't trashed something. Breaking changes to note: - Will no longer render HTML for nonexistent entries - Months and years which are flatfiles or contain an index are handled differently, albeit less brokenly - EntryStore includes index files in its overall list of entries (this seems to break less than I thought), which trickles out to bin/wrt-ls Overall changes herein: - App::WRT::Date - Move month_name() in here from App::WRT, add tests. - App::WRT::EntryStore - Hash file types for entries (directory or flatfile) - Use keys of file type hash for complete list of entries. - has_prop($entry, $property) - is_dir($entry), is_file($entry), is_extant($entry) - parent_of($entry) - has_index($entry) - Make EntryStore cache whether a file is a flatfile or a directory, as well as its existence, in a single hash. - Include index flatfiles in @source_files for use by has_index() - Various tests. - App::WRT::FileIO - Still duplicates a bunch of shit from Util, so that needs sorted. - App::WRT::Renderer - Convert to a proper class. - Add experimental FileIO class to use in Renderer (imperfect, tricky, still thinking about this). The idea is to separate out the concerns of reading and writing the filesystem. - App::WRT - Refactor display() and improve tests - Use "@entries" instead of "@options" for clarity - Handle entry names that might evaluate as false - Test running display() without any params - Rename expand_option() -> expand_alias(), refactor - Use EntryStore::has_prop() to detect wrt-noexpand.prop - year(), month(), entry() partially rewritten to use EntryStore - year() should handle months which are a flatfile - Refactor icon_markup() to use is_file() / is_dir() / is_extant(), add tests. - Add subtitle to feeds - bin/wrt-ls is now a "modulino" with tests - bin/display errors on non-existent entries - Build.PL - Remove bogus XML::Feed dependency
5 years ago
cache rendered html; extract titles; all as utf-8 This stashes the HTML version of every entry in memory and uses Mojo::DOM to extract headers from the markup for use as titles. Titles are displayed in $self->{page_navigation}, now available inside templates as ${page_navigation}. In order to keep Mojo::DOM from choking on other input, it uses the open pragma to open everything as UTF-8 by default, which also eliminates a whole class of character encoding bugs and removes some fiddling with Encode::decode() from feed_print(). This is all obviously a more memory-intensive, but caching the markup turns out to have the side effect of making it much faster to render even a large site, probably as much as anything because the HTML in question is only getting generated once per entry instead of (potentially) 2-3 times. This commit isn't very atomic. In the process of roughing it out and testing it, I made a small pile of minor but potentially breaking changes: - Removed entry_map from settings and hardcoded handling of various types of entry as some if-statements instead. - Removed embedded_perl flag in settings - was always turned on in practice, and wasn't very coherent since templating would have broken without it. - bin/wrt-display - now handles the "feed" alias correctly - EntryStore: now supports retrieving values for properties with prop_value() - this isn't currently used, but it seems like a reasonable extension of the property idea. - Added `wrt ls --with-titles`. - Added dependency versions to Build.PL. - Refactored Markup's line_parse() a little. - Refactored some tests to give cleaner / more useful output. - Renamed default template file to "default".
4 years ago
v6.0.0: expand EntryStore, test more, cache harder This commit is something of a hairball, the result of evenings-and-weekends hacking building up a set of changes that got out of hand in parallel. If I had the energy to spare, I would break it apart into semantically-related changes, but I don't - and I suppose all this crap being rolled together is at least reflective of how the code was written. These changes are really half-finished, at best. Eventual goals: - App::WRT shouldn't directly touch the filesystem - App::WRT::EntryStore should model the entry archive completely - App::WRT::Renderer should say what to write to the publication directory - This one's a maybe: Filesystem interaction should pass through App::WRT::FileIO or something like it so that EntryStore and Renderer can be more usefully tested, with mocked writes (maybe) I do think this represents an inflection point in the long, silly life of this program: It includes a handful of new tests, and a number of the code changes were in turn easy to make because the test suite begins to model the code in a useful way. It's less and less necessary to run wrt against the p1k3.com archives to be sure that I haven't trashed something. Breaking changes to note: - Will no longer render HTML for nonexistent entries - Months and years which are flatfiles or contain an index are handled differently, albeit less brokenly - EntryStore includes index files in its overall list of entries (this seems to break less than I thought), which trickles out to bin/wrt-ls Overall changes herein: - App::WRT::Date - Move month_name() in here from App::WRT, add tests. - App::WRT::EntryStore - Hash file types for entries (directory or flatfile) - Use keys of file type hash for complete list of entries. - has_prop($entry, $property) - is_dir($entry), is_file($entry), is_extant($entry) - parent_of($entry) - has_index($entry) - Make EntryStore cache whether a file is a flatfile or a directory, as well as its existence, in a single hash. - Include index flatfiles in @source_files for use by has_index() - Various tests. - App::WRT::FileIO - Still duplicates a bunch of shit from Util, so that needs sorted. - App::WRT::Renderer - Convert to a proper class. - Add experimental FileIO class to use in Renderer (imperfect, tricky, still thinking about this). The idea is to separate out the concerns of reading and writing the filesystem. - App::WRT - Refactor display() and improve tests - Use "@entries" instead of "@options" for clarity - Handle entry names that might evaluate as false - Test running display() without any params - Rename expand_option() -> expand_alias(), refactor - Use EntryStore::has_prop() to detect wrt-noexpand.prop - year(), month(), entry() partially rewritten to use EntryStore - year() should handle months which are a flatfile - Refactor icon_markup() to use is_file() / is_dir() / is_extant(), add tests. - Add subtitle to feeds - bin/wrt-ls is now a "modulino" with tests - bin/display errors on non-existent entries - Build.PL - Remove bogus XML::Feed dependency
5 years ago
v6.0.0: expand EntryStore, test more, cache harder This commit is something of a hairball, the result of evenings-and-weekends hacking building up a set of changes that got out of hand in parallel. If I had the energy to spare, I would break it apart into semantically-related changes, but I don't - and I suppose all this crap being rolled together is at least reflective of how the code was written. These changes are really half-finished, at best. Eventual goals: - App::WRT shouldn't directly touch the filesystem - App::WRT::EntryStore should model the entry archive completely - App::WRT::Renderer should say what to write to the publication directory - This one's a maybe: Filesystem interaction should pass through App::WRT::FileIO or something like it so that EntryStore and Renderer can be more usefully tested, with mocked writes (maybe) I do think this represents an inflection point in the long, silly life of this program: It includes a handful of new tests, and a number of the code changes were in turn easy to make because the test suite begins to model the code in a useful way. It's less and less necessary to run wrt against the p1k3.com archives to be sure that I haven't trashed something. Breaking changes to note: - Will no longer render HTML for nonexistent entries - Months and years which are flatfiles or contain an index are handled differently, albeit less brokenly - EntryStore includes index files in its overall list of entries (this seems to break less than I thought), which trickles out to bin/wrt-ls Overall changes herein: - App::WRT::Date - Move month_name() in here from App::WRT, add tests. - App::WRT::EntryStore - Hash file types for entries (directory or flatfile) - Use keys of file type hash for complete list of entries. - has_prop($entry, $property) - is_dir($entry), is_file($entry), is_extant($entry) - parent_of($entry) - has_index($entry) - Make EntryStore cache whether a file is a flatfile or a directory, as well as its existence, in a single hash. - Include index flatfiles in @source_files for use by has_index() - Various tests. - App::WRT::FileIO - Still duplicates a bunch of shit from Util, so that needs sorted. - App::WRT::Renderer - Convert to a proper class. - Add experimental FileIO class to use in Renderer (imperfect, tricky, still thinking about this). The idea is to separate out the concerns of reading and writing the filesystem. - App::WRT - Refactor display() and improve tests - Use "@entries" instead of "@options" for clarity - Handle entry names that might evaluate as false - Test running display() without any params - Rename expand_option() -> expand_alias(), refactor - Use EntryStore::has_prop() to detect wrt-noexpand.prop - year(), month(), entry() partially rewritten to use EntryStore - year() should handle months which are a flatfile - Refactor icon_markup() to use is_file() / is_dir() / is_extant(), add tests. - Add subtitle to feeds - bin/wrt-ls is now a "modulino" with tests - bin/display errors on non-existent entries - Build.PL - Remove bogus XML::Feed dependency
5 years ago
v6.0.0: expand EntryStore, test more, cache harder This commit is something of a hairball, the result of evenings-and-weekends hacking building up a set of changes that got out of hand in parallel. If I had the energy to spare, I would break it apart into semantically-related changes, but I don't - and I suppose all this crap being rolled together is at least reflective of how the code was written. These changes are really half-finished, at best. Eventual goals: - App::WRT shouldn't directly touch the filesystem - App::WRT::EntryStore should model the entry archive completely - App::WRT::Renderer should say what to write to the publication directory - This one's a maybe: Filesystem interaction should pass through App::WRT::FileIO or something like it so that EntryStore and Renderer can be more usefully tested, with mocked writes (maybe) I do think this represents an inflection point in the long, silly life of this program: It includes a handful of new tests, and a number of the code changes were in turn easy to make because the test suite begins to model the code in a useful way. It's less and less necessary to run wrt against the p1k3.com archives to be sure that I haven't trashed something. Breaking changes to note: - Will no longer render HTML for nonexistent entries - Months and years which are flatfiles or contain an index are handled differently, albeit less brokenly - EntryStore includes index files in its overall list of entries (this seems to break less than I thought), which trickles out to bin/wrt-ls Overall changes herein: - App::WRT::Date - Move month_name() in here from App::WRT, add tests. - App::WRT::EntryStore - Hash file types for entries (directory or flatfile) - Use keys of file type hash for complete list of entries. - has_prop($entry, $property) - is_dir($entry), is_file($entry), is_extant($entry) - parent_of($entry) - has_index($entry) - Make EntryStore cache whether a file is a flatfile or a directory, as well as its existence, in a single hash. - Include index flatfiles in @source_files for use by has_index() - Various tests. - App::WRT::FileIO - Still duplicates a bunch of shit from Util, so that needs sorted. - App::WRT::Renderer - Convert to a proper class. - Add experimental FileIO class to use in Renderer (imperfect, tricky, still thinking about this). The idea is to separate out the concerns of reading and writing the filesystem. - App::WRT - Refactor display() and improve tests - Use "@entries" instead of "@options" for clarity - Handle entry names that might evaluate as false - Test running display() without any params - Rename expand_option() -> expand_alias(), refactor - Use EntryStore::has_prop() to detect wrt-noexpand.prop - year(), month(), entry() partially rewritten to use EntryStore - year() should handle months which are a flatfile - Refactor icon_markup() to use is_file() / is_dir() / is_extant(), add tests. - Add subtitle to feeds - bin/wrt-ls is now a "modulino" with tests - bin/display errors on non-existent entries - Build.PL - Remove bogus XML::Feed dependency
5 years ago
v6.0.0: expand EntryStore, test more, cache harder This commit is something of a hairball, the result of evenings-and-weekends hacking building up a set of changes that got out of hand in parallel. If I had the energy to spare, I would break it apart into semantically-related changes, but I don't - and I suppose all this crap being rolled together is at least reflective of how the code was written. These changes are really half-finished, at best. Eventual goals: - App::WRT shouldn't directly touch the filesystem - App::WRT::EntryStore should model the entry archive completely - App::WRT::Renderer should say what to write to the publication directory - This one's a maybe: Filesystem interaction should pass through App::WRT::FileIO or something like it so that EntryStore and Renderer can be more usefully tested, with mocked writes (maybe) I do think this represents an inflection point in the long, silly life of this program: It includes a handful of new tests, and a number of the code changes were in turn easy to make because the test suite begins to model the code in a useful way. It's less and less necessary to run wrt against the p1k3.com archives to be sure that I haven't trashed something. Breaking changes to note: - Will no longer render HTML for nonexistent entries - Months and years which are flatfiles or contain an index are handled differently, albeit less brokenly - EntryStore includes index files in its overall list of entries (this seems to break less than I thought), which trickles out to bin/wrt-ls Overall changes herein: - App::WRT::Date - Move month_name() in here from App::WRT, add tests. - App::WRT::EntryStore - Hash file types for entries (directory or flatfile) - Use keys of file type hash for complete list of entries. - has_prop($entry, $property) - is_dir($entry), is_file($entry), is_extant($entry) - parent_of($entry) - has_index($entry) - Make EntryStore cache whether a file is a flatfile or a directory, as well as its existence, in a single hash. - Include index flatfiles in @source_files for use by has_index() - Various tests. - App::WRT::FileIO - Still duplicates a bunch of shit from Util, so that needs sorted. - App::WRT::Renderer - Convert to a proper class. - Add experimental FileIO class to use in Renderer (imperfect, tricky, still thinking about this). The idea is to separate out the concerns of reading and writing the filesystem. - App::WRT - Refactor display() and improve tests - Use "@entries" instead of "@options" for clarity - Handle entry names that might evaluate as false - Test running display() without any params - Rename expand_option() -> expand_alias(), refactor - Use EntryStore::has_prop() to detect wrt-noexpand.prop - year(), month(), entry() partially rewritten to use EntryStore - year() should handle months which are a flatfile - Refactor icon_markup() to use is_file() / is_dir() / is_extant(), add tests. - Add subtitle to feeds - bin/wrt-ls is now a "modulino" with tests - bin/display errors on non-existent entries - Build.PL - Remove bogus XML::Feed dependency
5 years ago
v6.0.0: expand EntryStore, test more, cache harder This commit is something of a hairball, the result of evenings-and-weekends hacking building up a set of changes that got out of hand in parallel. If I had the energy to spare, I would break it apart into semantically-related changes, but I don't - and I suppose all this crap being rolled together is at least reflective of how the code was written. These changes are really half-finished, at best. Eventual goals: - App::WRT shouldn't directly touch the filesystem - App::WRT::EntryStore should model the entry archive completely - App::WRT::Renderer should say what to write to the publication directory - This one's a maybe: Filesystem interaction should pass through App::WRT::FileIO or something like it so that EntryStore and Renderer can be more usefully tested, with mocked writes (maybe) I do think this represents an inflection point in the long, silly life of this program: It includes a handful of new tests, and a number of the code changes were in turn easy to make because the test suite begins to model the code in a useful way. It's less and less necessary to run wrt against the p1k3.com archives to be sure that I haven't trashed something. Breaking changes to note: - Will no longer render HTML for nonexistent entries - Months and years which are flatfiles or contain an index are handled differently, albeit less brokenly - EntryStore includes index files in its overall list of entries (this seems to break less than I thought), which trickles out to bin/wrt-ls Overall changes herein: - App::WRT::Date - Move month_name() in here from App::WRT, add tests. - App::WRT::EntryStore - Hash file types for entries (directory or flatfile) - Use keys of file type hash for complete list of entries. - has_prop($entry, $property) - is_dir($entry), is_file($entry), is_extant($entry) - parent_of($entry) - has_index($entry) - Make EntryStore cache whether a file is a flatfile or a directory, as well as its existence, in a single hash. - Include index flatfiles in @source_files for use by has_index() - Various tests. - App::WRT::FileIO - Still duplicates a bunch of shit from Util, so that needs sorted. - App::WRT::Renderer - Convert to a proper class. - Add experimental FileIO class to use in Renderer (imperfect, tricky, still thinking about this). The idea is to separate out the concerns of reading and writing the filesystem. - App::WRT - Refactor display() and improve tests - Use "@entries" instead of "@options" for clarity - Handle entry names that might evaluate as false - Test running display() without any params - Rename expand_option() -> expand_alias(), refactor - Use EntryStore::has_prop() to detect wrt-noexpand.prop - year(), month(), entry() partially rewritten to use EntryStore - year() should handle months which are a flatfile - Refactor icon_markup() to use is_file() / is_dir() / is_extant(), add tests. - Add subtitle to feeds - bin/wrt-ls is now a "modulino" with tests - bin/display errors on non-existent entries - Build.PL - Remove bogus XML::Feed dependency
5 years ago
v6.0.0: expand EntryStore, test more, cache harder This commit is something of a hairball, the result of evenings-and-weekends hacking building up a set of changes that got out of hand in parallel. If I had the energy to spare, I would break it apart into semantically-related changes, but I don't - and I suppose all this crap being rolled together is at least reflective of how the code was written. These changes are really half-finished, at best. Eventual goals: - App::WRT shouldn't directly touch the filesystem - App::WRT::EntryStore should model the entry archive completely - App::WRT::Renderer should say what to write to the publication directory - This one's a maybe: Filesystem interaction should pass through App::WRT::FileIO or something like it so that EntryStore and Renderer can be more usefully tested, with mocked writes (maybe) I do think this represents an inflection point in the long, silly life of this program: It includes a handful of new tests, and a number of the code changes were in turn easy to make because the test suite begins to model the code in a useful way. It's less and less necessary to run wrt against the p1k3.com archives to be sure that I haven't trashed something. Breaking changes to note: - Will no longer render HTML for nonexistent entries - Months and years which are flatfiles or contain an index are handled differently, albeit less brokenly - EntryStore includes index files in its overall list of entries (this seems to break less than I thought), which trickles out to bin/wrt-ls Overall changes herein: - App::WRT::Date - Move month_name() in here from App::WRT, add tests. - App::WRT::EntryStore - Hash file types for entries (directory or flatfile) - Use keys of file type hash for complete list of entries. - has_prop($entry, $property) - is_dir($entry), is_file($entry), is_extant($entry) - parent_of($entry) - has_index($entry) - Make EntryStore cache whether a file is a flatfile or a directory, as well as its existence, in a single hash. - Include index flatfiles in @source_files for use by has_index() - Various tests. - App::WRT::FileIO - Still duplicates a bunch of shit from Util, so that needs sorted. - App::WRT::Renderer - Convert to a proper class. - Add experimental FileIO class to use in Renderer (imperfect, tricky, still thinking about this). The idea is to separate out the concerns of reading and writing the filesystem. - App::WRT - Refactor display() and improve tests - Use "@entries" instead of "@options" for clarity - Handle entry names that might evaluate as false - Test running display() without any params - Rename expand_option() -> expand_alias(), refactor - Use EntryStore::has_prop() to detect wrt-noexpand.prop - year(), month(), entry() partially rewritten to use EntryStore - year() should handle months which are a flatfile - Refactor icon_markup() to use is_file() / is_dir() / is_extant(), add tests. - Add subtitle to feeds - bin/wrt-ls is now a "modulino" with tests - bin/display errors on non-existent entries - Build.PL - Remove bogus XML::Feed dependency
5 years ago
  1. package App::WRT::FileIO;
  2. use strict;
  3. use warnings;
  4. use open qw(:std :utf8);
  5. use Carp;
  6. use Encode;
  7. use File::Copy;
  8. use File::Path qw(make_path);
  9. use Data::Dumper;
  10. use App::WRT::Util;
  11. =pod
  12. =head1 NAME
  13. App::WRT::FileIO - read and write directories and files
  14. =head1 SYNOPSIS
  15. use App::WRT::FileIO;
  16. my $io = App::WRT::FileIO->new();
  17. =head1 METHODS
  18. =over
  19. =item new($class)
  20. Get a new FileIO object.
  21. =cut
  22. sub new {
  23. my $class = shift;
  24. my %params = (
  25. last_error => '',
  26. );
  27. my $self = \%params;
  28. bless $self, $class;
  29. }
  30. =item is_dir($path)
  31. =cut
  32. sub is_dir {
  33. return -d $_[0];
  34. }
  35. =item is_regular_file($path)
  36. =cut
  37. sub is_regular_file {
  38. return -f $_[0];
  39. }
  40. =item exists($path)
  41. =cut
  42. sub exists {
  43. return -e $_[0];
  44. }
  45. =item dir_list($dir, $sort_order, $pattern)
  46. Return a $sort_order sorted list of files matching regex $pattern in a
  47. directory.
  48. Calls $sort_order, which can be one of:
  49. alpha - alphabetical
  50. reverse_alpha - alphabetical, reversed
  51. high_to_low - numeric, high to low
  52. low_to_high - numeric, low to high
  53. =cut
  54. sub dir_list {
  55. my $self = shift;
  56. return App::WRT::Util::dir_list(@_);
  57. }
  58. # Various named sorts for dir_list:
  59. sub alpha { $a cmp $b; } # alphabetical
  60. sub high_to_low { $b <=> $a; } # numeric, high to low
  61. sub low_to_high { $a <=> $b; } # numberic, low to high
  62. sub reverse_alpha { $b cmp $a; } # alphabetical, reversed
  63. =item file_put_contents($file, $contents)
  64. Write $contents string to $file path. Because:
  65. L<https://secure.php.net/manual/en/function.file-put-contents.php>
  66. =cut
  67. sub file_put_contents {
  68. my $self = shift;
  69. App::WRT::Util::file_put_contents(@_);
  70. }
  71. =item file_get_contents($file)
  72. Get contents string of $file path. Because:
  73. L<https://secure.php.net/manual/en/function.file-get-contents.php>
  74. =cut
  75. sub file_get_contents {
  76. my $self = shift;
  77. return App::WRT::Util::file_get_contents(@_);
  78. }
  79. =item file_copy($source, $dest)
  80. =cut
  81. sub file_copy {
  82. my ($self, $source, $dest) = @_;
  83. copy($source, $dest);
  84. }
  85. =item dir_make($source, $dest)
  86. =cut
  87. sub dir_make {
  88. my ($self, $path) = @_;
  89. my $path_err;
  90. make_path($path, { error => \$path_err });
  91. if (@{ $path_err }) {
  92. $self->{last_error} = Dumper($path_err);
  93. return 0;
  94. }
  95. return 1;
  96. }
  97. =back
  98. =cut
  99. 1;