Caching process @ GSL post-processing

[canercandan]

Some other tools such as the Sphinx document generator implements a cache process that avoid rebuilding every time every file if some remain unchanged, since GSL doesnt have any caching process, that results in creating again and again every time the same files without checking if files were even been changed, that why I hacked a little caching system that only operates at GSL post-processing stage, once files are generated. It will be nice to see such a feature as another built-in GSL class.

Basically, the following functions can be used in-place of the usual .output .close GSL directives:

• cache_open(FILENAME [, FILEPATH [, BUILDDIR [, CACHEDIR ] ] ] ) -> FILE_CACHED
• cache_close(FILE_CACHED [, BUILDDIR [, CACHEDIR ] ] )

that use the samefiles(f1, f2) function in order to compare the content of two files thanks to the file.slurp and string.hash builtin functions.

Here is a basic usage of these functions:

.template 1
.
.cached = cache_open('helloworld.txt')
Hello \
World
.cache_close(cached)
.
.endtemplate

Calling this code twice will only create the build/helloworld.txt once while the build/.cache/helloworld.txt will be built twice.

And here is source code:

.template 0

global.builddir = 'build/'
global.cachedir = 'build/.cache'

directory.create(global.builddir)
directory.create(global.cachedir)

function global.check_arg_missing(ctx, key, value)
  if !defined(my.value)
    abort '[$(my.ctx)] arg "$(my.key:)" is missing'
  endif
endfunction

function global.samefiles(f1, f2)
  check_arg_missing('samefiles', 'f1', my.f1)
  check_arg_missing('samefiles', 'f2', my.f2)

  if !file.exists(my.f1)
    # my.f1 file doesnot exist
    return 0
  endif

  if !file.exists(my.f2)
    # my.f2 file doesnot exist
    return 0
  endif

  my.err = ''
  my.cnt1 = file.slurp(my.f1, my.err)?

  if !defined(my.cnt1)
    # Error: my.err
    return 0
  endif

  my.err = ''
  my.cnt2 = file.slurp(my.f2, my.err)?

  if !defined(my.cnt2)
    # Error: my.err
    return 0
  endif

  my.h1 = string.hash(my.cnt1)
  my.h2 = string.hash(my.cnt2)

  return my.h1 = my.h2
endfunction

function global.cache_open(fn, path, bdir, cdir)
  my.cdir ?= cachedir
  my.bdir ?= builddir
  my.path ?= ""

  if my.path <> ""
    directory.create("$(my.bdir:)/$(my.path:)")
    directory.create("$(my.cdir:)/$(my.path:)")
    my.fn = my.path + '/' + my.fn
  endif

  # create and cache file my.fn
  output "$(my.cdir:)/$(my.fn:)"

  return my.fn
endfunction

function global.cache_close(fn, bdir, cdir)
  my.cdir ?= cachedir
  my.bdir ?= builddir

  close

  my.src = my.cdir + '/' + my.fn
  my.dst = my.bdir + '/' + my.fn

  if !samefiles(my.src, my.dst)
    # create file my.fn
    file.delete(my.dst)
    file.copy(my.src, my.dst)
  else
    # file my.fn remains unchanged
  endif
endfunction

.endtemplate

[evoskuil]

+1 ... I really needed this!

[jschultz]

My first response would be that this is something better implemented
outside GSL, eg in make scripts.

Jonathan

On 13/12/14 08:32, Eric Voskuil wrote:

+1 ... I really needed this!

—
Reply to this email directly or view it on GitHub
#63 (comment).