matlabdomain does not recursively document folder/module?

[zouhairm]

consider the folder structure of https://github.com/sphinx-contrib/matlabdomain/tree/master/tests/test_data

I would have expected that:

.. automodule:: test_data

would generate the documentation for the whole folder, but it seems to print nothing according to the example:
https://bwanamarko.alwaysdata.net/matlabdomain/#test-data

Instead, one has to do:

.. automodule:: test_data.+package

to get something like: https://bwanamarko.alwaysdata.net/matlabdomain/#packagefunc

How do I get all of the packages/classes/scripts inside of the 'test_data' module to get all listed?
or do I have to manually list them (or somehow autogenerate them since api-doc seems to not work with matlab?)

Thanks!

[joeced]

The site https://bwanamarko.alwaysdata.net/matlabdomain was maintained by the original author of the package. I have no control of it, and therefore it is rather outdated :(

We don't have any "autosummary" feature yet. The automodule directive instructs the module to parse everything, it should do it recursively.

[joeced]

Ah. Now I understand! Given

.. automodule:: test_data

Will not do anything (except parsing the MATLAB source files). If you do

.. automodule:: test_data
    :members:

It will generate rst docs for everything in the folder that automodule points to. This is similar to how the Python autodoc feature works.

[DutchGFX]

Without hijacking the thread, is there any known workaround to get autosummary to work, or are there plans to add that feature?

[zouhairm]

FWIW, I wrote a script to sort of replicate the behavior of api-doc. Probably could be improved, but works for the type of classes/packages I have in my repo

#!/usr/bin/env python
"""
Replicate behavior of sphinx-apidoc.

Run with --help to see options.
"""
import argparse
import sys
from pathlib import Path
import collections

import logging
_log = logging.getLogger(sys.argv[0])


def parse_args():
    """Parse the command line arguments using argparse."""
    parser = argparse.ArgumentParser(
        description="Command Line Interface to generate matlab apidoc",
        formatter_class=argparse.ArgumentDefaultsHelpFormatter)

    arg = parser.add_argument
    arg('target')
    arg('--verbose', '-v', action='count', default=1,
        help='-v for info -vv for info')
    arg('--force', action='store_true', default=False,
        help='overwrite files')
    arg('--no-toc', action='store_true', default=False,
        help='Skip TOC file')
    arg('--output', '-o', default='./',
        help='Output folder')

    args = parser.parse_args()

    args.log_verbose = 40 - (10 * args.verbose) if args.verbose > 0 else 0
    return args


def _sanitize(s):
    return s.replace('_', r'\_')


def main(args):
    """
    Replicate behavior of sphinx-apidoc.

    :param  args:  arguments parsed from command line options (see :py:func:`parse_args`)
    """
    logging.basicConfig(level=args.log_verbose, format='%(levelname)s@%(filename)s:%(funcName)s:%(lineno)d:\n\t%(message)s')
    _log.info(f'CLI arguments: {vars(args)}')

    target_path = Path(args.target)
    if not target_path.is_dir():
        _log.error(f'{target_path} is not a directory')
        return -1

    for candidate_pkg in target_path.iterdir():
        if not candidate_pkg.is_dir() or candidate_pkg.name == 'build':
            continue

        toolbox_doc_strings = _autodoc_toolbox(candidate_pkg)
        if len(toolbox_doc_strings) == 0:
            continue

        toolbox_name = candidate_pkg.stem
        ouput_path = Path(args.output)
        toolbox_output_path = ouput_path / toolbox_name
        toolbox_output_path.mkdir(exist_ok=True, parents=True)
        if not toolbox_output_path.is_dir():
            _log.error(f'Could not create output directory {toolbox_output_path}')
            return -1

        with open(ouput_path / f'{toolbox_name}.rst', 'wt') as f:
            f.write(f'''
{_sanitize(toolbox_name)} Toolbox
{'#'*len(toolbox_name+' Toolbox')}
This is a Matlab toolbox which provides the following packages/namespaces:

.. toctree::
   :maxdepth: 2
   :glob:

   {toolbox_name}/*
''')
        for pkg_name, pkg_doc_string in toolbox_doc_strings.items():
            if pkg_name is not None:
                api_filename = toolbox_output_path / (pkg_name + '.rst')
                if api_filename.is_file() and not args.force:
                    _log.error(f'Output file {api_filename} already exists, use --force to overwrite')
                    return - 1

                with open(api_filename, 'wt') as f:
                    f.write(pkg_doc_string)

    if not args.no_toc:
        raise NotImplementedError('TOC not implemented')

    return 0


def _autodoc_toolbox(path):
    """
    Autogenerates sphinx rst files for a matlab toolbox.

    :param      path:  The path to the toolbox
    """
    toolbox_doc_strings = collections.defaultdict(str)
    toolbox_name = path.stem

    pkg_string = '''
{pkg_sanitized} package
{equal_signs}

.. mat:automodule:: {pkg}
   :members:
   :undoc-members:
   :show-inheritance:

'''

    subpkg_string = '''
{subpkg_sanitized} {subtype}
{dash_signs}
.. mat:automodule:: {toolbox_name}.{subpkg}
   :members:
   :undoc-members:
   :show-inheritance:
'''

    n_sub_packages = 0
    for p in path.rglob('*'):
        if p.is_dir() and p.stem.startswith('+'):
            subtype = 'package'
        elif p.is_dir() and p.stem.startswith('@'):
            subtype = 'class'
        else:
            # TODO: handle scripts ?
            continue

        _log.info(p)

        subpkg_path = str(p.relative_to(path)).split('/')
        subpkg = '.'.join(subpkg_path)

        if subtype == 'package' and len(subpkg_path) == 1:
            _log.info(f'package = {subpkg_path}')
            parent_key = f'{toolbox_name}.{subpkg}'
            n_sub_packages += 1
            toolbox_doc_strings[parent_key] = pkg_string.format(
                pkg=parent_key,
                pkg_sanitized=_sanitize(subpkg),
                equal_signs='=' * len(_sanitize(subpkg)) + '=' * 7)
        else:
            _log.info(f'package = {subpkg_path}')
            parent_key = '.'.join(subpkg_path[:-1])
            parent_key = f'{toolbox_name}.{parent_key}'
            _log.info(f'parent_key = {parent_key} -> {toolbox_doc_strings[parent_key]}')

            toolbox_doc_strings[parent_key] += subpkg_string.format(
                toolbox_name=toolbox_name,
                subpkg=subpkg, subtype=subtype,
                subpkg_sanitized=_sanitize(subpkg),
                dash_signs='-' * (len(subpkg) + len(subtype) + 1)
            )

    if n_sub_packages > 0:
        return toolbox_doc_strings
    else:
        return {}


sys.exit(main(parse_args()))

[joeced]

That's really nice. It's great with some inspiration on what you would like to get as output.
I can't promise anything, but it seems that I will have more time to work on this project in 3-4 weeks

[DutchGFX]

That script seems solid, but does it give you the summary table like autosummary does? To me, being able to separate the module out into different tables inside an RST where each function has the description and variables in the table, is the benefit of autosummary.

I actually have some code where I made my own "mat:autosummary" directive which might be a good starting point. I basically attempted to copy as little code as possible from the autosummary module, while swapping in pieces of sphinxcontrib-matlabdomain where required. It seems to work, but I think it could be cleaned up by someone more familiar with Sphinx and this (sphinxcontrib-matlabdomain). I need to pull them from a different computer, but maybe I can commit them to a new branch for you to see or just attach the files here?

[zouhairm]

I have not used :autosummary before and the script I wrote didn't try to provide the functionality.

[rdzman]

I actually have some code where I made my own "mat:autosummary" directive which might be a good starting point.

I for one would be very interested in seeing this code, if you're able to post it somewhere.

[joeced]

@zouhairm with the new version 0.19.0 it should work better. It will not give you sections and tables, you could try and test if

.. automodule:: .
    :members:

is good enough for you

[changlichun]

Hi @joeced
I have tried modifying the python autosummary ext to got it working for matlab domain. However

• Most of the codes were copied from python autosummary directly. Is it allowed by the license?
• I just made the minimal modification to get it barely running. There must be some known or unknown bugs here.

So, should I start a PR, or I just use it myself?

[DutchGFX]

Hi @joeced I have tried modifying the python autosummary ext to got it working for matlab domain. However

• Most of the codes were copied from python autosummary directly. Is it allowed by the license?
• I just made the minimal modification to get it barely running. There must be some known or unknown bugs here.

So, should I start a PR, or I just use it myself?

This is what I had done originally, which seemed to work OK, but I am not allowed to transfer the files from my work network back onto the public network unfortunately.

I would be happy to help test things if a PR is opened. I think this would be a great feature if we can collectively get it to work. It would basically make this a complete autodoc/autosummary solution for MATLAB

[joeced]

Hi @joeced I have tried modifying the python autosummary ext to got it working for matlab domain. However

* Most of the codes were copied from python autosummary directly. Is it allowed by the license?

* I just made the minimal modification to get it barely running. There must be some known or unknown bugs here.

So, should I start a PR, or I just use it myself?
hi @changlichun
Yes please. PR's are very welcome. It's the two-clause BSD license, so I think it should be OK, as long as you retain the license text in the file.