Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reports #314

Merged
merged 22 commits into from
Oct 20, 2022
Merged
Show file tree
Hide file tree
Changes from 21 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions report/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
from .report import *
106 changes: 106 additions & 0 deletions report/report.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
__all__ = ["GenerateReportStringJob", "MailJob"]
from sisyphus import *
michelwi marked this conversation as resolved.
Show resolved Hide resolved

import getpass
import gzip
from typing import Dict, Union, Callable, Optional
import pprint
import subprocess

_Report_Type = Dict[str, Union[tk.AbstractPath, str]]


class GenerateReportStringJob(Job):
"""
Job to generate and output a report string
"""

def __init__(
self,
report_values: Union[_Report_Type, Callable],
report_template: Optional[Union[Callable[[_Report_Type], str], str]] = None,
compress: bool = True,
):
"""

:param report_values:
:param report_template:
Atticus1806 marked this conversation as resolved.
Show resolved Hide resolved
:param compress:
"""
self.report_values = report_values
self.report_template = report_template
self.compress = compress

self.out_report = self.output_path(
"report.txt.gz" if self.compress else "report.txt"
)

def tasks(self):
yield Task("run", mini_task=True)

def run(self):

if self.report_template:
if isinstance(self.report_template, str):
report = self.report_template.format(**self.report_values)
else:
report = self.report_template(self.report_values)
elif callable(self.report_values):
report = str(self.report_values())
else:
report = pprint.pformat(self.report_values, width=140)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure about this if...elif...else clause.

why should the report_values be callable?? it is only of type Dict?!

the check on report_template could be restricted to None or even a callable..

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The typing was wrong. So this can be callable (similar to how sisyphus register report has it as a possibilty) , for example some function that returns some string (interpretable) output. You cannot pass the string directly because you dont want to generate the string at graph construction different reasons.
This is quite close to just passing a format function but since sisyphus supported it we decided to also do this here.


fopen = gzip.open if self.compress else open
with fopen(self.out_report.get_path(), "wt") as f:
Atticus1806 marked this conversation as resolved.
Show resolved Hide resolved
f.write(report)


class MailJob(Job):
"""
Job that sends a mail upon completion of an output
"""

def __init__(
self,
result: tk.AbstractPath,
subject: Optional[str] = None,
mail_address: str = getpass.getuser(),
send_contents: bool = False,
):
"""

:param result: graph output that triggers sending the mail
:param subject: Subject of the mail
:param mail_address: Mail address of recipient (default: user)
:param send_contents: send the contents of result in body of the mail
"""
self.result = result
self.subject = subject
self.mail_address = mail_address
self.send_contents = send_contents

self.out_status = self.output_var("out_status")

def tasks(self):
yield Task("run", mini_task=True)

def run(self):

if self.subject is None:
subject = f"Output {str(self.result)} is finished"
else:
subject = self.subject

if self.send_contents:
p1 = subprocess.Popen(
["zcat", "-f", self.result.get_path()], stdout=subprocess.PIPE
)
value = subprocess.check_output(
["mail", "-s", subject, self.mail_address], stdin=p1.stdout
)
else:
out = subprocess.run(
Atticus1806 marked this conversation as resolved.
Show resolved Hide resolved
["mail", "-s", subject, self.mail_address], input="", check=True
)
value = out.returncode
self.out_status.set(value)