Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add job metrics #49

Open
cmerrick opened this issue Jun 18, 2019 · 1 comment · May be fixed by #50
Open

Add job metrics #49

cmerrick opened this issue Jun 18, 2019 · 1 comment · May be fixed by #50

Comments

@cmerrick
Copy link
Contributor

Add singer.metrics.job_timer around calls to wait_for_export.

See https://github.com/singer-io/tap-facebook/blob/master/tap_facebook/__init__.py#L488 for an example use of job_timer.

Verify behavior by ensuring that existing tests pass and METRICS messages are emitted in the expected places.

@aiguofer
Copy link

aiguofer commented Jun 22, 2019

Ultimately, wait_for_export calls Client.wait_for_export.

I see 3 ways to do it, each with their pros/cons:

  1. Wrap the body of Client.wait_for_export with the job_timer context manager. This would ensure that if any new code is added that either calls this directly or calls wait_for_export, the job will be tracked. The one potential downside to this approach is if it's important to track how long update_state_with_export_info takes when Client.wait_for_export fails.
  2. Wrap the entire body of wait_for_export. This would capture the time it takes to handle the ExportFailed exception. The issue would be that this won't be captured if something calls Client.wait_for_export directly.
  3. Wrap each call to wait_for_export (and potentially to Client.wait_for_export). This has the same benefits and downside as 2 with the added downside that if something else were to make the call, they'd have to remember to wrap that as well.

I imagine the best way to go here is 1, what do others think?

Also, I'm not sure what would be a good job name. I'm thinking something like: "Export {} for {}".format(export_id, stream_type), but I don't know if there's a general standard used for the names.

aiguofer added a commit to aiguofer/tap-marketo that referenced this issue Jun 27, 2019
@aiguofer aiguofer linked a pull request Jun 27, 2019 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants