feat: create task

[JaeAeich]

Summary by Sourcery

Add functionality to create TES tasks and convert them into Kubernetes jobs, including new modules and API endpoints. Refactor existing code for consistency and enhance error handling.

New Features:

• Introduce a new feature to create TES tasks and convert them into Kubernetes jobs, including handling of taskmaster and executor jobs.
• Add a new module for converting TES tasks to Kubernetes jobs, including handling of taskmaster and executor jobs.
• Implement a new API endpoint for creating tasks in the TESK API, which processes task requests and returns responses.

Enhancements:

• Refactor the existing code to use lowercase constants for better consistency and readability.
• Enhance the KubernetesError class to include a method for checking if an object name is duplicated.

[sourcery-ai]

Reviewer's Guide by Sourcery

This pull request implements the creation of a task in the TESK (TES on Kubernetes) system. It introduces new modules for converting TES tasks to Kubernetes jobs, handling task creation requests, and managing the execution of tasks. The changes primarily focus on the backend implementation of task creation, including the conversion of TES task specifications to Kubernetes job configurations.

File-Level Changes

Change	Details	Files
Implement TES task to Kubernetes job conversion	Create TesKubernetesConverter class for converting TES tasks to Kubernetes jobs Implement methods to convert TES executors to Kubernetes jobs Add utility methods for generating Kubernetes job and config map templates	tesk/k8s/converter/converter.py tesk/k8s/converter/template.py
Implement task creation API endpoint	Create CreateTask function in the API controller Implement CreateTesTask class for handling task creation requests Add error handling and retry logic for task creation	tesk/api/ga4gh/tes/controllers.py tesk/api/ga4gh/tes/task/create_task.py
Add data structures for managing Kubernetes resources	Create Job class to represent Kubernetes job objects Implement Task class to manage the relationship between taskmaster and executor jobs Add utility methods for manipulating job and task objects	tesk/k8s/converter/data/job.py tesk/k8s/converter/data/task.py
Update utility functions and constants	Modify get_taskmaster_template function to include new configuration options Update constants and configuration structures Add new utility functions for handling Pydantic models and Kubernetes resources	tesk/utils.py tesk/exceptions.py tesk/api/ga4gh/tes/models.py

---

Tips

• Trigger a new Sourcery review by commenting @sourcery-ai review on the pull request.
• Continue your discussion with Sourcery by replying directly to review comments.
• You can change your review settings at any time by accessing your dashboard:
  • Enable or disable the Sourcery-generated pull request summary or reviewer's guide;
  • Change the review language;
• You can always contact us if you have any questions or feedback.

[sourcery-ai]

Hey @JaeAeich - I've reviewed your changes - here's some feedback:

Overall Comments:

• Consider adding more unit tests for the new converter and task creation logic to ensure robustness and catch potential edge cases.
• The error handling in CreateTesTask is good, but consider adding more detailed logging throughout the conversion process to aid in debugging and monitoring.

Here's what I looked at during the review

• 🟡 General issues: 4 issues found
• 🟡 Security: 1 issue found
• 🟢 Testing: all looks good
• 🟡 Complexity: 1 issue found
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment to tell me if it was helpful.}

[jemaltahir]

Similar comments from PR 206, can be revisited here as well. All in all great work :)

[uniqueg]

Haven't yet managed to review the converter subpackage. Let's see that we get #206 merged and all of the other comments addressed first as things are becoming too complex to review effectively.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.21%. Comparing base (a3e2cc2) to head (5607e62).

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #207   +/-   ##
=======================================
  Coverage   98.21%   98.21%           
=======================================
  Files           8        8           
  Lines         561      561           
=======================================
  Hits          551      551           
  Misses         10       10           

Flag	Coverage Δ
test_unit	98.21% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[uniqueg]

I need to leave it off here for now - it's very late. I'll try my best to go through this, but note that it is 2000 lines or so of complex code. It will take several more hours to go through this.

[uniqueg]

Yeah, that second part is where I would see this going then. Maybe even ending up in FOCA.

I mean, if we already have 21 changed files (not including tests) for the addition of a single controller, we might as well go all the way, right? 😛

[uniqueg]

@JaeAeich: Why is it that with your PRs I often see the entire file changed, even though it's just an iteration over the last time I've viewed? I don't have this with other people, so I guess it's something with your editor or Git flow.

Please have a look at this, because it's really quite annoying. Instead of just focusing on what changed since the last time, I have to look through the entire file again - which is not only not fun, but also holds up the reviews big time, especially when they tend to end up being huge.

And even apart from that, it's also really not good practice in terms of provenance. If this were previously existing code (e.g., maybe some of the old TES code from TES-core still remains), you'd end up being listed as git blame for every line, taking credit and blame for other people's work.

[uniqueg]

Another one of these files that appears entirely new to me of 350+ lines... 🙈

[JaeAeich]

All of these methods are needed for the createTask to work 😅

[uniqueg]

Please also note that some checks are failing and unit tests for the code are entirely missing as of yet. I suppose you can already go ahead addressing/adding these while I finalize the review (it's obviously mostly good code, so I wouldn't expect that a lot of the tests will need to be rewritten).