Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add support for analysis of source code/scripted languages #1080

Draft
wants to merge 51 commits into
base: master
Choose a base branch
from
Draft

add support for analysis of source code/scripted languages #1080

wants to merge 51 commits into from

Commits on Jul 19, 2022

  1. Added initial capa control flow for scripts in C#.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    bbd3f70 View commit details
    Browse the repository at this point in the history

  2. Implemented some further basic TreeSitter Extractor-related concepts … 

    …such as byte-range address.
    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    8173397 View commit details
    Browse the repository at this point in the history

  3. Modified mypy config file to ignore tree-sitter's missing exports.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    428f6bc View commit details
    Browse the repository at this point in the history

  4. Implemented core tree sitter engine component with C# queries that se… 

    …rves as an interface to the language-specific tree-sitter queries.
    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    a6d7ba2 View commit details
    Browse the repository at this point in the history

  5. Implemented script global extraction handlers (mostly wrapping existi… 

    …ng language-independent extractors).
    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    80bf78b View commit details
    Browse the repository at this point in the history

  6. Reworked format parsing to align better with the rest of capa logic.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    cf3dc7e View commit details
    Browse the repository at this point in the history

  7. Implemented a large part of the C# functionality; refactored the Tree… 

    …SitterEngine class.
    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    9d7f575 View commit details
    Browse the repository at this point in the history

  8. Added function-level feature extraction.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    3d4b4ec View commit details
    Browse the repository at this point in the history

  9. Bug fixes and code refactoring of the Tree Sitter extractor.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    eca7ead View commit details
    Browse the repository at this point in the history

  10. Added tree_sitter to requirements in setup.py.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    5fd953f View commit details
    Browse the repository at this point in the history

  11. Added tests for TreeSitterExtractorEngine initialization, new object … 

    …and function definition parsing for a pure C# sample.
    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    1f79db9 View commit details
    Browse the repository at this point in the history

  12. Added more TreeSitterExtractorEngine tests for pure C#.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    a58bc0b View commit details
    Browse the repository at this point in the history

  13. Added last remaining tests for the TreeSitterExtractorEngine class an… 

    …d fixed bugs found in the process.
    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    5ddb8ba View commit details
    Browse the repository at this point in the history

  14. Reverted yielding only non-empty strings in order to stay consistent … 

    …and not introduce unspecified rule-exceptions.
    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    31e2fb9 View commit details
    Browse the repository at this point in the history

  15. Removing functions that should not be used in tree-sitter extractor (… 

    …default to the base extractor level).
    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    5bf3f18 View commit details
    Browse the repository at this point in the history

  16. Modifying extraction of global statements to omit local function decl… 

    …arations.
    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    a4529fc View commit details
    Browse the repository at this point in the history

  17. Added script language feature to freeze.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    d5de9a1 View commit details
    Browse the repository at this point in the history

  18. Added test cases for TS Extractor.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    6c10458 View commit details
    Browse the repository at this point in the history

  19. Refactored query bindings.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    9bd9824 View commit details
    Browse the repository at this point in the history

  20. Added support for template parsing.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    2594849 View commit details
    Browse the repository at this point in the history

  21. Added support for HTML parsing.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    619ed94 View commit details
    Browse the repository at this point in the history

  22. Implemented the necessary modifications to support embedded templates… 

    …/html: aspx.
    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    5e23802 View commit details
    Browse the repository at this point in the history

  23. Added more buildings to build; minor style improvement.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    5d83e8d View commit details
    Browse the repository at this point in the history

  24. Further refactored the Tree-sitter queries and fixed minor template e… 

    …ngine bugs.
    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    9570523 View commit details
    Browse the repository at this point in the history

  25. Refactored extractor engine tests and began adding new template tests.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    7c5e6e3 View commit details
    Browse the repository at this point in the history

  26. Added new tests for embedded template testing and refactored a few al… 

    …ready implemented extractor engine tests.
    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    1e0326a View commit details
    Browse the repository at this point in the history

  27. Bug fixes in extractor and HTML Tree-sitter engine.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    ca1939f View commit details
    Browse the repository at this point in the history

  28. Fixed important namespace-parsing bugs.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    d7ab2db View commit details
    Browse the repository at this point in the history

  29. Further improvement to namespace parsing, including default namespace… 

    …s for aspx; introducing TSFunctionInner dataclass to store both the node and the name of a function to make test-case creation easier.
    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    5cfbecc View commit details
    Browse the repository at this point in the history

  30. Added more tests and a few minor bug fixes.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    26cc1bc View commit details
    Browse the repository at this point in the history

  31. Added language-specific integer parsing.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    2a9e76f View commit details
    Browse the repository at this point in the history

  32. Fixed an important bug in FileOffsetRangeAddress comparison method.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    672ca71 View commit details
    Browse the repository at this point in the history

  33. Added more ASPX tests.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    ca426ca View commit details
    Browse the repository at this point in the history

  34. Fixed the capa control flow to fully support capa scripts.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    fd80277 View commit details
    Browse the repository at this point in the history

  35. Major changes: switching imports and function names to properties, st… 

    …reamlining queries, introducing language toolkit class to help move all additional language specific methods to the sample place.
    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    d0c4acb View commit details
    Browse the repository at this point in the history

  36. Fixed property-extraction bugs.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    ad31d83 View commit details
    Browse the repository at this point in the history

  37. Added few more test cases.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    e52a9b3 View commit details
    Browse the repository at this point in the history

  38. Minor style improvements.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    b27713b View commit details
    Browse the repository at this point in the history

  39. Removed deprecated parse_integer.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    b2df2b0 View commit details
    Browse the repository at this point in the history

  40. Added more tests; fixed integer parsing related bugs.

    @adamstorek
    adamstorek committed Jul 19, 2022
    Configuration menu
    Copy the full SHA
    a0379a6 View commit details
    Browse the repository at this point in the history

Commits on Jul 20, 2022

  1. Fixing address range bug; refactoring and cleanup.

    @adamstorek
    adamstorek committed Jul 20, 2022
    Configuration menu
    Copy the full SHA
    eeecb63 View commit details
    Browse the repository at this point in the history

  2. Incorporated more tests.

    @adamstorek
    adamstorek committed Jul 20, 2022
    Configuration menu
    Copy the full SHA
    cebc5e1 View commit details
    Browse the repository at this point in the history

Commits on Jul 26, 2022

  1. Added support for Python.

    @adamstorek
    adamstorek committed Jul 26, 2022
    Configuration menu
    Copy the full SHA
    d7dcc94 View commit details
    Browse the repository at this point in the history

Commits on Jul 29, 2022

  1. Added more python test cases; fixed a number of python bugs; further … 

    …refactored language toolkit code; added extraction of global constants.
    @adamstorek
    adamstorek committed Jul 29, 2022
    Configuration menu
    Copy the full SHA
    32dc5ff View commit details
    Browse the repository at this point in the history

Commits on Aug 2, 2022

  1. Implemented namespace aliasing; further refactored the codebase.

    @adamstorek
    adamstorek committed Aug 2, 2022
    Configuration menu
    Copy the full SHA
    5e85a6e View commit details
    Browse the repository at this point in the history

Commits on Aug 3, 2022

  1. Refactored/simplified parts of the codebase to improve readability; a… 

    …ddressed most of the GH pull request comments/suggestions.
    @adamstorek
    adamstorek committed Aug 3, 2022
    Configuration menu
    Copy the full SHA
    614900f View commit details
    Browse the repository at this point in the history

  2. Implemented script language auto-detection.

    @adamstorek
    adamstorek committed Aug 3, 2022
    Configuration menu
    Copy the full SHA
    bb08181 View commit details
    Browse the repository at this point in the history

  3. Removed a spurious import.

    @adamstorek
    adamstorek committed Aug 3, 2022
    Configuration menu
    Copy the full SHA
    1fd9d4a View commit details
    Browse the repository at this point in the history

Commits on Aug 5, 2022

  1. Added more test cases; moved script language feature to global featur… 

    …es in order to make rules clearer; refactored the codebase to address the latest PR comments/suggestions.
    @adamstorek
    adamstorek committed Aug 5, 2022
    Configuration menu
    Copy the full SHA
    7ba978f View commit details
    Browse the repository at this point in the history

Commits on Aug 10, 2022

  1. Introduced auto-detection to template-script parsing, builtins namesp… 

    …ace, and modified tests.
    @adamstorek
    adamstorek committed Aug 10, 2022
    Configuration menu
    Copy the full SHA
    25cf09b View commit details
    Browse the repository at this point in the history

Commits on Aug 12, 2022

  1. Attempted to implement the class extraction as specified last Friday … 

    …(passes all test cases but by no means perfect); further clean up, especially of the signatures; synced with new Python test cases.
    @adamstorek
    adamstorek committed Aug 12, 2022
    Configuration menu
    Copy the full SHA
    e693573 View commit details
    Browse the repository at this point in the history