Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

introduce ZyteAPITextResponse and ZyteAPIResponse to store raw Zyte Data API Response #10

Merged
merged 30 commits into from
May 30, 2022
Merged

introduce ZyteAPITextResponse and ZyteAPIResponse to store raw Zyte Data API Response #10

merged 30 commits into from
May 30, 2022

Commits on Apr 26, 2022

  1. create ZyteAPITextResponse and ZyteAPIResponse

    @BurnzZ
    BurnzZ committed Apr 26, 2022
    Configuration menu
    Copy the full SHA
    9a83471 View commit details
    Browse the repository at this point in the history

Commits on Apr 27, 2022

  1. update README and CHANGES with notes on new response classes

    @BurnzZ
    BurnzZ committed Apr 27, 2022
    Configuration menu
    Copy the full SHA
    8909473 View commit details
    Browse the repository at this point in the history

Commits on Apr 28, 2022

  1. set the encoding consistently to be 'utf-8'

    @BurnzZ
    BurnzZ committed Apr 28, 2022
    2 Configuration menu
    Copy the full SHA
    d0dc08d View commit details
    Browse the repository at this point in the history

  2. improve example and docs

    @BurnzZ
    BurnzZ committed Apr 28, 2022
    Configuration menu
    Copy the full SHA
    109dbf0 View commit details
    Browse the repository at this point in the history

  3. override replace() to prevent 'zyte_api_response' attribute from bein… 

    …g lost
    @BurnzZ
    BurnzZ committed Apr 28, 2022
    Configuration menu
    Copy the full SHA
    9695880 View commit details
    Browse the repository at this point in the history

  4. fix mypy failures

    @BurnzZ
    BurnzZ committed Apr 28, 2022
    Configuration menu
    Copy the full SHA
    8812a05 View commit details
    Browse the repository at this point in the history

  5. enforce 'utf-8' encoding on Text responses

    @BurnzZ
    BurnzZ committed Apr 28, 2022
    Configuration menu
    Copy the full SHA
    ba64103 View commit details
    Browse the repository at this point in the history

Commits on Apr 29, 2022

  1. update expectation for replacing zyte_api_response attribute

    @BurnzZ
    BurnzZ committed Apr 29, 2022
    Configuration menu
    Copy the full SHA
    84dac7d View commit details
    Browse the repository at this point in the history

  2. update README regarding default params

    @BurnzZ
    BurnzZ committed Apr 29, 2022
    Configuration menu
    Copy the full SHA
    5b83443 View commit details
    Browse the repository at this point in the history

Commits on May 2, 2022

  1. remove 'Content-Encoding' header when returning responses

    @BurnzZ
    BurnzZ committed May 2, 2022
    Configuration menu
    Copy the full SHA
    fb0b412 View commit details
    Browse the repository at this point in the history

  2. remove the ZYTE_API_ENABLED setting

    @BurnzZ
    BurnzZ committed May 2, 2022
    Configuration menu
    Copy the full SHA
    10a4603 View commit details
    Browse the repository at this point in the history

  3. remove zyte_api_default_params in the spider

    @BurnzZ
    BurnzZ committed May 2, 2022
    Configuration menu
    Copy the full SHA
    b7102fa View commit details
    Browse the repository at this point in the history

  4. refactor TestAPI to have single producer of requests and responses

    @BurnzZ
    BurnzZ committed May 2, 2022
    Configuration menu
    Copy the full SHA
    2b4a0fb View commit details
    Browse the repository at this point in the history

Commits on May 3, 2022

  1. implement ZYTE_API_DEFAULT_PARAMS in the settings

    @BurnzZ
    BurnzZ committed May 3, 2022
    Configuration menu
    Copy the full SHA
    97ea1e4 View commit details
    Browse the repository at this point in the history

  2. fix failing tests

    @BurnzZ
    BurnzZ committed May 3, 2022
    Configuration menu
    Copy the full SHA
    5dd1bec View commit details
    Browse the repository at this point in the history

Commits on May 11, 2022

  1. Merge pull request #14 from scrapy-plugins/fix-decompression-error 

    remove 'Content-Encoding' header when returning responses
    @kmike
    kmike authored May 11, 2022
    Configuration menu
    Copy the full SHA
    052d0d6 View commit details
    Browse the repository at this point in the history

Commits on May 19, 2022

  1. rename zyte_api_response into zyte_api

    @BurnzZ
    BurnzZ committed May 19, 2022
    Configuration menu
    Copy the full SHA
    48a4766 View commit details
    Browse the repository at this point in the history

  2. Merge pull request #13 from scrapy-plugins/default-settings 

    introduce new settings: ZYTE_API_DEFAULT_PARAMS
    @BurnzZ
    BurnzZ authored May 19, 2022
    Configuration menu
    Copy the full SHA
    2455bdf View commit details
    Browse the repository at this point in the history

Commits on May 25, 2022

  1. add tests for css/xpath selectors

    @BurnzZ
    BurnzZ committed May 25, 2022
    Configuration menu
    Copy the full SHA
    910085b View commit details
    Browse the repository at this point in the history

Commits on May 26, 2022

  1. enable css/xpath selectors on httpResponseBody

    @BurnzZ
    BurnzZ committed May 26, 2022
    2 Configuration menu
    Copy the full SHA
    e3214d8 View commit details
    Browse the repository at this point in the history

  2. handle empty 'browserHtml' or 'httpResponseBody'

    @BurnzZ
    BurnzZ committed May 26, 2022
    Configuration menu
    Copy the full SHA
    e530053 View commit details
    Browse the repository at this point in the history

Commits on May 27, 2022

  1. Fix typos in docs 

    Co-authored-by: Adrián Chaves <[email protected]>
    @BurnzZ @Gallaecio
    BurnzZ and Gallaecio authored May 27, 2022
    Configuration menu
    Copy the full SHA
    27c7a7d View commit details
    Browse the repository at this point in the history

  2. update how replace() works

    @BurnzZ
    BurnzZ committed May 27, 2022
    Configuration menu
    Copy the full SHA
    5b7cf6f View commit details
    Browse the repository at this point in the history

  3. update README in line with the ZYTE_API_DEFAULT_PARAMS expectations

    @BurnzZ
    BurnzZ committed May 27, 2022
    Configuration menu
    Copy the full SHA
    2adc8a6 View commit details
    Browse the repository at this point in the history

  4. add test case to ensure zyte_api is intact when replacing other attribs

    @BurnzZ
    BurnzZ committed May 27, 2022
    Configuration menu
    Copy the full SHA
    32faf3d View commit details
    Browse the repository at this point in the history

  5. make process_response() private

    @BurnzZ
    BurnzZ committed May 27, 2022
    Configuration menu
    Copy the full SHA
    cec0677 View commit details
    Browse the repository at this point in the history

  6. update tests to ensure other response attribs are not updated on .rep… 

    …lace()
    @BurnzZ
    BurnzZ committed May 27, 2022
    Configuration menu
    Copy the full SHA
    e0865e7 View commit details
    Browse the repository at this point in the history

  7. raise an error if zyte_api is passed to .replace()

    @BurnzZ
    BurnzZ committed May 27, 2022
    Configuration menu
    Copy the full SHA
    34a427f View commit details
    Browse the repository at this point in the history

  8. rename '.zyte_api' attribute as '.raw_api_response'

    @BurnzZ
    BurnzZ committed May 27, 2022
    Configuration menu
    Copy the full SHA
    37a4cc7 View commit details
    Browse the repository at this point in the history

Commits on May 30, 2022

  1. refactor to accept 'True' and '{}' to trigger Zyte API Requests

    @BurnzZ
    BurnzZ committed May 30, 2022
    Configuration menu
    Copy the full SHA
    f5a9bb0 View commit details
    Browse the repository at this point in the history