Error when syncing from 2 streams clicks and catalogs

[toandm]

Hi @jeffhuth-bytecode , I'm using Stitch to get data from Impact using this Singer tap. During the replication, this error happened (redacted Advertiser ID and Campaign ID). I'm getting the tables clicks, catalogs

clicks table error log is below (similar to issue #18):

2023-01-04 01:23:04,767Z    tap - INFO START Sync for Stream: clicks, parent_stream: campaigns, parent_id: xxxxx
2023-01-04 01:23:04,768Z target - INFO Serializing batch with 1928 messages for table action_updates
2023-01-04 01:23:04,768Z    tap - INFO Stream: clicks, Syncing bookmark_date = 2022-05-17
2023-01-04 01:23:04,768Z    tap - INFO URL for Stream clicks: https://api.impact.com/Advertisers/xxxxx/Campaigns/xxxx/Clicks.json?PageSize=1000&Date=2022-05-17
2023-01-04 01:23:04,822Z target - INFO Sending batch of 2103810 bytes to https://api.stitchdata.com/v2/import/batch
2023-01-04 01:23:04,824Z target - INFO replicated 1928 records from "action_updates" endpoint
2023-01-04 01:23:04,869Z    tap - INFO HTTP request to "clicks" endpoint took 0.100s, returned status code 404
2023-01-04 01:23:04,870Z    tap - ERROR ERROR 404: {"Status":"ERROR","Message":"Support for this endpoint has ended and has been replaced with the new /ClickExport endpoint. Please use /Advertisers/xxxxx/Programs/xxxxx/ClickExport.json?Date=2022-05-17&PageSize=1000 to access your data. Please also review the API documentation if you require additional information."}, REASON: Not Found
2023-01-04 01:23:04,870Z    tap - CRITICAL 404 Client Error: Not Found for url: https://api.impact.com/Advertisers/xxxxx/Campaigns/xxxxx/Clicks.json?PageSize=1000&Date=2022-05-17
2023-01-04 01:23:04,870Z    tap - Traceback (most recent call last):
2023-01-04 01:23:04,871Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/tap_impact/client.py", line 80, in raise_for_error
2023-01-04 01:23:04,871Z    tap -     response.raise_for_status()
2023-01-04 01:23:04,871Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/requests/models.py", line 940, in raise_for_status
2023-01-04 01:23:04,871Z    tap -     raise HTTPError(http_error_msg, response=self)
2023-01-04 01:23:04,871Z    tap - requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://api.impact.com/Advertisers/xxxxx/Campaigns/xxxxx/Clicks.json?PageSize=1000&Date=2022-05-17
2023-01-04 01:23:04,871Z    tap - 
2023-01-04 01:23:04,871Z    tap - During handling of the above exception, another exception occurred:
2023-01-04 01:23:04,871Z    tap - 
2023-01-04 01:23:04,871Z    tap - Traceback (most recent call last):
2023-01-04 01:23:04,871Z    tap -   File "/code/orchestrator/tap-env/bin/tap-impact", line 33, in <module>
2023-01-04 01:23:04,871Z    tap -     sys.exit(load_entry_point('tap-impact==1.0.0', 'console_scripts', 'tap-impact')())
2023-01-04 01:23:04,871Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/singer/utils.py", line 229, in wrapped
2023-01-04 01:23:04,871Z    tap -     return fnc(*args, **kwargs)
2023-01-04 01:23:04,872Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/tap_impact/__init__.py", line 47, in main
2023-01-04 01:23:04,872Z    tap -     sync(client=client,
2023-01-04 01:23:04,872Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/tap_impact/sync.py", line 406, in sync
2023-01-04 01:23:04,872Z    tap -     total_records = sync_endpoint(
2023-01-04 01:23:04,872Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/tap_impact/sync.py", line 318, in sync_endpoint
2023-01-04 01:23:04,872Z    tap -     child_total_records = sync_endpoint(
2023-01-04 01:23:04,872Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/tap_impact/sync.py", line 204, in sync_endpoint
2023-01-04 01:23:04,872Z    tap -     data = client.get(
2023-01-04 01:23:04,872Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/tap_impact/client.py", line 208, in get
2023-01-04 01:23:04,872Z    tap -     return self.request('GET', path=path, **kwargs)
2023-01-04 01:23:04,872Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/backoff/_sync.py", line 94, in retry
2023-01-04 01:23:04,872Z    tap -     ret = target(*args, **kwargs)
2023-01-04 01:23:04,872Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/singer/utils.py", line 95, in wrapper
2023-01-04 01:23:04,872Z    tap -     return func(*args, **kwargs)
2023-01-04 01:23:04,872Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/tap_impact/client.py", line 203, in request
2023-01-04 01:23:04,872Z    tap -     raise_for_error(response)
2023-01-04 01:23:04,872Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/tap_impact/client.py", line 96, in raise_for_error
2023-01-04 01:23:04,872Z    tap -     raise ImpactError(error)
2023-01-04 01:23:04,872Z    tap - tap_impact.client.ImpactError: 404 Client Error: Not Found for url: https://api.impact.com/Advertisers/xxxxx/Campaigns/xxxxx/Clicks.json?PageSize=1000&Date=2022-05-17
2023-01-04 01:23:05,658Z target - INFO Requests complete, stopping loop
2023-01-04 01:23:05,740Z   main - INFO Target exited normally with status 0
2023-01-04 01:23:05,743Z   main - INFO No tunnel subprocess to tear down
2023-01-04 01:23:05,743Z   main - INFO Exit status is: Discovery succeeded. Tap failed with code 1 and error message: "404 Client Error: Not Found for url: https://api.impact.com/Advertisers/xxxxx/Campaigns/xxxxx/Clicks.json?PageSize=1000&Date=2022-05-17". Target succeeded.

catalogs table error:

2023-01-04 01:32:49,905Z    tap - INFO START Syncing: catalogs
2023-01-04 01:32:49,905Z    tap - INFO URL for Stream catalogs: https://api.impact.com/Advertisers/xxxxx/Catalogs.json?PageSize=1000
2023-01-04 01:32:49,906Z   main - INFO State update: changing currently_syncing from "campaigns" to "catalogs"
2023-01-04 01:32:49,960Z    tap - INFO HTTP request to "catalogs" endpoint took 0.054s, returned status code 400
2023-01-04 01:32:49,960Z    tap - ERROR ERROR 400: {
2023-01-04 01:32:49,960Z    tap -   "Status": "ERROR",
2023-01-04 01:32:49,960Z    tap -   "Message": "This API version is no longer supported. Please upgrade to one of the supported API versions: v10"
2023-01-04 01:32:49,960Z    tap - }, REASON: Bad Request
2023-01-04 01:32:49,961Z    tap - CRITICAL 400 Client Error: Bad Request for url: https://api.impact.com/Advertisers/xxxxx/Catalogs.json?PageSize=1000
2023-01-04 01:32:49,961Z    tap - Traceback (most recent call last):
2023-01-04 01:32:49,961Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/tap_impact/client.py", line 80, in raise_for_error
2023-01-04 01:32:49,961Z    tap -     response.raise_for_status()
2023-01-04 01:32:49,961Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/requests/models.py", line 940, in raise_for_status
2023-01-04 01:32:49,961Z    tap -     raise HTTPError(http_error_msg, response=self)
2023-01-04 01:32:49,961Z    tap - requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://api.impact.com/Advertisers/xxxxx/Catalogs.json?PageSize=1000
2023-01-04 01:32:49,962Z    tap - 
2023-01-04 01:32:49,962Z    tap - During handling of the above exception, another exception occurred:
2023-01-04 01:32:49,962Z    tap - 
2023-01-04 01:32:49,962Z    tap - Traceback (most recent call last):
2023-01-04 01:32:49,962Z    tap -   File "/code/orchestrator/tap-env/bin/tap-impact", line 33, in <module>
2023-01-04 01:32:49,962Z    tap -     sys.exit(load_entry_point('tap-impact==1.0.0', 'console_scripts', 'tap-impact')())
2023-01-04 01:32:49,962Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/singer/utils.py", line 229, in wrapped
2023-01-04 01:32:49,962Z    tap -     return fnc(*args, **kwargs)
2023-01-04 01:32:49,962Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/tap_impact/__init__.py", line 47, in main
2023-01-04 01:32:49,962Z    tap -     sync(client=client,
2023-01-04 01:32:49,962Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/tap_impact/sync.py", line 406, in sync
2023-01-04 01:32:49,962Z    tap -     total_records = sync_endpoint(
2023-01-04 01:32:49,962Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/tap_impact/sync.py", line 204, in sync_endpoint
2023-01-04 01:32:49,962Z    tap -     data = client.get(
2023-01-04 01:32:49,962Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/tap_impact/client.py", line 208, in get
2023-01-04 01:32:49,962Z    tap -     return self.request('GET', path=path, **kwargs)
2023-01-04 01:32:49,962Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/backoff/_sync.py", line 94, in retry
2023-01-04 01:32:49,962Z    tap -     ret = target(*args, **kwargs)
2023-01-04 01:32:49,962Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/singer/utils.py", line 95, in wrapper
2023-01-04 01:32:49,962Z    tap -     return func(*args, **kwargs)
2023-01-04 01:32:49,962Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/tap_impact/client.py", line 203, in request
2023-01-04 01:32:49,962Z    tap -     raise_for_error(response)
2023-01-04 01:32:49,962Z    tap -   File "/code/orchestrator/tap-env/lib/python3.9/site-packages/tap_impact/client.py", line 96, in raise_for_error
2023-01-04 01:32:49,962Z    tap -     raise ImpactError(error)
2023-01-04 01:32:49,963Z    tap - tap_impact.client.ImpactError: 400 Client Error: Bad Request for url: https://api.impact.com/Advertisers/xxxxx/Catalogs.json?PageSize=1000
2023-01-04 01:32:49,987Z target - INFO Requests complete, stopping loop
2023-01-04 01:32:50,023Z   main - INFO Target exited normally with status 0
2023-01-04 01:32:50,025Z   main - INFO No tunnel subprocess to tear down
2023-01-04 01:32:50,026Z   main - INFO Exit status is: Discovery succeeded. Tap failed with code 1 and error message: "400 Client Error: Bad Request for url: https://api.impact.com/Advertisers/xxxxx/Catalogs.json?PageSize=1000". Target succeeded.

Other than those 2 tables, the rest are working well. Please have a look.