Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Extraction errors on Stitch #17

Open
s5 opened this issue Jul 9, 2022 · 0 comments
Open

Extraction errors on Stitch #17

s5 opened this issue Jul 9, 2022 · 0 comments

Comments

@s5
Copy link

s5 commented Jul 9, 2022

Hello! I recently tried to run the Impact integration on Stitch, and I'm getting two different (but possibly related?) errors each time I run an extraction. For reference, my Impact API version is V8, but would be willing to upgrade depending on what the tap requires.

I've redacted my Account SID for privacy. I also left the Model ID blank in the config (I couldn't find that anywhere on our account) but I'm not sure if that's needed.

Any idea how I can get this working? Thanks!

Error 1:

2022-07-09 00:19:27,819Z   main - INFO Running tap-impact version 1.0.0 and target-stitch version 3.2.0
2022-07-09 00:19:29,225Z   main - INFO [smart-services] building smart service producer: topic( com.stitchdata.extractionJobStarted ), use_ssl( true )
2022-07-09 00:19:29,228Z   main - INFO [smart-services] building smart service producer: topic( com.stitchdata.extractionJobFinished ), use_ssl( true )
2022-07-09 00:19:29,229Z   main - INFO [smart-services] building smart service producer: topic( com.stitchdata.streamRecordCount ), use_ssl( true )
2022-07-09 00:19:30,245Z   main - INFO [smart-services] event successfully sent to kafka: com.stitchdata.extractionJobStarted [33] at offset None
2022-07-09 00:19:30,248Z   main - INFO Starting tap to discover schemas: tap-env/bin/tap-impact --config /tmp/tap_discover_config.json --discover
2022-07-09 00:19:30,637Z    tap - INFO Starting discover
2022-07-09 00:19:30,665Z    tap - INFO Finished discover
2022-07-09 00:19:30,696Z   main - INFO Tap exited normally.
2022-07-09 00:19:30,700Z   main - INFO Saving list of discovered streams
2022-07-09 00:19:30,720Z   main - INFO Saving structure of stream action_updates (tap_stream_id: action_updates)
2022-07-09 00:19:30,744Z   main - INFO Saving structure of stream report_metadata (tap_stream_id: report_metadata)
2022-07-09 00:19:30,757Z   main - INFO Saving structure of stream media_partner_groups (tap_stream_id: media_partner_groups)
2022-07-09 00:19:30,779Z   main - INFO Saving structure of stream ftp_file_submissions (tap_stream_id: ftp_file_submissions)
2022-07-09 00:19:30,805Z   main - INFO Saving structure of stream phone_numbers (tap_stream_id: phone_numbers)
2022-07-09 00:19:30,821Z   main - INFO Saving structure of stream deals (tap_stream_id: deals)
2022-07-09 00:19:30,843Z   main - INFO Saving structure of stream catalog_items (tap_stream_id: catalog_items)
2022-07-09 00:19:30,872Z   main - INFO Saving structure of stream contacts (tap_stream_id: contacts)
2022-07-09 00:19:30,897Z   main - INFO Saving structure of stream invoices (tap_stream_id: invoices)
2022-07-09 00:19:30,923Z   main - INFO Saving structure of stream campaigns (tap_stream_id: campaigns)
2022-07-09 00:19:30,950Z   main - INFO Saving structure of stream unique_urls (tap_stream_id: unique_urls)
2022-07-09 00:19:30,966Z   main - INFO Saving structure of stream clicks (tap_stream_id: clicks)
2022-07-09 00:19:30,984Z   main - INFO Saving structure of stream api_submissions (tap_stream_id: api_submissions)
2022-07-09 00:19:31,006Z   main - INFO Saving structure of stream media_partners (tap_stream_id: media_partners)
2022-07-09 00:19:31,024Z   main - INFO Saving structure of stream ads (tap_stream_id: ads)
2022-07-09 00:19:31,055Z   main - INFO Saving structure of stream company_information (tap_stream_id: company_information)
2022-07-09 00:19:31,072Z   main - INFO Saving structure of stream tracking_value_requests (tap_stream_id: tracking_value_requests)
2022-07-09 00:19:31,102Z   main - INFO Saving structure of stream reports (tap_stream_id: reports)
2022-07-09 00:19:31,115Z   main - INFO Saving structure of stream exception_lists (tap_stream_id: exception_lists)
2022-07-09 00:19:31,129Z   main - INFO Saving structure of stream exception_list_items (tap_stream_id: exception_list_items)
2022-07-09 00:19:31,146Z   main - INFO Saving structure of stream notes (tap_stream_id: notes)
2022-07-09 00:19:31,168Z   main - INFO Saving structure of stream catalogs (tap_stream_id: catalogs)
2022-07-09 00:19:31,189Z   main - INFO Saving structure of stream actions (tap_stream_id: actions)
2022-07-09 00:19:31,209Z   main - INFO Saving structure of stream action_inquiries (tap_stream_id: action_inquiries)
2022-07-09 00:19:31,231Z   main - INFO Saving structure of stream promo_codes (tap_stream_id: promo_codes)
2022-07-09 00:19:31,254Z   main - INFO Writing catalog to file
2022-07-09 00:19:31,319Z   main - INFO Selected fields for stream ads: minimum_purchase_amount, deal_end_date, description, bogo_get_scope, deal_restricted_media_partners, deal_name, limited_time_start_date, discount_amount, deal_state, bogo_get_discount_amount, restricted_media_partners, bogo_get_name, bogo_buy_name, custom_ad_serving_url, bogo_buy_quantity, deal_description, landing_page, promo_code_tracking, language, third_party_servable_ad_creative_height, discount_currency, limited_time_end_date, gift, synch_ads_promo_codes, deal_categories, iab_ad_unit, campaign_id, mobile_ready, uri, banner_alternative_tag, campaign_name, name, bogo_get_image_url, rebate_amount, coupon_link_name, get_html_code_type, discount_percent_range_end, maximum_savings_amount, deal_start_date, bogo_get_quantity, link_text, id, discount_maximum_percent, phone_tracking, bogo_get_discount_percent, top_seller, deal_type, bogo_get_discount_type, season, restricted_media_partner_groups, discount_percent, maximum_savings_currency, minimum_purchase_amount_currency, bogo_buy_scope, customisation_charge, bogo_buy_image_url, coupon_allow_custom_promo_code, discount_percent_range_start, ad_type, discount_type, rebate_currency, bogo_get_discount_currency, deal_scope, deal_default_promo_code, allow_deep_linking, purchase_limit_quantity, labels, deal_id, ad_code_template, deal_products, third_party_servable_ad_creative_width, deal_restricted_media_partner_groups
2022-07-09 00:19:31,319Z   main - INFO Selected fields for stream company_information: timezone, billing_address, commercial_contact, indirect_tax_number, primary_phone_number_country, corporate_address, currency, industry, organization_type, secondary_phone_number_country, website, minimum_contact_rating, primary_phone_number, registered_for_indirect_tax, financial_contact, secondary_phone_number, uri, ein_ssn_foreign_tax_id, company_name, technical_contact
2022-07-09 00:19:31,319Z   main - INFO Selected fields for stream unique_urls: campaign_name, url, date_last_assigned, media_partner_name, state, date_last_released, id, date_created, media_partner_id, campaign_id, landing_page, uri
2022-07-09 00:19:31,319Z   main - INFO Selected fields for stream media_partner_groups: id, name, media_partners, uri, campaign_id
2022-07-09 00:19:31,319Z   main - INFO Selected fields for stream catalog_items: original_price, shipping_height, mobile_url, is_parent, description, shipping_length, text2, manufacturer, money3, age_group, currency, catalog_id, shipping_rate, product_bid, original_format_category_id, expiration_date, url, category, money1, numeric2, estimated_ship_date, catalog_item_id, parent_sku, launch_date, weight, uri, money2, shipping_width, name, parent_name, manufacturing_cost, shipping_label, mpn, shipping_length_unit, text3, current_price, weight_unit, asin, material, gtin_type, shipping_weight, gender, text1, shipping_weight_unit, numeric1, multi_pack, size_unit, additional_image_urls, gtin, adult, stock_availability, pattern, condition, size, colors, inventory, numeric3, labels, original_format_category, bullets, image_url
2022-07-09 00:19:31,320Z   main - INFO Selected fields for stream phone_numbers: id, international_number, last_date_assigned, last_date_released, date_created, number, uri, country
2022-07-09 00:19:31,320Z   main - INFO Selected fields for stream media_partners: timezone, postal_code, name, country_state, mp_value1, id, rating, campaigns, relationship_state, address_line1, currency, groups, contact, primary_promotional_method, partner_type, promoting_countries, phone_number, promotional_methods, address_line2, description, mp_value3, country, website, city, mp_value2, date_created, uri, state
2022-07-09 00:19:31,320Z   main - INFO Selected fields for stream campaigns: name, tracking_domain, display_future_ads, identity_collapsing, state, display_servicing_agency, company_contacts, id, rating, list_in_marketplace, short_description, long_description, promo_code_tracking, third_party_impression_pixel, shipping_regions, trademark_bidding, resources, categories, view_through_crediting, gift_card_payouts, direct_tracking_settings, search_keywords, mobile_ready_ads, impression_tracking, gateway_tracking_settings, site_definition
2022-07-09 00:19:31,320Z   main - INFO Selected fields for stream deals: minimum_purchase_amount, name, bogo_get_image_url, rebate_amount, start_date, restricted_media_partner_groups, bogo_get_discount_currency, discount_percent_range_end, maximum_savings_amount, state, bogo_get_scope, discount_amount, bogo_get_quantity, id, restricted_media_partners, discount_maximum_percent, bogo_get_name, bogo_get_discount_percent, bogo_buy_name, bogo_buy_quantity, scope, discount_percent, maximum_savings_currency, minimum_purchase_amount_currency, type, bogo_get_discount_type, bogo_buy_image_url, discount_percent_range_start, default_promo_code, purchase_limit_quantity, discount_currency, rebate_currency, end_date, gift, synch_ads_promo_codes, categories, bogo_buy_scope, bogo_get_discount_amount, description, campaign_id, products, uri, discount_type
2022-07-09 00:19:31,320Z   main - INFO Selected fields for stream tracking_value_requests: minimum_purchase_amount, campaign_name, deal_end_date, unique_urls, deal_restricted_media_partners, state, deal_name, bogo_get_scope, discount_amount, phone_numbers, deal_state, bogo_get_discount_amount, bogo_get_name, bogo_buy_name, bogo_buy_quantity, deal_description, date_completed, promo_codes, processing_comments, discount_currency, date_placed, gift, synch_ads_promo_codes, discount_percent_range_end, media_partner_id, campaign_id, deal_default_promo_code, discount_percent, uri, additional_instructions, bogo_get_image_url, rebate_amount, media_partner_name, maximum_savings_amount, deal_start_date, bogo_get_quantity, id, discount_maximum_percent, bogo_get_discount_percent, request_type, deal_type, maximum_savings_currency, minimum_purchase_amount_currency, bogo_buy_scope, bogo_get_discount_type, bogo_buy_image_url, deal_categories, discount_percent_range_start, purchase_limit_quantity, discount_type, rebate_currency, bogo_get_discount_currency, number_requested, deal_scope, date_needed, deal_id, deal_products, deal_restricted_media_partner_groups
2022-07-09 00:19:31,320Z   main - INFO Selected fields for stream exception_list_items: id, name, value, created_date, match_mode, uri, list_id
2022-07-09 00:19:31,320Z   main - INFO Selected fields for stream exception_lists: id, campaign_id, uri, created_date, deactivation_date, name, number_of_items, state, action_trackers, items_uri, type
2022-07-09 00:19:31,320Z   main - INFO Selected fields for stream api_submissions: error_type, batch_id, type, event_code, status, action_tracker_id, completion_date, error_reason, account_id, payload, media_partner_id, campaign_id, order_id, uri, submission_date
2022-07-09 00:19:31,320Z   main - INFO Selected fields for stream reports: id, name, description, run_uri, meta_data_uri
2022-07-09 00:19:31,320Z   main - INFO Selected fields for stream promo_codes: minimum_purchase_amount, start_date, deal_end_date, deal_restricted_media_partners, state, deal_name, bogo_get_scope, discount_amount, deal_state, bogo_get_discount_amount, bogo_get_name, bogo_buy_name, bogo_buy_quantity, deal_description, discount_currency, end_date, gift, synch_ads_promo_codes, campaign_id, discount_percent, uri, campaign_name, bogo_get_image_url, rebate_amount, discount_percent_range_end, maximum_savings_amount, deal_start_date, bogo_get_quantity, code, id, discount_maximum_percent, bogo_buy_scope, credit_policy, assigned_partner_id, deal_id, deal_type, assigned_partner_name, maximum_savings_currency, minimum_purchase_amount_currency, type, bogo_get_discount_type, bogo_buy_image_url, deal_categories, discount_percent_range_start, purchase_limit_quantity, discount_type, rebate_currency, bogo_get_discount_currency, deal_scope, deal_default_promo_code, created_date, bogo_get_discount_percent, deal_products, deal_restricted_media_partner_groups
2022-07-09 00:19:31,320Z   main - INFO Selected fields for stream contacts: work_phone_number, email_address, title, cellphone_number_country, first_name, id, accounts, work_phone_number_country, cellphone_number, profile_image, last_name, labels, campaign_id, uri
2022-07-09 00:19:31,320Z   main - INFO Selected fields for stream notes: id, creator, attachments, type, creation_date, media_name, content, campaign_id, modification_date, uri, media_id
2022-07-09 00:19:31,320Z   main - INFO Selected fields for stream action_updates: clearing_date, ad_id, delta_payout, sku, action_id, caller_id, state, media_partner_id, id, customer_status, quantity, update_date, currency, customer_id, shared_id, disposition, state_detail_description, locking_date, state_detail, action_uri, action_tracker_id, action_date, delta_amount, campaign_id, order_id, category, uri
2022-07-09 00:19:31,320Z   main - INFO Selected fields for stream invoices: pdf, id, line_items, created_date, media_name, currency, total_vat_amount, total_amount, media_id
2022-07-09 00:19:31,320Z   main - INFO Selected fields for stream clicks: campaign_name, ad_id, channel, bid_keyword, payout, ad_group, deal_name, id, keyword, customer_country", deal_type, event_date, search_text, shared_id, landing_page_url, traffic_source, referring_url, unique_click, media_id, referring_domain, product_sku, customer_area, cpc_bid, device_family, ip_address, ad_type, traffic_category, customer_city, device_type, customer_region, browser, ad_campaign, deal_scope, os, media_name, campaign_id, match_type, ad_name, profile_id
2022-07-09 00:19:31,320Z   main - INFO Selected fields for stream ftp_file_submissions: account_id, batch_id, errors_uri, completion_date, status, file_name, total_errors, uri, submission_date, type, total_records
2022-07-09 00:19:31,320Z   main - INFO Selected fields for stream report_metadata: id, filters, uri, name, description, attributes, run_uri
2022-07-09 00:19:31,320Z   main - INFO Selected fields for stream actions: campaign_name, media_partner_id, ad_id, delta_payout, referring_date, payout, media_partner_name, client_cost, state, intended_payout, action_tracker_name, delta_amount, note, id, customer_status, cleared_date, currency, customer_country, shared_id, promo_code, referring_domain, customer_area, amount, locking_date, ip_address, customer_city, referring_type, action_tracker_id, event_date, creation_date, customer_region, caller_id, campaign_id, order_id, customer_post_code, uri, customer_id, intended_amount
2022-07-09 00:19:31,320Z   main - INFO Selected fields for stream action_inquiries: campaign_name, expected_payout, auto_approval_date, media_partner_name, action_id, id, transaction_date, comments, transaction_amount, final_payout, tracking_link, resolution_status, action_uri, resolution_date, creation_date, inquiry_type, reject_reason, media_partner_id, campaign_id, order_id, uri, resolution_deadline_date
2022-07-09 00:19:31,320Z   main - INFO Selected fields for stream catalogs: id, campaign_id, date_last_updated, filename, upload_method, name, number_of_items, advertiser_id, uri, items_uri, status
2022-07-09 00:19:31,326Z   main - INFO Current state: {"bookmarks": {"api_submissions": "2022-07-08T21:45:07.000000Z", "action_updates": "2022-07-08T18:38:26.000000Z", "invoices": "2022-07-02T12:34:04.000000Z", "clicks": "2022-07-08T22:50:02.000000Z", "actions": "2022-07-08T18:26:29.000000Z", "action_inquiries": "2022-06-16T21:06:22.000000Z"}, "currently_syncing": "catalogs"}
2022-07-09 00:19:31,327Z   main - INFO Starting tap: tap-env/bin/tap-impact --config /tmp/tap_config.json --properties /tmp/properties.json --catalog /tmp/catalog.json --state /tmp/tap_state.json
2022-07-09 00:19:31,332Z   main - INFO Starting target: target-env/bin/target-stitch --config /tmp/target_config.json
2022-07-09 00:19:31,730Z    tap - INFO last/currently syncing stream: catalogs
2022-07-09 00:19:31,730Z    tap - INFO selected_streams: ['catalogs', 'actions', 'action_inquiries', 'promo_codes', 'action_updates', 'report_metadata', 'media_partner_groups', 'ftp_file_submissions', 'phone_numbers', 'deals', 'catalog_items', 'contacts', 'invoices', 'campaigns', 'unique_urls', 'clicks', 'api_submissions', 'media_partners', 'ads', 'company_information', 'tracking_value_requests', 'reports', 'exception_lists', 'exception_list_items', 'notes']
2022-07-09 00:19:31,730Z    tap - INFO START Syncing: promo_codes
2022-07-09 00:19:31,731Z    tap - INFO URL for Stream promo_codes: https://api.impact.com/Advertisers/****/PromoCodes.json?PageSize=1000
2022-07-09 00:19:31,799Z    tap - INFO HTTP request to "promo_codes" endpoint took 0.065s, returned status code 200
2022-07-09 00:19:31,799Z    tap - INFO FINISHED Syncing: promo_codes, total_records: 0
2022-07-09 00:19:31,799Z    tap - INFO START Syncing: ftp_file_submissions
2022-07-09 00:19:31,800Z    tap - INFO URL for Stream ftp_file_submissions: https://api.impact.com/Advertisers/****/FTPFileSubmissions.json?PageSize=1000
2022-07-09 00:19:31,846Z target - INFO Using batch_size_prefernces of {'user_batch_size_preference': None, 'batch_size_preference': None, 'full_table_streams': []}
2022-07-09 00:19:31,847Z   main - INFO State update: adding bookmarks.action_inquiries = "2022-06-16T21:06:22.000000Z"
2022-07-09 00:19:31,847Z   main - INFO State update: adding bookmarks.action_updates = "2022-07-08T18:38:26.000000Z"
2022-07-09 00:19:31,847Z   main - INFO State update: adding bookmarks.actions = "2022-07-08T18:26:29.000000Z"
2022-07-09 00:19:31,847Z   main - INFO State update: adding bookmarks.api_submissions = "2022-07-08T21:45:07.000000Z"
2022-07-09 00:19:31,847Z   main - INFO State update: adding bookmarks.clicks = "2022-07-08T22:50:02.000000Z"
2022-07-09 00:19:31,847Z   main - INFO State update: adding bookmarks.invoices = "2022-07-02T12:34:04.000000Z"
2022-07-09 00:19:31,847Z   main - INFO State update: adding currently_syncing = "promo_codes"
2022-07-09 00:19:31,853Z    tap - INFO HTTP request to "ftp_file_submissions" endpoint took 0.054s, returned status code 200
2022-07-09 00:19:31,853Z    tap - INFO FINISHED Syncing: ftp_file_submissions, total_records: 0
2022-07-09 00:19:31,853Z    tap - INFO START Syncing: reports
2022-07-09 00:19:31,854Z    tap - INFO URL for Stream reports: https://api.impact.com/Advertisers/****/Reports.json?PageSize=1000
2022-07-09 00:19:31,860Z   main - INFO State update: changing currently_syncing from "promo_codes" to "ftp_file_submissions"
2022-07-09 00:19:31,868Z   main - INFO State update: changing currently_syncing from "ftp_file_submissions" to "reports"
2022-07-09 00:19:32,144Z    tap - INFO HTTP request to "reports" endpoint took 0.290s, returned status code 200
2022-07-09 00:19:32,148Z    tap - INFO Stream: reports, Missing key id in record: {'description': 'Shows action listing as category level based on locking date of the action', 'category': 'Custom', 'api_accessible': False, 'run_uri': '/secure/advertiser/report/viewReport.report?id=6282', 'name': 'Action Listing (Category Level) by Locking Date'}
2022-07-09 00:19:32,149Z    tap - CRITICAL 
2022-07-09 00:19:32,150Z    tap - Traceback (most recent call last):
2022-07-09 00:19:32,150Z    tap -   File "tap-env/bin/tap-impact", line 33, in <module>
2022-07-09 00:19:32,150Z    tap -     sys.exit(load_entry_point('tap-impact==1.0.0', 'console_scripts', 'tap-impact')())
2022-07-09 00:19:32,150Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/singer/utils.py", line 229, in wrapped
2022-07-09 00:19:32,150Z    tap -     return fnc(*args, **kwargs)
2022-07-09 00:19:32,151Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_impact/__init__.py", line 50, in main
2022-07-09 00:19:32,151Z    tap -     state=state)
2022-07-09 00:19:32,151Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_impact/sync.py", line 421, in sync
2022-07-09 00:19:32,151Z    tap -     selected_streams=selected_streams)
2022-07-09 00:19:32,151Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_impact/sync.py", line 267, in sync_endpoint
2022-07-09 00:19:32,151Z    tap -     raise RuntimeError
2022-07-09 00:19:32,151Z    tap - RuntimeError
2022-07-09 00:19:32,180Z target - INFO Requests complete, stopping loop
2022-07-09 00:19:32,252Z   main - INFO Target exited normally with status 0
2022-07-09 00:19:33,819Z   main - INFO [smart-services] event successfully sent to kafka: com.stitchdata.extractionJobFinished [5] at offset None
2022-07-09 00:19:33,821Z   main - INFO No tunnel subprocess to tear down
2022-07-09 00:19:33,821Z   main - INFO Exit status is: Discovery succeeded. Tap failed with code 1. Target succeeded.

Error 2:

2022-07-09 00:11:58,959Z   main - INFO Running tap-impact version 1.0.0 and target-stitch version 3.2.0
2022-07-09 00:12:00,114Z   main - INFO [smart-services] building smart service producer: topic( com.stitchdata.extractionJobStarted ), use_ssl( true )
2022-07-09 00:12:00,117Z   main - INFO [smart-services] building smart service producer: topic( com.stitchdata.extractionJobFinished ), use_ssl( true )
2022-07-09 00:12:00,117Z   main - INFO [smart-services] building smart service producer: topic( com.stitchdata.streamRecordCount ), use_ssl( true )
2022-07-09 00:12:01,141Z   main - INFO [smart-services] event successfully sent to kafka: com.stitchdata.extractionJobStarted [18] at offset None
2022-07-09 00:12:01,144Z   main - INFO Starting tap to discover schemas: tap-env/bin/tap-impact --config /tmp/tap_discover_config.json --discover
2022-07-09 00:12:01,580Z    tap - INFO Starting discover
2022-07-09 00:12:01,613Z    tap - INFO Finished discover
2022-07-09 00:12:01,648Z   main - INFO Tap exited normally.
2022-07-09 00:12:01,652Z   main - INFO Saving list of discovered streams
2022-07-09 00:12:01,678Z   main - INFO Saving structure of stream promo_codes (tap_stream_id: promo_codes)
2022-07-09 00:12:01,713Z   main - INFO Saving structure of stream contacts (tap_stream_id: contacts)
2022-07-09 00:12:01,746Z   main - INFO Saving structure of stream campaigns (tap_stream_id: campaigns)
2022-07-09 00:12:01,791Z   main - INFO Saving structure of stream phone_numbers (tap_stream_id: phone_numbers)
2022-07-09 00:12:01,815Z   main - INFO Saving structure of stream actions (tap_stream_id: actions)
2022-07-09 00:12:01,854Z   main - INFO Saving structure of stream tracking_value_requests (tap_stream_id: tracking_value_requests)
2022-07-09 00:12:01,895Z   main - INFO Saving structure of stream exception_list_items (tap_stream_id: exception_list_items)
2022-07-09 00:12:01,918Z   main - INFO Saving structure of stream company_information (tap_stream_id: company_information)
2022-07-09 00:12:01,947Z   main - INFO Saving structure of stream notes (tap_stream_id: notes)
2022-07-09 00:12:01,986Z   main - INFO Saving structure of stream report_metadata (tap_stream_id: report_metadata)
2022-07-09 00:12:02,017Z   main - INFO Saving structure of stream action_inquiries (tap_stream_id: action_inquiries)
2022-07-09 00:12:02,053Z   main - INFO Saving structure of stream api_submissions (tap_stream_id: api_submissions)
2022-07-09 00:12:02,080Z   main - INFO Saving structure of stream clicks (tap_stream_id: clicks)
2022-07-09 00:12:02,110Z   main - INFO Saving structure of stream reports (tap_stream_id: reports)
2022-07-09 00:12:02,162Z   main - INFO Saving structure of stream media_partners (tap_stream_id: media_partners)
2022-07-09 00:12:02,183Z   main - INFO Saving structure of stream media_partner_groups (tap_stream_id: media_partner_groups)
2022-07-09 00:12:02,209Z   main - INFO Saving structure of stream deals (tap_stream_id: deals)
2022-07-09 00:12:02,260Z   main - INFO Saving structure of stream catalogs (tap_stream_id: catalogs)
2022-07-09 00:12:02,286Z   main - INFO Saving structure of stream action_updates (tap_stream_id: action_updates)
2022-07-09 00:12:02,334Z   main - INFO Saving structure of stream exception_lists (tap_stream_id: exception_lists)
2022-07-09 00:12:02,385Z   main - INFO Saving structure of stream invoices (tap_stream_id: invoices)
2022-07-09 00:12:02,424Z   main - INFO Saving structure of stream catalog_items (tap_stream_id: catalog_items)
2022-07-09 00:12:02,468Z   main - INFO Saving structure of stream ads (tap_stream_id: ads)
2022-07-09 00:12:02,523Z   main - INFO Saving structure of stream ftp_file_submissions (tap_stream_id: ftp_file_submissions)
2022-07-09 00:12:02,547Z   main - INFO Saving structure of stream unique_urls (tap_stream_id: unique_urls)
2022-07-09 00:12:02,586Z   main - INFO Writing catalog to file
2022-07-09 00:12:02,653Z   main - INFO Selected fields for stream actions: promo_code, customer_area, customer_post_code, referring_domain, ad_id, id, currency, intended_payout, shared_id, creation_date, action_tracker_id, note, media_partner_id, event_date, amount, customer_city, customer_id, action_tracker_name, uri, locking_date, campaign_name, customer_region, referring_date, payout, intended_amount, delta_payout, state, campaign_id, delta_amount, customer_status, caller_id, customer_country, cleared_date, ip_address, referring_type, media_partner_name, order_id, client_cost
2022-07-09 00:12:02,653Z   main - INFO Selected fields for stream campaigns: mobile_ready_ads, list_in_marketplace, gift_card_payouts, company_contacts, third_party_impression_pixel, id, gateway_tracking_settings, direct_tracking_settings, display_servicing_agency, resources, impression_tracking, shipping_regions, identity_collapsing, name, categories, site_definition, promo_code_tracking, long_description, display_future_ads, trademark_bidding, tracking_domain, search_keywords, short_description, rating, view_through_crediting, state
2022-07-09 00:12:02,653Z   main - INFO Selected fields for stream unique_urls: date_created, campaign_id, uri, id, media_partner_name, media_partner_id, date_last_released, landing_page, url, date_last_assigned, state, campaign_name
2022-07-09 00:12:02,653Z   main - INFO Selected fields for stream action_updates: category, action_uri, ad_id, id, currency, state_detail, shared_id, locking_date, action_tracker_id, media_partner_id, action_id, quantity, customer_id, uri, order_id, sku, state_detail_description, delta_payout, disposition, clearing_date, campaign_id, delta_amount, update_date, caller_id, customer_status, action_date, state
2022-07-09 00:12:02,653Z   main - INFO Selected fields for stream catalog_items: expiration_date, catalog_item_id, category, current_price, shipping_length, is_parent, currency, mpn, numeric3, pattern, colors, image_url, weight, shipping_height, text2, uri, parent_sku, asin, gender, condition, additional_image_urls, age_group, money1, bullets, text1, material, numeric1, size, inventory, weight_unit, shipping_weight, parent_name, shipping_width, shipping_length_unit, money3, numeric2, estimated_ship_date, shipping_label, original_price, manufacturing_cost, launch_date, text3, shipping_rate, gtin, shipping_weight_unit, size_unit, money2, multi_pack, manufacturer, name, mobile_url, catalog_id, stock_availability, product_bid, labels, original_format_category, original_format_category_id, url, adult, description, gtin_type
2022-07-09 00:12:02,653Z   main - INFO Selected fields for stream phone_numbers: id, country, date_created, last_date_released, last_date_assigned, number, uri, international_number
2022-07-09 00:12:02,653Z   main - INFO Selected fields for stream ftp_file_submissions: submission_date, status, account_id, total_records, batch_id, errors_uri, completion_date, type, total_errors, uri, file_name
2022-07-09 00:12:02,653Z   main - INFO Selected fields for stream media_partner_groups: campaign_id, id, uri, name, media_partners
2022-07-09 00:12:02,653Z   main - INFO Selected fields for stream promo_codes: maximum_savings_currency, discount_type, bogo_get_name, deal_name, bogo_buy_quantity, id, start_date, bogo_buy_name, discount_amount, type, discount_currency, minimum_purchase_amount_currency, uri, campaign_name, campaign_id, bogo_get_discount_currency, code, deal_description, discount_percent_range_start, credit_policy, deal_start_date, maximum_savings_amount, minimum_purchase_amount, deal_categories, gift, deal_id, bogo_get_scope, bogo_buy_image_url, deal_state, deal_default_promo_code, rebate_amount, bogo_get_discount_amount, bogo_get_discount_percent, deal_products, bogo_get_image_url, synch_ads_promo_codes, created_date, assigned_partner_id, purchase_limit_quantity, bogo_get_discount_type, discount_percent_range_end, bogo_buy_scope, deal_restricted_media_partner_groups, deal_restricted_media_partners, deal_type, bogo_get_quantity, discount_maximum_percent, end_date, discount_percent, rebate_currency, assigned_partner_name, deal_scope, state, deal_end_date
2022-07-09 00:12:02,653Z   main - INFO Selected fields for stream catalogs: campaign_id, upload_method, status, name, advertiser_id, date_last_updated, id, uri, items_uri, number_of_items, filename
2022-07-09 00:12:02,653Z   main - INFO Selected fields for stream exception_lists: campaign_id, type, deactivation_date, created_date, name, action_trackers, id, uri, items_uri, number_of_items, state
2022-07-09 00:12:02,653Z   main - INFO Selected fields for stream api_submissions: submission_date, error_type, batch_id, completion_date, campaign_id, media_partner_id, action_tracker_id, status, account_id, type, event_code, error_reason, uri, payload, order_id
2022-07-09 00:12:02,653Z   main - INFO Selected fields for stream tracking_value_requests: maximum_savings_currency, discount_type, bogo_get_name, date_needed, deal_name, bogo_buy_quantity, date_completed, id, media_partner_id, bogo_buy_name, discount_amount, discount_currency, deal_description, date_placed, uri, campaign_name, number_requested, campaign_id, bogo_get_discount_currency, processing_comments, minimum_purchase_amount_currency, discount_percent_range_start, deal_start_date, maximum_savings_amount, minimum_purchase_amount, deal_categories, gift, deal_id, bogo_get_scope, bogo_buy_image_url, deal_state, deal_default_promo_code, rebate_amount, request_type, bogo_get_discount_amount, unique_urls, bogo_get_discount_percent, phone_numbers, bogo_get_image_url, synch_ads_promo_codes, promo_codes, purchase_limit_quantity, bogo_get_discount_type, discount_percent_range_end, bogo_buy_scope, deal_restricted_media_partners, deal_products, deal_type, additional_instructions, bogo_get_quantity, discount_maximum_percent, discount_percent, rebate_currency, media_partner_name, deal_restricted_media_partner_groups, deal_scope, state, deal_end_date
2022-07-09 00:12:02,653Z   main - INFO Selected fields for stream report_metadata: filters, name, description, attributes, id, uri, run_uri
2022-07-09 00:12:02,653Z   main - INFO Selected fields for stream invoices: total_amount, pdf, created_date, total_vat_amount, currency, line_items, id, media_name, media_id
2022-07-09 00:12:02,653Z   main - INFO Selected fields for stream company_information: financial_contact, minimum_contact_rating, primary_phone_number_country, secondary_phone_number_country, primary_phone_number, commercial_contact, timezone, currency, billing_address, corporate_address, website, ein_ssn_foreign_tax_id, organization_type, company_name, industry, technical_contact, indirect_tax_number, secondary_phone_number, uri, registered_for_indirect_tax
2022-07-09 00:12:02,653Z   main - INFO Selected fields for stream clicks: keyword, ad_group, customer_country", match_type, referring_domain, os, id, ad_id, browser, shared_id, traffic_category, event_date, device_family, channel, customer_city, media_name, ad_campaign, product_sku, customer_region, unique_click, payout, deal_name, cpc_bid, traffic_source, campaign_id, profile_id, customer_area, ad_name, deal_scope, search_text, ip_address, device_type, referring_url, ad_type, campaign_name, landing_page_url, bid_keyword, deal_type, media_id
2022-07-09 00:12:02,653Z   main - INFO Selected fields for stream ads: maximum_savings_currency, discount_type, bogo_get_name, deal_name, bogo_buy_quantity, id, link_text, promo_code_tracking, bogo_buy_name, custom_ad_serving_url, purchase_limit_quantity, language, discount_currency, deal_description, banner_alternative_tag, phone_tracking, coupon_link_name, restricted_media_partner_groups, restricted_media_partners, uri, campaign_name, get_html_code_type, allow_deep_linking, campaign_id, bogo_get_discount_currency, minimum_purchase_amount_currency, limited_time_start_date, iab_ad_unit, coupon_allow_custom_promo_code, top_seller, ad_code_template, deal_type, ad_type, deal_restricted_media_partners, deal_categories, gift, deal_id, bogo_get_scope, bogo_buy_image_url, deal_start_date, deal_default_promo_code, rebate_amount, bogo_get_discount_amount, bogo_get_discount_percent, landing_page, bogo_get_image_url, limited_time_end_date, season, discount_amount, bogo_get_discount_type, discount_percent_range_end, bogo_buy_scope, third_party_servable_ad_creative_width, deal_restricted_media_partner_groups, discount_percent_range_start, deal_products, minimum_purchase_amount, deal_state, name, third_party_servable_ad_creative_height, description, mobile_ready, bogo_get_quantity, discount_maximum_percent, synch_ads_promo_codes, labels, discount_percent, rebate_currency, customisation_charge, deal_scope, maximum_savings_amount, deal_end_date
2022-07-09 00:12:02,653Z   main - INFO Selected fields for stream deals: maximum_savings_currency, discount_percent_range_end, discount_type, bogo_get_name, bogo_buy_image_url, bogo_buy_quantity, bogo_get_discount_amount, id, bogo_get_discount_percent, start_date, bogo_buy_name, discount_percent_range_start, purchase_limit_quantity, bogo_get_image_url, type, discount_currency, minimum_purchase_amount_currency, discount_amount, bogo_get_discount_type, restricted_media_partner_groups, restricted_media_partners, uri, description, default_promo_code, name, categories, campaign_id, products, bogo_get_quantity, discount_maximum_percent, end_date, rebate_amount, discount_percent, bogo_get_discount_currency, rebate_currency, scope, bogo_buy_scope, synch_ads_promo_codes, maximum_savings_amount, minimum_purchase_amount, bogo_get_scope, state, gift
2022-07-09 00:12:02,653Z   main - INFO Selected fields for stream notes: type, attachments, creator, modification_date, content, campaign_id, media_name, id, uri, creation_date, media_id
2022-07-09 00:12:02,654Z   main - INFO Selected fields for stream exception_list_items: match_mode, created_date, name, value, id, uri, list_id
2022-07-09 00:12:02,654Z   main - INFO Selected fields for stream action_inquiries: inquiry_type, action_uri, id, media_partner_name, creation_date, tracking_link, comments, media_partner_id, action_id, uri, auto_approval_date, order_id, resolution_status, campaign_id, transaction_date, resolution_deadline_date, transaction_amount, final_payout, resolution_date, expected_payout, reject_reason, campaign_name
2022-07-09 00:12:02,654Z   main - INFO Selected fields for stream contacts: campaign_id, work_phone_number_country, last_name, cellphone_number, accounts, work_phone_number, id, profile_image, first_name, labels, cellphone_number_country, uri, email_address, title
2022-07-09 00:12:02,654Z   main - INFO Selected fields for stream reports: meta_data_uri, id, description, run_uri, name
2022-07-09 00:12:02,654Z   main - INFO Selected fields for stream media_partners: id, campaigns, address_line2, date_created, timezone, country, currency, partner_type, website, rating, groups, mp_value3, mp_value1, phone_number, promotional_methods, relationship_state, country_state, primary_promotional_method, postal_code, name, uri, city, contact, address_line1, mp_value2, promoting_countries, description, state
2022-07-09 00:12:02,658Z   main - INFO Current state: {"bookmarks": {"actions": "2022-07-08T18:26:29.000000Z", "api_submissions": "2022-07-08T21:45:07.000000Z", "action_updates": "2022-07-08T18:38:26.000000Z", "action_inquiries": "2022-06-16T21:06:22.000000Z", "clicks": "2022-07-08T22:50:02.000000Z"}, "currently_syncing": "reports"}
2022-07-09 00:12:02,659Z   main - INFO Starting tap: tap-env/bin/tap-impact --config /tmp/tap_config.json --properties /tmp/properties.json --catalog /tmp/catalog.json --state /tmp/tap_state.json
2022-07-09 00:12:02,662Z   main - INFO Starting target: target-env/bin/target-stitch --config /tmp/target_config.json
2022-07-09 00:12:03,122Z    tap - INFO last/currently syncing stream: reports
2022-07-09 00:12:03,122Z    tap - INFO selected_streams: ['reports', 'media_partners', 'media_partner_groups', 'deals', 'catalogs', 'action_updates', 'exception_lists', 'invoices', 'catalog_items', 'ads', 'ftp_file_submissions', 'unique_urls', 'promo_codes', 'contacts', 'campaigns', 'phone_numbers', 'actions', 'tracking_value_requests', 'exception_list_items', 'company_information', 'notes', 'report_metadata', 'action_inquiries', 'api_submissions', 'clicks']
2022-07-09 00:12:03,122Z    tap - INFO START Syncing: deals
2022-07-09 00:12:03,123Z    tap - INFO URL for Stream deals: https://api.impact.com/Advertisers/****/Deals.json?PageSize=1000
2022-07-09 00:12:03,146Z target - INFO Using batch_size_prefernces of {'user_batch_size_preference': None, 'batch_size_preference': None, 'full_table_streams': []}
2022-07-09 00:12:03,147Z   main - INFO State update: adding bookmarks.action_inquiries = "2022-06-16T21:06:22.000000Z"
2022-07-09 00:12:03,147Z   main - INFO State update: adding bookmarks.action_updates = "2022-07-08T18:38:26.000000Z"
2022-07-09 00:12:03,147Z   main - INFO State update: adding bookmarks.actions = "2022-07-08T18:26:29.000000Z"
2022-07-09 00:12:03,147Z   main - INFO State update: adding bookmarks.api_submissions = "2022-07-08T21:45:07.000000Z"
2022-07-09 00:12:03,147Z   main - INFO State update: adding bookmarks.clicks = "2022-07-08T22:50:02.000000Z"
2022-07-09 00:12:03,147Z   main - INFO State update: adding currently_syncing = "deals"
2022-07-09 00:12:03,226Z target - INFO Serializing batch with 3 messages for table deals
2022-07-09 00:12:03,227Z    tap - INFO HTTP request to "deals" endpoint took 0.098s, returned status code 200
2022-07-09 00:12:03,227Z target - INFO Sending batch of 9706 bytes to https://api.stitchdata.com/v2/import/batch
2022-07-09 00:12:03,299Z   main - INFO State update: changing currently_syncing from "deals" to "ftp_file_submissions"
2022-07-09 00:12:03,363Z   main - INFO State update: changing currently_syncing from "ftp_file_submissions" to "invoices"
2022-07-09 00:12:04,210Z    tap - INFO [smart-services] event successfully sent to kafka: com.stitchdata.streamRecordCount [9] at offset None
2022-07-09 00:12:04,674Z    tap - INFO [smart-services] event successfully sent to kafka: com.stitchdata.streamRecordCount [2] at offset None
2022-07-09 00:12:04,675Z    tap - INFO replicated 3 records from "deals" endpoint
2022-07-09 00:12:04,675Z    tap - INFO Stream deals, batch processed 3 records
2022-07-09 00:12:04,675Z    tap - INFO Synced Stream: deals, page: 1, 0 to 3 of total records: 3
2022-07-09 00:12:04,675Z    tap - INFO FINISHED Syncing: deals, total_records: 3
2022-07-09 00:12:04,675Z    tap - INFO START Syncing: ftp_file_submissions
2022-07-09 00:12:04,675Z    tap - INFO URL for Stream ftp_file_submissions: https://api.impact.com/Advertisers/****/FTPFileSubmissions.json?PageSize=1000
2022-07-09 00:12:04,675Z    tap - INFO HTTP request to "ftp_file_submissions" endpoint took 0.135s, returned status code 200
2022-07-09 00:12:04,675Z    tap - INFO FINISHED Syncing: ftp_file_submissions, total_records: 0
2022-07-09 00:12:04,675Z    tap - INFO START Syncing: invoices
2022-07-09 00:12:04,675Z    tap - INFO URL for Stream invoices: https://api.impact.com/Advertisers/****/Invoices.json?PageSize=1000&StartDate=2021-07-08T00:00:00Z
2022-07-09 00:12:04,675Z    tap - INFO HTTP request to "invoices" endpoint took 0.970s, returned status code 200
2022-07-09 00:12:04,742Z   main - INFO State update: adding bookmarks.invoices = "2022-07-02T12:34:04.000000Z"
2022-07-09 00:12:04,742Z   main - INFO State update: changing currently_syncing from "invoices" to "catalogs"
2022-07-09 00:12:05,136Z    tap - INFO [smart-services] event successfully sent to kafka: com.stitchdata.streamRecordCount [4] at offset None
2022-07-09 00:12:05,136Z    tap - INFO replicated 194 records from "invoices" endpoint
2022-07-09 00:12:05,136Z    tap - INFO Stream invoices, batch processed 194 records
2022-07-09 00:12:05,136Z    tap - INFO Write state for stream: invoices, value: 2022-07-02T12:34:04.000000Z
2022-07-09 00:12:05,136Z    tap - INFO Synced Stream: invoices, page: 1, 0 to 194 of total records: 194
2022-07-09 00:12:05,136Z    tap - INFO FINISHED Syncing: invoices, total_records: 194
2022-07-09 00:12:05,136Z    tap - INFO START Syncing: catalogs
2022-07-09 00:12:05,137Z    tap - INFO URL for Stream catalogs: https://api.impact.com/Advertisers/****/Catalogs.json?PageSize=1000
2022-07-09 00:12:05,137Z    tap - INFO HTTP request to "catalogs" endpoint took 0.214s, returned status code 401
2022-07-09 00:12:05,137Z    tap - ERROR ERROR 401: {"Status":"ERROR","Message":"Bad credentials"}, REASON: Unauthorized
2022-07-09 00:12:05,137Z    tap - CRITICAL 401 Client Error: Unauthorized for url: https://api.impact.com/Advertisers/****/Catalogs.json?PageSize=1000
2022-07-09 00:12:05,137Z    tap - Traceback (most recent call last):
2022-07-09 00:12:05,137Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_impact/client.py", line 80, in raise_for_error
2022-07-09 00:12:05,137Z    tap -     response.raise_for_status()
2022-07-09 00:12:05,137Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/requests/models.py", line 940, in raise_for_status
2022-07-09 00:12:05,137Z    tap -     raise HTTPError(http_error_msg, response=self)
2022-07-09 00:12:05,137Z    tap - requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://api.impact.com/Advertisers/****/Catalogs.json?PageSize=1000
2022-07-09 00:12:05,137Z    tap - 
2022-07-09 00:12:05,137Z    tap - During handling of the above exception, another exception occurred:
2022-07-09 00:12:05,137Z    tap - 
2022-07-09 00:12:05,137Z    tap - Traceback (most recent call last):
2022-07-09 00:12:05,137Z    tap -   File "tap-env/bin/tap-impact", line 33, in <module>
2022-07-09 00:12:05,137Z    tap -     sys.exit(load_entry_point('tap-impact==1.0.0', 'console_scripts', 'tap-impact')())
2022-07-09 00:12:05,138Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/singer/utils.py", line 229, in wrapped
2022-07-09 00:12:05,138Z    tap -     return fnc(*args, **kwargs)
2022-07-09 00:12:05,138Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_impact/__init__.py", line 50, in main
2022-07-09 00:12:05,138Z    tap -     state=state)
2022-07-09 00:12:05,138Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_impact/sync.py", line 421, in sync
2022-07-09 00:12:05,138Z    tap -     selected_streams=selected_streams)
2022-07-09 00:12:05,138Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_impact/sync.py", line 208, in sync_endpoint
2022-07-09 00:12:05,138Z    tap -     endpoint=stream_name)
2022-07-09 00:12:05,138Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_impact/client.py", line 208, in get
2022-07-09 00:12:05,138Z    tap -     return self.request('GET', path=path, **kwargs)
2022-07-09 00:12:05,138Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/backoff/_sync.py", line 94, in retry
2022-07-09 00:12:05,138Z    tap -     ret = target(*args, **kwargs)
2022-07-09 00:12:05,138Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/singer/utils.py", line 95, in wrapper
2022-07-09 00:12:05,138Z    tap -     return func(*args, **kwargs)
2022-07-09 00:12:05,138Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_impact/client.py", line 203, in request
2022-07-09 00:12:05,138Z    tap -     raise_for_error(response)
2022-07-09 00:12:05,138Z    tap -   File "/code/orchestrator/tap-env/lib/python3.5/site-packages/tap_impact/client.py", line 96, in raise_for_error
2022-07-09 00:12:05,138Z    tap -     raise ImpactError(error)
2022-07-09 00:12:05,138Z    tap - tap_impact.client.ImpactError: 401 Client Error: Unauthorized for url: https://api.impact.com/Advertisers/****/Catalogs.json?PageSize=1000
2022-07-09 00:12:05,139Z   main - INFO Target exited normally with status 0
2022-07-09 00:12:05,143Z target - INFO replicated 3 records from "deals" endpoint
2022-07-09 00:12:05,143Z target - I/O operation on closed file.
2022-07-09 00:12:06,673Z   main - INFO [smart-services] event successfully sent to kafka: com.stitchdata.extractionJobFinished [31] at offset None
2022-07-09 00:12:06,677Z   main - INFO No tunnel subprocess to tear down
2022-07-09 00:12:06,678Z   main - INFO Exit status is: Discovery succeeded. Tap failed with code 1 and error message: "401 Client Error: Unauthorized for url: https://api.impact.com/Advertisers/****/Catalogs.json?PageSize=1000". Target succeeded.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@s5