Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restore the googleadservices.com rule #865

Closed
fmarier opened this issue May 27, 2022 · 2 comments · Fixed by #912
Closed

Restore the googleadservices.com rule #865

fmarier opened this issue May 27, 2022 · 2 comments · Fixed by #912
Assignees
Labels
Debounce Debouncing list related

Comments

@fmarier
Copy link
Member

fmarier commented May 27, 2022

I took another look at the original Slack thread that caused us to remove the *.googleadservices.com debouncing rule.

I believe that the initial rule that we removed was fine (*.googleadservices.com), though maybe it should have been www.googleadservices.com. Removing that rule solved the initial breakage report we received, but covered up the two other broken rules involved in the redirect chain.

These two rules were introduced in #726 and should be removed before we restore the googleadservices.com one:

@fmarier
Copy link
Member Author

fmarier commented May 27, 2022

Here's the full flow:

  1. Original URL: https://www.googleadservices.com/pagead/aclk?sa=L&ai=CpwMy_pA7YueBHMWV9AOwxrvgBPXF1eJm38Xi3KwMvLjk5vMNCAQQASgLYMmO8IbIo6AZoAGJg8X-A8gBB8gD2CCqBFxP0BVaXHBw0bTsk4SdtAz6eTEkpO5bWMb0JJ9zZZjWgQ4Tv1T4gTUaNmPp2H5322pGRBg-y3P7jmsjL3XDZO_AP6DTNRPxMNYb_zx-BXL8fbGHuo08uK48cCAPJcAEvoPWoo8DiAW0qZjfKcAFBaAGJoAH3_y6AYgHAZAHAagHpr4bqAe5mrECqAfz0RuoB-7SG6gH_5yxAqgHytwbqAfNo7ECoAjA8j2wCAHACAHSCAwQAiCEATICgkA6AQCaCVJodHRwczovL3d3dy5zd2VldHdhdGVyLmNvbS9zdG9yZS9kZXRhaWwvRDQ1WTgtLW1hcnRpbi1kLTQ1LWFjb3VzdGljLWd1aXRhci1uYXR1cmFssQmJ4KBqN-lXPbkJXM7iZAP2Jnn4CQHQCyrgCwGqDAIIAbgMAegMBuAS5vSK6fz787uAAYIUBAgBEgDQFQHpFY-4wNPDibHa8BWodvoVBUQ0NVk4gRa7V5yQTETaeYgWUJgWAbgWwPm8sSPAFsD5vLEj6hYECKzqIPgWAYAXAZIXCRIHCAEQAxjBAg&ctype=5&ved=2ahUKEwjawK7Jlt32AhX8Mn0KHaQUC5wQww96BAgBED0&cid=CAASFeRo9mEa81pQI8r5S7W87ZZ_MHUHVQ&dblrd=1&sival=AF15MEBme8-y7XM2ojdjUz9fPC5ilL9_jZ_4bwjLYo_PY-5CKrjcK2pH8B8y4aUUpseGeC5QdxGD4HpvRTSoslTwUXuVGrOPyUT5FHwykqwCMJv-7fHk3NWTk_ArH_nz0bfn1pJtHAWHFJAw8820rzgIbB48F3ZbiaBTfESZY29iIzi1ngqq6mDaMpINh3X7MuQq3L90K_rr&sig=AOD64_2IpBc4vty-6Tc066z47dc0zFEJ0g&adurl=https://clickserve.dartsearch.net/link/click%3Flid%3D92700057522574369%26ds_s_kwgid%3D58700006368161508%26ds_s_inventory_feed_id%3D97700000007215323%26ds_a_cid%3D2001181%26ds_a_caid%3D11205481652%26ds_a_agid%3D107178459582%26ds_a_fiid%3D%26ds_a_lid%3Dpla-477762231356%26ds_a_extid%3D%26%26ds_e_adid%3D470980643466%26ds_e_matchtype%3Dsearch%26ds_e_device%3Dc%26ds_e_network%3Dg%26ds_e_product_group_id%3D477762231356%26ds_e_product_id%3DD45Y8%26ds_e_product_merchant_id%3D15144%26ds_e_product_country%3DUS%26ds_e_product_language%3Den%26ds_e_product_channel%3Donline%26ds_e_product_store_id%3D%26ds_url_v%3D2%26ds_dest_url%3Dhttps://www.sweetwater.com/store/detail/D45Y8--martin-d-45-acoustic-guitar-natural%3Futm_source%3Dgoogle%26utm_medium%3Dorganicpla%26mrkgadid%3D%26mrkgcl%3D28%26mrkgen%3Dgpla%26mrkgbflag%3D0%26mrkgcat%3D%26acctid%3D21700000001645388%26dskeywordid%3D92700057522574369%26lid%3D92700057522574369%26ds_s_kwgid%3D58700006368161508%26ds_s_inventory_feed_id%3D97700000007215323%26dsproductgroupid%3D477762231356%26product_id%3DD45Y8%26prodctry%3DUS%26prodlang%3Den%26channel%3Donline%26storeid%3D%26device%3Dc%26network%3Dg%26matchtype%3D%26adpos%3Dlargenumber%26locationid%3D1014080%26creative%3D470980643466%26targetid%3Dpla-477762231356%26campaignid%3D11205481652%26awsearchcpc%3D1%26gclsrc%3Daw.ds%26
  2. Gets redirected to: https://clickserve.dartsearch.net/link/click?lid=92700057522574369&ds_s_kwgid=58700006368161508&ds_s_inventory_feed_id=97700000007215323&ds_a_cid=2001181&ds_a_caid=11205481652&ds_a_agid=107178459582&ds_a_fiid=&ds_a_lid=pla-477762231356&ds_a_extid=&&ds_e_adid=470980643466&ds_e_matchtype=search&ds_e_device=c&ds_e_network=g&ds_e_product_group_id=477762231356&ds_e_product_id=D45Y8&ds_e_product_merchant_id=15144&ds_e_product_country=US&ds_e_product_language=en&ds_e_product_channel=online&ds_e_product_store_id=&ds_url_v=2&ds_dest_url=https://www.sweetwater.com/store/detail/D45Y8--martin-d-45-acoustic-guitar-natural?utm_source=google&utm_medium=organicpla&mrkgadid=&mrkgcl=28&mrkgen=gpla&mrkgbflag=0&mrkgcat=&acctid=21700000001645388&dskeywordid=92700057522574369&lid=92700057522574369&ds_s_kwgid=58700006368161508&ds_s_inventory_feed_id=97700000007215323&dsproductgroupid=477762231356&product_id=D45Y8&prodctry=US&prodlang=en&channel=online&storeid=&device=c&network=g&matchtype=&adpos=largenumber&locationid=1014080&creative=470980643466&targetid=pla-477762231356&campaignid=11205481652&awsearchcpc=1&gclsrc=aw.ds&gclid=CjwKCAjwiuuRBhBvEiwAFXKaNJw_rPjzQFK0QeGjUec0VuCQzl7QPQwRtI6XsFD_DTB9NPRW0mSb_RoCseYQAvD_BwE
  3. Then the gclid param is removed: https://clickserve.dartsearch.net/link/click?lid=92700057522574369&ds_s_kwgid=58700006368161508&ds_s_inventory_feed_id=97700000007215323&ds_a_cid=2001181&ds_a_caid=11205481652&ds_a_agid=107178459582&ds_a_fiid=&ds_a_lid=pla-477762231356&ds_a_extid=&&ds_e_adid=470980643466&ds_e_matchtype=search&ds_e_device=c&ds_e_network=g&ds_e_product_group_id=477762231356&ds_e_product_id=D45Y8&ds_e_product_merchant_id=15144&ds_e_product_country=US&ds_e_product_language=en&ds_e_product_channel=online&ds_e_product_store_id=&ds_url_v=2&ds_dest_url=https://www.sweetwater.com/store/detail/D45Y8--martin-d-45-acoustic-guitar-natural?utm_source=google&utm_medium=organicpla&mrkgadid=&mrkgcl=28&mrkgen=gpla&mrkgbflag=0&mrkgcat=&acctid=21700000001645388&dskeywordid=92700057522574369&lid=92700057522574369&ds_s_kwgid=58700006368161508&ds_s_inventory_feed_id=97700000007215323&dsproductgroupid=477762231356&product_id=D45Y8&prodctry=US&prodlang=en&channel=online&storeid=&device=c&network=g&matchtype=&adpos=largenumber&locationid=1014080&creative=470980643466&targetid=pla-477762231356&campaignid=11205481652&awsearchcpc=1&gclsrc=aw.ds
  4. If we don't debounce this one, it gets server-redirected to: https://ad.doubleclick.net/ddm/clk/449830104;253494287;m;u=ds&sv1=57522574369&sv2=3307222181951115&sv3=3423267513265722070&gclid=CMW2sq21_vcCFa0lrQYdVccDXw;%3fhttps://www.sweetwater.com/store/detail/D45Y8--martin-d-45-acoustic-guitar-natural?utm_source=google&utm_medium=organicpla&mrkgadid=&mrkgcl=28&mrkgen=gpla&mrkgbflag=0&mrkgcat=&acctid=21700000001645388&dskeywordid=92700057522574369&lid=92700057522574369&ds_s_kwgid=58700006368161508&ds_s_inventory_feed_id=97700000007215323&dsproductgroupid=477762231356&product_id=D45Y8&prodctry=US&prodlang=en&channel=online&storeid=&device=c&network=g&matchtype=&adpos=largenumber&locationid=1014080&creative=470980643466&targetid=pla-477762231356&campaignid=11205481652&awsearchcpc=1&gclid=CMW2sq21_vcCFa0lrQYdVccDXw&gclsrc=ds
  5. But we remove the new gclid first: https://ad.doubleclick.net/ddm/clk/449830104;253494287;m;u=ds&sv1=57522574369&sv2=3307222181951115&sv3=3423267513265722070&gclid=CMW2sq21_vcCFa0lrQYdVccDXw;%3fhttps://www.sweetwater.com/store/detail/D45Y8--martin-d-45-acoustic-guitar-natural?utm_source=google&utm_medium=organicpla&mrkgadid=&mrkgcl=28&mrkgen=gpla&mrkgbflag=0&mrkgcat=&acctid=21700000001645388&dskeywordid=92700057522574369&lid=92700057522574369&ds_s_kwgid=58700006368161508&ds_s_inventory_feed_id=97700000007215323&dsproductgroupid=477762231356&product_id=D45Y8&prodctry=US&prodlang=en&channel=online&storeid=&device=c&network=g&matchtype=&adpos=largenumber&locationid=1014080&creative=470980643466&targetid=pla-477762231356&campaignid=11205481652&awsearchcpc=1&gclsrc=ds
  6. Then it gets server-redirected to the destination: https://www.sweetwater.com/store/detail/D45Y8--martin-d-45-acoustic-guitar-natural?utm_source=google&utm_medium=organicpla&mrkgadid=&mrkgcl=28&mrkgen=gpla&mrkgbflag=0&mrkgcat=&acctid=21700000001645388&dskeywordid=92700057522574369&lid=92700057522574369&ds_s_kwgid=58700006368161508&ds_s_inventory_feed_id=97700000007215323&dsproductgroupid=477762231356&product_id=D45Y8&prodctry=US&prodlang=en&channel=online&storeid=&device=c&network=g&matchtype=&adpos=largenumber&locationid=1014080&creative=470980643466&targetid=pla-477762231356&campaignid=11205481652&awsearchcpc=1&gclsrc=ds&gclid=CMW2sq21_vcCFa0lrQYdVccDXw&gclsrc=ds
  7. But we strip yet another gclid before going there: https://www.sweetwater.com/store/detail/D45Y8--martin-d-45-acoustic-guitar-natural?utm_source=google&utm_medium=organicpla&mrkgadid=&mrkgcl=28&mrkgen=gpla&mrkgbflag=0&mrkgcat=&acctid=21700000001645388&dskeywordid=92700057522574369&lid=92700057522574369&ds_s_kwgid=58700006368161508&ds_s_inventory_feed_id=97700000007215323&dsproductgroupid=477762231356&product_id=D45Y8&prodctry=US&prodlang=en&channel=online&storeid=&device=c&network=g&matchtype=&adpos=largenumber&locationid=1014080&creative=470980643466&targetid=pla-477762231356&campaignid=11205481652&awsearchcpc=1&gclsrc=ds&gclsrc=ds

The destination server changes the request from:

  • /store/detail/D45Y8--martin-d-45-acoustic-guitar-natural?utm_source=google&utm_medium=organicpla&mrkgadid=&mrkgcl=28&mrkgen=gpla&mrkgbflag=0&mrkgcat=&acctid=21700000001645388&dskeywordid=92700057522574369&lid=92700057522574369&ds_s_kwgid=58700006368161508&ds_s_inventory_feed_id=97700000007215323&dsproductgroupid=477762231356&product_id=D45Y8&prodctry=US&prodlang=en&channel=online&storeid=&device=c&network=g&matchtype=&adpos=largenumber&locationid=1014080&creative=470980643466&targetid=pla-477762231356&campaignid=11205481652&awsearchcpc=1&gclsrc=ds&gclsrc=ds

to:

  • /store/detail/D45Y8--martin-d-45-natural?utm_source=google&utm_medium=organicpla&mrkgadid=&mrkgcl=28&mrkgen=gpla&mrkgbflag=0&mrkgcat=&acctid=21700000001645388&dskeywordid=92700057522574369&lid=92700057522574369&ds_s_kwgid=58700006368161508&ds_s_inventory_feed_id=97700000007215323&dsproductgroupid=477762231356&product_id=D45Y8&prodctry=US&prodlang=en&channel=online&storeid=&device=c&network=g&matchtype=&adpos=largenumber&locationid=1014080&creative=470980643466&targetid=pla-477762231356&campaignid=11205481652&awsearchcpc=1&gclsrc=ds

though that's not really important.

@fmarier
Copy link
Member Author

fmarier commented May 27, 2022

The first thing I noticed in that flow is that that the *.dartsearch.net rule is too broad (it should be clickserve.dartsearch.net) and using the wrong query string parameter (red which belongs to t.myvisualiq.com instead of ds_dest_url which belongs to dartsearch) because it attempts to do two debounces in one go.

The other thing I noticed is that several of these query strings are not encoded in "proper" HTML5 form format but we attempt to parse the query string nevertheless and fail to grab everything that's needed. Our currently available debouncing actions do not yet support the following hosts:

@fmarier fmarier added the Debounce Debouncing list related label Jun 24, 2022
fmarier added a commit that referenced this issue Jul 4, 2022
Sample URL: https://www.googleadservices.com/pagead/aclk?sa=L&ai=CpwMy_pA7YueBHMWV9AOwxrvgBPXF1eJm38Xi3KwMvLjk5vMNCAQQASgLYMmO8IbIo6AZoAGJg8X-A8gBB8gD2CCqBFxP0BVaXHBw0bTsk4SdtAz6eTEkpO5bWMb0JJ9zZZjWgQ4Tv1T4gTUaNmPp2H5322pGRBg-y3P7jmsjL3XDZO_AP6DTNRPxMNYb_zx-BXL8fbGHuo08uK48cCAPJcAEvoPWoo8DiAW0qZjfKcAFBaAGJoAH3_y6AYgHAZAHAagHpr4bqAe5mrECqAfz0RuoB-7SG6gH_5yxAqgHytwbqAfNo7ECoAjA8j2wCAHACAHSCAwQAiCEATICgkA6AQCaCVJodHRwczovL3d3dy5zd2VldHdhdGVyLmNvbS9zdG9yZS9kZXRhaWwvRDQ1WTgtLW1hcnRpbi1kLTQ1LWFjb3VzdGljLWd1aXRhci1uYXR1cmFssQmJ4KBqN-lXPbkJXM7iZAP2Jnn4CQHQCyrgCwGqDAIIAbgMAegMBuAS5vSK6fz787uAAYIUBAgBEgDQFQHpFY-4wNPDibHa8BWodvoVBUQ0NVk4gRa7V5yQTETaeYgWUJgWAbgWwPm8sSPAFsD5vLEj6hYECKzqIPgWAYAXAZIXCRIHCAEQAxjBAg&ctype=5&ved=2ahUKEwjawK7Jlt32AhX8Mn0KHaQUC5wQww96BAgBED0&cid=CAASFeRo9mEa81pQI8r5S7W87ZZ_MHUHVQ&dblrd=1&sival=AF15MEBme8-y7XM2ojdjUz9fPC5ilL9_jZ_4bwjLYo_PY-5CKrjcK2pH8B8y4aUUpseGeC5QdxGD4HpvRTSoslTwUXuVGrOPyUT5FHwykqwCMJv-7fHk3NWTk_ArH_nz0bfn1pJtHAWHFJAw8820rzgIbB48F3ZbiaBTfESZY29iIzi1ngqq6mDaMpINh3X7MuQq3L90K_rr&sig=AOD64_2IpBc4vty-6Tc066z47dc0zFEJ0g&adurl=https://clickserve.dartsearch.net/link/click%3Flid%3D92700057522574369%26ds_s_kwgid%3D58700006368161508%26ds_s_inventory_feed_id%3D97700000007215323%26ds_a_cid%3D2001181%26ds_a_caid%3D11205481652%26ds_a_agid%3D107178459582%26ds_a_fiid%3D%26ds_a_lid%3Dpla-477762231356%26ds_a_extid%3D%26%26ds_e_adid%3D470980643466%26ds_e_matchtype%3Dsearch%26ds_e_device%3Dc%26ds_e_network%3Dg%26ds_e_product_group_id%3D477762231356%26ds_e_product_id%3DD45Y8%26ds_e_product_merchant_id%3D15144%26ds_e_product_country%3DUS%26ds_e_product_language%3Den%26ds_e_product_channel%3Donline%26ds_e_product_store_id%3D%26ds_url_v%3D2%26ds_dest_url%3Dhttps://www.sweetwater.com/store/detail/D45Y8--martin-d-45-acoustic-guitar-natural%3Futm_source%3Dgoogle%26utm_medium%3Dorganicpla%26mrkgadid%3D%26mrkgcl%3D28%26mrkgen%3Dgpla%26mrkgbflag%3D0%26mrkgcat%3D%26acctid%3D21700000001645388%26dskeywordid%3D92700057522574369%26lid%3D92700057522574369%26ds_s_kwgid%3D58700006368161508%26ds_s_inventory_feed_id%3D97700000007215323%26dsproductgroupid%3D477762231356%26product_id%3DD45Y8%26prodctry%3DUS%26prodlang%3Den%26channel%3Donline%26storeid%3D%26device%3Dc%26network%3Dg%26matchtype%3D%26adpos%3Dlargenumber%26locationid%3D1014080%26creative%3D470980643466%26targetid%3Dpla-477762231356%26campaignid%3D11205481652%26awsearchcpc%3D1%26gclsrc%3Daw.ds%26
@fmarier fmarier self-assigned this Jul 4, 2022
ryanbr added a commit that referenced this issue Jul 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Debounce Debouncing list related
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant