Extract web search data

[joelrorseth]

Hi, I'm hoping to extract data related to Bing Chat's use of the regular Bing Search engine. Specifically, I'd like to extract its reformulated query (i.e. often Bing Chat reformulates your natural language prompt into "Searching for: < Reformulated query >"), as well as the search results that are retrieved from regular Bing Search before answering the original prompt. I wouldn't have assumed that the latter would be returned by the Sydney API (since Bing Search results are not explicitly shown in the Bing Chat UI), however it looks like both of these data points are in fact returned in the Edge UI.

Immediately after ChatHub connection sends its usual initial "requestId" message, it sends a message that contains the reformulated query. In the following example, I followed up my original prompt for "who is donald trump" with "what are his achievements", which yields the following message:

{
    "type": 1,
    "target": "update",
    "arguments": [
        {
            "messages": [
                {
                    "text": "Searching the web for: `donald trump achievements`",
                    "hiddenText": "donald trump achievements",
                    "author": "bot",
                    "createdAt": "2023-05-02T03:16:14.672353+00:00",
                    "timestamp": "2023-05-02T03:16:14.672353+00:00",
                    "messageId": "44c19edd-d47b-437e-a2fa-a86b80c48491",
                    "messageType": "InternalSearchQuery",
                    "offense": "None",
                    "adaptiveCards": [
                        {
                            "type": "AdaptiveCard",
                            "version": "1.0",
                            "body": [
                                {
                                    "type": "RichTextBlock",
                                    "inlines": [
                                        {
                                            "type": "TextRun",
                                            "isSubtle": true,
                                            "italic": true,
                                            "text": "Searching the web for: `donald trump achievements`",
                                        }
                                    ],
                                }
                            ],
                        }
                    ],
                    "feedback": {"tag": null, "updatedOn": null, "type": "None"},
                    "contentOrigin": "DeepLeo",
                    "privacy": null,
                    "spokenText": "Searching for: donald trump achievements",
                }
            ],
            "requestId": "<myRequestId>",
        }
    ],
}

Then, immediately following this message, ChatHub sends a message containing the abridged search results retrieved by Bing Search. Here is a partial example for the prompt "who is bill gates":

{
    "type":1,
    "target":"update",
    "arguments":[
        {
            "messages":[
                {
                    "hiddenText": "```json\n{\"fact_results\":[ ....

If you parse "hiddenText" here, it seems to be an object with two keys: "fact_results" and "web_search_results". "fact_results" is sometimes omitted, leaving only "web_search_results". Here is the abridged parse for this example:

{
    "fact_results":[
        {
            "index":"1",
            "title":"Bill Gates",
            "snippets":[
                "William Henry Gates III is an American business magnate, philanthropist, and investor..."
            ],
            "data":{
                "DominantLabel":"American businessman and philanthropist",
                "Facts":[
                    {
                        "Name":"Born",
                        "Values":[
                            {
                                "Text":"William Henry Gates III · October 28, 1955 · Seattle, Washington, U.S."
                            }
                        ]
                    },
                    {
                        "Name":"Education",
                        "Values":[
                            {
                                "Text":"Harvard University · (dropped out)"
                            }
                        ]
                    },
                    ...
                ]
            },
            "url":"https://bing.com/search?q=bill+gates"
        }
    ],
    "web_search_results":[
        {
            "index":"2",
            "title":"Bill Gates - Wikipedia",
            "snippets":[
                "Bill Gates - Wikipedia Bill Gates 163 languages Article ...",
                ...
            ],
            "url":"https://en.wikipedia.org/wiki/Bill_Gates"
        },
        {
            "index":"3",
            "title":"Bill Gates | Biography, Microsoft, & Facts | Britannica",
            "snippets":[
                "Bill Gates | Biography, Microsoft, & Facts | Britannica Bill Gates ...",
                "Bill Gates, in full William Henry Gates III, (born October 28, 1955, ...",
                ...
            ],
            "url":"https://www.britannica.com/biography/Bill-Gates"
        },
        ...
    ]
}

I experimented with _ChatHub.ask_stream, but it doesn't appear that either of the above messages are received by EdgeGPT. I do see the initial "requestId" message, all the intermediary (partial) Bing Chat response messages, and the "final" message with full response, but I have not seen the reformulation or web results messages. Does anyone know why these messages aren't being received by EdgeGPT? Could we add support for this somehow? Thanks!

[ghost]

raw=True
My GUI has the ability to extract web search data, you can have a look at it
#365

[Nu11Const]

If you extract it,It's too long.
I think this many days ago.