Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement advanced web scraping with Playwright and Perplex integration #120

Open
wants to merge 6 commits into
base: development
Choose a base branch
from
Open

Implement advanced web scraping with Playwright and Perplex integration #120

wants to merge 6 commits into from

Conversation

linuztx
Copy link

@linuztx linuztx commented Sep 12, 2024

  • Enhance webpage_content_tool for targeted information extraction from web pages
  • Implement Playwright URL scraper in python/helpers/url_scraper.py
  • Create Perplex scraper in python/helpers/perplex_scraper.py
  • Add new prompts: perplex.prompt.md and tool.webpage.response.md
  • Update agent.tools.md with webpage_content_tool queries instruction
  • Modify WebpageContentTool to utilize new scraping functionality
  • Update installation docs and requirements.txt

RahulVadisetty91
RahulVadisetty91 reviewed Sep 13, 2024
View reviewed changes
Copy link

@RahulVadisetty91 RahulVadisetty91 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reqas

Update to the ChatPromptTemplate and chain.ainvoke usage

chain = prompt | self.client
response = await chain.ainvoke(inputs)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@RahulVadisetty91 RahulVadisetty91 RahulVadisetty91 left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@linuztx @RahulVadisetty91 @frdel