Guardrails Langchain integration with streamable LCEL Runnable

[adreo00]

Is there any updated docs/GitHub Issues I can refer to on how to add guardrails to an arbitrary langchain runnable, in LCEL style?

This example looks like exactly what I'm looking for, but using guardrails-ai=0.4.1, I get this error when I run your try it out the:

ModuleNotFoundError: No module named 'guardrails.functional'

[adreo00]

@CalebCourier Any thoughts on this? I would love to be able to replicate your example in #568

[adreo00]

The langchain docs seem to use the langchain-output-parser.

I found an 2 month old implementation here , but it looks like it looks like its using guardrails 0.2.4.

[ShreyaR]

@adreo00 here's how to use @CalebCourier's new syntax to support using Guardrails in LCEL streaming. Update to docs coming next week to showcase the support.

# Langchain imports
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI

# Guardrails imports
from guardrails.hub import RegexMatch, ReadingTime
from guardrails import Guard


# Set up Langchain components
topic = "Ice cream"
prompt = ChatPromptTemplate.from_template("ELIF: {topic}")
model = ChatOpenAI(model="gpt-3.5-turbo")
output_parser = StrOutputParser()

# Set up a Guard
guard = (
    Guard()
    .use(RegexMatch(topic, match_type="search"))
    .use(ReadingTime(1))
)

# Create an LCEL chain
chain = prompt | model | guard | output_parser

# Call chain
response = chain.invoke({"topic": topic})

# View guard history
print(guard.history.last.tree)

Let us know if you run into any issues. Also, what chains are you planning on using guardrails with? Can test it on our end as well

[adreo00]

Done: #651

[zsimjee]

Updated the issue with the way to get agent executor flows working. It really comes down to how we're chaining guards in for that particular pattern. I'll write up a langchain doc that shows this as one of the primary patterns

[adreo00]

@zsimjee In #651, I think you correctly implemented with a way to call guardrails synchronously using chain with agent executors, and I appreciate you investing the time to do so!

Continuing this discussion, in real a usecase of this agent executor, we are currently using the beta astream_events method (source) of this chain rather than invoke. If we replace chain.invoke with chain.astream_events and return some of these events back to the client, guardrails is called after we have already streamed the entire response back to the user.

This seems like a very big ask on my end, but is there any current way to use guardrails on streamed responses via something like astream_events, or are there plans to support this in the future? It seems like guardrails has some stream-handling capabilities, but this doc references direct client calls to openai.

[zsimjee]

Yes it does have stream handling support! I'm writing the lcel docs for guardrails today, I'll try to do some surface-level exploration and leave some notes. I'll do a deep dive in the next few days.

[alex-dreo-persefoni]

Thanks @zsimjee, looking forward to the docs! I'll try to take a look from my end as well to see if I can figure out streaming in the meantime