Skip to content

Commit

Permalink
feat(client)!: remove OpenAI helper from the Node.js SDK
Browse files Browse the repository at this point in the history
BREAKING CHANGE: use LangChain collector instead
  • Loading branch information
tothandras committed Jan 17, 2024
1 parent 7812ce7 commit 327b6ba
Show file tree
Hide file tree
Showing 6 changed files with 9 additions and 609 deletions.
55 changes: 0 additions & 55 deletions api/client/node/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -122,58 +122,3 @@ Invalidate portal tokens for all or specific subjects.
```ts
await openmeter.portal.invalidateTokens()
```

## Helpers

### Vercel AI SDK / Next.js

The OpenAI streaming API used by the Vercel AI SDK doesn't return token usage metadata by default.
The OpenMeter `createOpenAIStreamCallback` helper function decorates the callback with a `onUsage`
callback which you can use to report usage to OpenMeter.

```ts
import OpenAI from 'openai'
import { OpenAIStream, StreamingTextResponse } from 'ai'
import { createOpenAIStreamCallback } from '@openmeter/sdk'

export async function POST(req: Request) {
const { messages } = await req.json()
const model = 'gpt-3.5-turbo'

const response = await openai.chat.completions.create({
model,
messages,
stream: true,
})

const streamCallbacks = await createOpenAIStreamCallback(
{
model,
prompts: messages.map(({ content }) => content),
},
{
// onToken() => {...}
// onFinal() => {...}
async onUsage(usage) {
try {
await openmeter.events.ingest({
source: 'my-app',
type: 'my-event-type',
subject: 'my-customer-id',
data: {
// Usage is { total_tokens, prompt_tokens, completion_tokens }
...usage,
model,
},
})
} catch (err) {
console.error('failed to ingest usage', err)
}
},
}
)

const stream = OpenAIStream(response, streamCallbacks)
return new StreamingTextResponse(stream)
}
```
2 changes: 0 additions & 2 deletions api/client/node/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,6 @@ export { OpenMeterConfig, RequestOptions } from './clients/client.js'
export { Event, IngestedEvent } from './clients/event.js'
export { Meter, MeterAggregation, WindowSize } from './clients/meter.js'

export { createOpenAIStreamCallback } from './next.js'

export class OpenMeter {
public events: EventsClient
public meters: MetersClient
Expand Down
76 changes: 0 additions & 76 deletions api/client/node/next.ts

This file was deleted.

4 changes: 0 additions & 4 deletions api/client/node/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -49,10 +49,6 @@
"current node"
],
"dependencies": {
"ai": "^2.2.19",
"undici": "^5.27.2"
},
"optionalDependencies": {
"js-tiktoken": "^1.0.7"
}
}
Loading

0 comments on commit 327b6ba

Please sign in to comment.