Skip to content

Instantly share code, notes, and snippets.

@thesephist
Created December 26, 2023 22:52
Show Gist options
  • Save thesephist/fabc5f56e24bb4eea61061e2942923f7 to your computer and use it in GitHub Desktop.
Save thesephist/fabc5f56e24bb4eea61061e2942923f7 to your computer and use it in GitHub Desktop.
Ranked summarize with Cascade.
import { cascade } from "../src/cascade.ts";
export const summarize = cascade
.chatCompletion("summarize", {
model: "gpt-3.5-turbo",
renderPrompt: (context: { title: string; content: string }) => [
{
type: "user",
content: [
`Write a one sentence summary about ${context.title}.\n\n${context.content}.`,
`First, within <reasoning></reasoning> tags, write a brief plan of what the most concise and effective summary would include and exclude.`,
`Then, write the summary within <summary></summary> tags.`,
``,
`Your summary should be between 50 and 100 words.`,
].join("\n"),
},
],
samplingParams: { temperature: 0.9, maxTokens: 1024 },
})
.then(cascade.capture("<summary>", "</summary>"));
export const rankedSummarize = cascade
.fork<{ title: string; content: string }>(3)
.then(summarize)
.then(cascade.collect())
.then(cascade.debug("summary"))
.then(
cascade.sortByScore({
score: cascade
.completion("summaryScore", {
model: "gpt-3.5-turbo-instruct",
renderPrompt: (summary: string) =>
[
`Write a score for the summary below:`,
``,
`"""`,
`Summary: ${summary}`,
`"""`,
``,
`First, within <reasoning></reasoning> tags, write a brief plan of what the most concise and effective summary would include and exclude.`,
`Then, write the final numeric score within <score></score> tags.`,
``,
`Your score should be between 0 and 10.`,
].join("\n"),
samplingParams: { temperature: 0.9, maxTokens: 512 },
})
.then(cascade.collect())
.then(
cascade.captureMany({
reasoning: ["<reasoning>", "</reasoning>"],
score: ["<score>", "</score>"],
}),
)
.then(cascade.debug("scoring"))
.then(
cascade.map("getScore", {
fn: ({ score }: { score: string }) => parseFloat(score),
}),
),
}),
)
.then(cascade.first());
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment