Refresh lang models for all users #536

drnic · 2024-11-06T07:39:15Z

In this PR, we provide a rake task rails models:import to create/update LanguageModel for all User from the models.yaml file in the root of the repo.

This models.yaml file can be manually modified, or can be regenerated from the first User in the local DB with rails models:export

revert local ffi 1.17.0 change needed for macos dev
update create_initial_assistants_etc
going to wait until Dynamic "best" Language Model #538 merged

Update it from local DB with: rails models:export

krschacht

I know you're still working on this, but I did a quick pass through since I had some time and shared a few thoughts. It's looking good! Thanks so much for tackling this!

krschacht · 2024-11-07T03:20:52Z

config/options.yml

@@ -51,6 +51,7 @@ shared:
    default_openai_key: <%= ENV["DEFAULT_OPENAI_KEY"] %>
    default_anthropic_key: <%= ENV["DEFAULT_ANTHROPIC_KEY"] %>
    default_groq_key: <%= ENV["DEFAULT_GROQ_KEY"] %>
+    lang_models_url: <%= ENV.fetch("LANG_MODELS_URL", "https://raw.githubusercontent.com/AllYourBot/hostedgpt/refs/heads/main/models.json") %>


For all new features, I've been trying to support them within options.yml. I've really come to like the rails pattern of getting to store secrets and configs in the bin/rails credentials:edit. Is that too unconventional?

To continue support of it I think this line would need to be:
<%= ENV["LANG_MODELS_URL"] || default_to(:lang_models_url) || "https://raw.githubusercontent.com/AllYourBot/hostedgpt/refs/heads/main/models.json" %>

krschacht · 2024-11-07T03:22:00Z

models.yaml

+  supports_images: true
+  supports_tools: true
+  input_token_cost_cents: '0.00025'
+  output_token_cost_cents: '0.001'


Just like you left the quotes off the strings, I'm pretty sure you don't need quotes around floats in a yml file

krschacht · 2024-11-07T03:25:12Z

app/models/language_model.rb

@@ -47,11 +49,51 @@ def created_by_current_user?
    user == Current.user
  end

+  def api_service_name
+    api_service.name
+  end


How about delegate :name, to: :api_service, prefix: true and then pull it up to the other delegate a few lines above

krschacht · 2024-11-07T03:26:04Z

app/models/language_model.rb

+        warn "Failed to import '#{model[:api_name]}': #{e.message} for #{model.inspect}"
+      end
+    end
+  end


Can we put these two methods and the as_json into a concern? I think their ancillary enough that they don't need to be in the core model file. I know this model doesn't have any other concerns yet but it's just about ready for it.

krschacht · 2024-11-07T03:26:24Z

lib/tasks/models.rake

+    args.with_defaults(path: Rails.root.join("models.yaml"))
+    users = User.all
+    LanguageModel.import_from_file(path: args[:path], users:)
+  end


good call, it's nice to have these helper tasks

krschacht · 2024-11-07T03:26:28Z

README.md

@@ -29,14 +29,14 @@ This project is led by an experienced rails developer, but I'm actively looking
 - [Deploy the app on Heroku](#deploy-the-app-on-heroku)
 - [Deploy on your own server](#deploy-on-your-own-server)
 - [Configure optional features](#configure-optional-features)
-  - [Give assistant access to your Google apps](#configuring-google-tools)
+  - [Configuring Google Tools](#configuring-google-tools)


Thanks for catching the broken anchors below. But on this change, since our readme has gotten so long, I like the more descriptive "Give assistant access to your Google apps" on this one. It gives people a vague idea of what's possible whereas I don't the more succinct "Configuring Google Tools" hints at enough.

krschacht · 2024-11-07T03:30:45Z

README.md

+
+```plain
+rails models:export[tmp/models.json]
+```


Thanks for adding this explanation. I am wondering though if we can add some smart defaults so that a new user can safely ignore this and still get reasonable behavior. Can we make it so that bin/dev also runs models:import on startup? It already runs migrations and bundle upon startup so I've been leaning into it.

We have a fair number of people using this app who are only mildly technical, or are technical but don't know rails, so it's been nice to keep the day-to-day running of it super simple.

But actually, now that I write all this, I don't think people will reliably restart their server. It's not like fly, render, heroku do an automatic nightly restart or anything. Now I'm on the fence... I think I still learn towards doing it on startup, since it's a bit of a catch all. But we'll also add a button to the Settings at some point. And maybe later on we could even add a nightly sync of models:import in the queue. (Scope creep for now, I'm just thinking out loud.)

krschacht · 2024-11-07T03:31:21Z

app/models/language_model.rb

@@ -4,12 +4,14 @@ class LanguageModel < ApplicationRecord
  BEST_CLAUDE = "claude-best"
  BEST_GROQ = "groq-best"

+  # TODO: infer these from models.yaml


Agreed, it's nice to eliminate this config

krschacht · 2024-11-07T03:33:05Z

app/models/language_model.rb

  BEST_MODELS = {
    BEST_GPT => "gpt-4o-2024-08-06",
    BEST_CLAUDE => "claude-3-5-sonnet-20240620",
    BEST_GROQ => "llama3-70b-8192",
  }

+  # TODO: what are these for?


The app now tracks your usage $. Currently, when you click "..." on a conversation you see this:

Soon I want to add it so that when you click on your profile pic (button left) then it shows something equivalent for the month-to-date across all conversations.

But... (note to self) these two things should probably be moved to a separate concern of their own. They're not so important as to be so prominent in this model.

drnic added 8 commits November 7, 2024 13:11

I need ffi 1.17 locally

67418de

Introduce Setting.lang_models_url via $LANG_MODELS_URL

36f300b

LanguageModel.export_to_file(path:)

e7a1aec

Communal list of models/prices in models.yaml

3b22586

Update it from local DB with: rails models:export

Import new or updated models

5b47342

Document importing models

a4d7cac

models.yaml has models nested under models: key

dc40e91

Revert ffi bump

d1c84bb

drnic force-pushed the refresh-lang-models branch from 3e3f195 to d1c84bb Compare November 7, 2024 03:12

krschacht requested changes Nov 7, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refresh lang models for all users #536

Refresh lang models for all users #536

drnic commented Nov 6, 2024 •

edited

Loading

krschacht left a comment

krschacht Nov 7, 2024

krschacht Nov 7, 2024

krschacht Nov 7, 2024

krschacht Nov 7, 2024

krschacht Nov 7, 2024

krschacht Nov 7, 2024

krschacht Nov 7, 2024

krschacht Nov 7, 2024

krschacht Nov 7, 2024

Refresh lang models for all users #536

Are you sure you want to change the base?

Refresh lang models for all users #536

Conversation

drnic commented Nov 6, 2024 • edited Loading

krschacht left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

drnic commented Nov 6, 2024 •

edited

Loading