Compare commits
32 commits
cbd798dfd5
...
2f9ab6a414
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2f9ab6a414 | ||
|
|
2c20b9a638 | ||
|
|
a38aadd048 | ||
|
|
43f1235482 | ||
|
|
01ff3a0579 | ||
|
|
95e1f77956 | ||
|
|
e9d764817f | ||
|
|
fed806bfc0 | ||
|
|
575622b45c | ||
|
|
29d94c13d5 | ||
|
|
2680abcf1f | ||
|
|
6b213f085e | ||
|
|
962469bd76 | ||
|
|
15f7f3ac96 | ||
|
|
9e95ccdca4 | ||
|
|
71cf7a927f | ||
|
|
6e64969bb6 | ||
|
|
b70cf17713 | ||
|
|
0e3ccac6a9 | ||
|
|
645ea9f2b9 | ||
|
|
c125d05f26 | ||
|
|
21a49c18a0 | ||
|
|
e4f58281fb | ||
|
|
d3ab367f6c | ||
|
|
0dd9b072e8 | ||
|
|
f7d97db569 | ||
|
|
fb92930e7a | ||
|
|
ab11490807 | ||
|
|
3ed38031be | ||
|
|
b993482b85 | ||
|
|
0e0d0ea824 | ||
|
|
8a6111aeb5 |
12 changed files with 958 additions and 336 deletions
13
CHANGELOG.md
13
CHANGELOG.md
|
|
@ -2,6 +2,19 @@
|
|||
|
||||
All notable changes to this project will be documented in this file.
|
||||
|
||||
## Beta 2 (v0.2.0)
|
||||
- Web UI overhaul: fixed header, single-scroll layout, auto pin-to-bottom, infinite scroll history
|
||||
- Channel selector moved into menubar brand; commit hash shown subtly
|
||||
- SSE realtime updates and JSON APIs for info, channels, tail, history, stream
|
||||
- Link cards: robust OG/Twitter parsing, X (Twitter) oEmbed embeds, YouTube oEmbed, direct image previews
|
||||
- On-demand link summaries with caching; inline actions (🌚/🌝, ▾) and collapse/expand
|
||||
- Dedicated link-only LLM prompt; improved fetch headers; readability extraction; image-to-vision support
|
||||
- Mention detection/rate limiting/quiet hours retained and refined
|
||||
- Store: case-insensitive channel queries, msgid de-dup, WAL, history paging
|
||||
- Metrics: messages ingested, notifications sent, pruned, connection gauge
|
||||
- Docker: multi-arch build, healthcheck; version/commit/build injected
|
||||
- Numerous bug fixes (template rendering, static assets path, scroll duplicates, initial bottom snap)
|
||||
|
||||
## Beta 1 (initial release)
|
||||
- soju-specific raw connector with event playback and CHATHISTORY fallback
|
||||
- Message storage (SQLite, WAL), msgid de-dup, retention job
|
||||
|
|
|
|||
76
README.md
76
README.md
|
|
@ -6,7 +6,7 @@ An IRC bouncer companion service for soju that:
|
|||
- Notifies you on mentions via Pushover (default)
|
||||
- Stores messages in SQLite for summaries and on-demand inspection
|
||||
- Generates AI digests (OpenAI by default) on schedule or on demand
|
||||
- Exposes a small HTTP API for health, tailing messages, metrics, and triggering digests
|
||||
- Exposes a small HTTP API and a minimal Web UI (Pico.css) for status, tail, history, link cards, and on-demand summaries
|
||||
|
||||
Note: this is not a bot and never replies in IRC. It passively attaches as a soju multi-client on your main account.
|
||||
|
||||
|
|
@ -18,20 +18,20 @@ If you use soju as a bouncer, you may want per-client alerts and AI summaries wi
|
|||
|
||||
- Language: Go (single static binary, low memory footprint)
|
||||
- Long-lived IRC client: raw IRC using a lightweight parser (sorcix/irc) with an irssi-style handshake tailored for soju
|
||||
- Message storage: SQLite via modernc.org/sqlite
|
||||
- Message storage: SQLite via modernc.org/sqlite (WAL enabled)
|
||||
- Scheduling: github.com/robfig/cron/v3
|
||||
- Notifications: github.com/gregdel/pushover
|
||||
- Summarization (LLM): github.com/sashabaranov/go-openai
|
||||
- HTTP API: Go stdlib `net/http`
|
||||
- HTTP API + Web UI: Go stdlib `net/http` + `html/template` + embedded static assets
|
||||
|
||||
Runtime modules:
|
||||
|
||||
- `internal/soju`: soju connection, capability negotiation, irssi-style PASS/USER auth, joins, message ingestion, event playback, CHATHISTORY fallback
|
||||
- `internal/store`: SQLite schema and queries
|
||||
- `internal/notifier`: Pushover notifier (pluggable interface)
|
||||
- `internal/summarizer`: OpenAI client with GPT-5 defaults, GPT-4o-mini fallback
|
||||
- `internal/summarizer`: OpenAI client with GPT-5 defaults, GPT-4o-mini fallback; separate link-summarization prompt
|
||||
- `internal/scheduler`: cron-based digest scheduling and daily retention job
|
||||
- `internal/httpapi`: `/healthz`, `/ready`, `/tail`, `/trigger`, `/metrics`
|
||||
- `internal/httpapi`: `/healthz`, `/ready`, `/tail`, `/trigger`, `/metrics`, Web UI and JSON APIs
|
||||
- `internal/config`: env config loader and helpers
|
||||
|
||||
## Features
|
||||
|
|
@ -41,7 +41,11 @@ Runtime modules:
|
|||
- AI digest generation: concise natural summaries (no rigid sections); integrates pasted multi-line posts and referenced link context; image links sent to GPT‑5 as vision inputs
|
||||
- Configurable schedules (cron), quiet hours, and summary parameters
|
||||
- Local persistence with retention pruning (daily at 03:00)
|
||||
- HTTP endpoints: health, tail, metrics, on-demand digests
|
||||
- Web UI with:
|
||||
- Realtime chat tail via SSE; auto-scroll to bottom; preload older history with infinite scroll-up
|
||||
- Link cards with OG/Twitter metadata (X posts via oEmbed), YouTube oEmbed embeds, direct image previews
|
||||
- Inline on-demand link summarization with caching (24h), and a single summarize toggle (🌚/🌝)
|
||||
- Channel selector in the menubar, login interstitial using `HTTP_TOKEN`
|
||||
|
||||
## How it works
|
||||
|
||||
|
|
@ -60,20 +64,21 @@ Runtime modules:
|
|||
4) Messages and mentions:
|
||||
- Each `PRIVMSG` is stored with server-time when available
|
||||
- Mentions trigger Pushover notifications subject to quiet hours, urgency, and rate limits
|
||||
- Debug logs include: mention delivered or suppression reason (backfill, quiet hours, rate limit)
|
||||
|
||||
5) Summarization:
|
||||
- `/trigger` or the scheduler loads a window and calls OpenAI
|
||||
- GPT‑5 context: ~272k input tokens + up to 128k output tokens (400k total)
|
||||
- Summaries are concise/natural and integrate multi-line posts, article text (readability-extracted), and image links (vision)
|
||||
- Digests: `/trigger` or the scheduler loads a window and calls OpenAI with a conversation-focused prompt
|
||||
- Link summaries: dedicated prompt that ignores chat context; fetches page content with readability; includes oEmbed hints for YouTube and X; passes images to vision models
|
||||
|
||||
6) HTTP API:
|
||||
6) HTTP + JSON API:
|
||||
- `/healthz` → `200 ok`
|
||||
- `/ready` → `200` only when connected to soju
|
||||
- `/tail?channel=#chan&limit=N` → plaintext tail (chronological)
|
||||
- `/trigger?channel=#chan&window=6h` → returns digest and sends via notifier
|
||||
- `/tail?channel=#chan&limit=N` → JSON tail for UI
|
||||
- `/history?channel=#chan&before=<RFC3339>&limit=N` → JSON older messages (infinite scroll)
|
||||
- `/trigger?channel=#chan&window=6h` → returns digest JSON and (optionally) pushes via notifier
|
||||
- `/linkcard?url=...` → card JSON (title/desc/image or embed HTML)
|
||||
- `/linksummary?url=...` → brief AI summary of a single URL (cached 24h)
|
||||
- `/metrics` → Prometheus text metrics
|
||||
- Protect `/tail` and `/trigger` with `HTTP_TOKEN` via Bearer, `token` query, `X-Auth-Token`, or basic auth (`token:<HTTP_TOKEN>`)
|
||||
- Protect UI + JSON with `HTTP_TOKEN` cookie; APIs also allow Bearer/query token
|
||||
|
||||
## Health and readiness
|
||||
|
||||
|
|
@ -222,7 +227,7 @@ Compose (with localhost bind suitable for Synology reverse proxy):
|
|||
```yaml
|
||||
services:
|
||||
sojuboy:
|
||||
image: code.cravey.net/your-user/sojuboy:v0.1.0-beta1
|
||||
image: code.cravey.net/your-user/sojuboy:v0.2.0-beta2
|
||||
restart: unless-stopped
|
||||
env_file: .env
|
||||
ports:
|
||||
|
|
@ -241,7 +246,7 @@ services:
|
|||
```yaml
|
||||
services:
|
||||
sojuboy:
|
||||
image: code.cravey.net/your-user/sojuboy:v0.1.0-beta1
|
||||
image: code.cravey.net/your-user/sojuboy:v0.2.0-beta2
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "127.0.0.1:8080:8080" # bind only to localhost; fronted by DSM Reverse Proxy
|
||||
|
|
@ -316,16 +321,16 @@ services:
|
|||
| OPENAI_API_KEY | (empty) |
|
||||
| OPENAI_BASE_URL | (empty) |
|
||||
| OPENAI_MODEL | gpt-5 |
|
||||
| OPENAI_MAX_TOKENS | 700 |
|
||||
| OPENAI_MAX_TOKENS | 128000 |
|
||||
| SUMM_FOLLOW_LINKS | true |
|
||||
| SUMM_LINK_TIMEOUT | 6s |
|
||||
| SUMM_LINK_MAX_BYTES | 262144 |
|
||||
| SUMM_GROUP_WINDOW | 90s |
|
||||
| SUMM_MAX_LINKS | 5 |
|
||||
| SUMM_LINK_TIMEOUT | 20s |
|
||||
| SUMM_LINK_MAX_BYTES | 1048576 |
|
||||
| SUMM_GROUP_WINDOW | 120s |
|
||||
| SUMM_MAX_LINKS | 20 |
|
||||
| SUMM_MAX_GROUPS | 0 |
|
||||
| SUMM_TIMEOUT | 5m |
|
||||
| SUMM_TIMEOUT | 10m |
|
||||
| DIGEST_CRON | 0 */6 * * * |
|
||||
| DIGEST_WINDOW | 6h |
|
||||
| DIGEST_WINDOW | 24h |
|
||||
| QUIET_HOURS | (empty) |
|
||||
| NOTIFY_BACKFILL | false |
|
||||
| MENTION_MIN_INTERVAL | 30s |
|
||||
|
|
@ -335,7 +340,7 @@ services:
|
|||
| HTTP_LISTEN | :8080 |
|
||||
| HTTP_TOKEN | (empty) |
|
||||
| STORE_PATH | /data/app.db |
|
||||
| STORE_RETENTION_DAYS | 7 |
|
||||
| STORE_RETENTION_DAYS | 365 |
|
||||
| LOG_LEVEL | info |
|
||||
|
||||
## Pushover setup
|
||||
|
|
@ -355,25 +360,22 @@ services:
|
|||
## HTTP API
|
||||
|
||||
- `GET /healthz` → `200 ok`
|
||||
- `GET /tail?channel=%23chan&limit=50`
|
||||
- Returns plaintext messages (chronological)
|
||||
- Auth: provide `HTTP_TOKEN` as a Bearer token (or query param `token=`)
|
||||
- `GET /trigger?channel=%23chan&window=6h`
|
||||
- Returns plaintext digest
|
||||
- Also sends via notifier when configured
|
||||
- Auth as above
|
||||
- `GET /tail?channel=%23chan&limit=50` (JSON)
|
||||
- `GET /history?channel=%23chan&before=<RFC3339>&limit=50` (JSON)
|
||||
- `GET /trigger?channel=%23chan&window=6h` (JSON)
|
||||
- `GET /linkcard?url=…` (JSON)
|
||||
- `GET /linksummary?url=…` (JSON)
|
||||
- `GET /metrics`
|
||||
- Prometheus metrics: `sojuboy_messages_ingested_total`, `sojuboy_notifications_sent_total`, `sojuboy_messages_pruned_total`, `sojuboy_connected`
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
- Empty tail while there’s activity
|
||||
- Ensure the service logs `join requested:` followed by `joined` for your channels
|
||||
- Ensure the service logs readiness and joins for your channels
|
||||
- Confirm `.env` `CHANNELS` contains your channels
|
||||
- Check for `/metrics` and logs for recent message ingestion
|
||||
- Check `/metrics` and logs for recent message ingestion
|
||||
|
||||
- 401 Unauthorized from `/tail` or `/trigger`
|
||||
- Provide `Authorization: Bearer $HTTP_TOKEN` or `?token=$HTTP_TOKEN`
|
||||
- 401 Unauthorized from UI/API
|
||||
- Log in at `/login` with `HTTP_TOKEN`, or pass it via Bearer/`token=`
|
||||
|
||||
- OpenAI 502/URL errors
|
||||
- Ensure `OPENAI_BASE_URL=https://api.openai.com/v1`
|
||||
|
|
@ -394,7 +396,7 @@ Project layout (selected):
|
|||
- `internal/store` – SQLite schema and queries
|
||||
- `internal/notifier` – Pushover notifier
|
||||
- `internal/summarizer` – OpenAI client and prompts
|
||||
- `internal/httpapi` – health, tail, trigger, metrics endpoints
|
||||
- `internal/httpapi` – UI and endpoints
|
||||
- `internal/scheduler` – cron jobs
|
||||
|
||||
Go toolchain: see `go.mod` (Go 1.23), Dockerfile builds static binary for a distroless image.
|
||||
|
|
|
|||
|
|
@ -223,10 +223,11 @@ func main() {
|
|||
Password: cfg.Password,
|
||||
Channels: cfg.Channels,
|
||||
BackfillLatest: backfill,
|
||||
Store: st,
|
||||
OnPrivmsg: func(channel, author, text, msgid string, at time.Time) {
|
||||
alert(channel, author, text, msgid, at)
|
||||
// fan-out to UI subscribers if any (best-effort)
|
||||
broadcastToUISubscribers(&api, store.Message{Channel: channel, Author: author, Body: text, Time: at.UTC(), MsgID: msgid})
|
||||
api.Broadcast(strings.ToLower(channel), store.Message{Channel: channel, Author: author, Body: text, Time: at.UTC(), MsgID: msgid})
|
||||
},
|
||||
ConnectedGauge: &metrics.ConnectedGauge,
|
||||
}
|
||||
|
|
@ -292,25 +293,3 @@ func main() {
|
|||
<-ctx.Done()
|
||||
logger.Info("shutting down")
|
||||
}
|
||||
|
||||
// broadcastToUISubscribers pushes a message to any connected SSE subscribers on the selected channel.
|
||||
func broadcastToUISubscribers(api *httpapi.Server, m store.Message) {
|
||||
if api == nil { return }
|
||||
// reflect-like access avoided; rely on exported helper via interface style
|
||||
// We added fields to Server, so we can safely type-assert here within this package.
|
||||
// Iterate subscribers with internal lock via small helper method.
|
||||
type subAccess interface{ Broadcast(channel string, m store.Message) }
|
||||
if b, ok := any(api).(subAccess); ok { b.Broadcast(strings.ToLower(m.Channel), m); return }
|
||||
// Fallback: best effort using unexported fields via a minimal shim function added below
|
||||
broadcastShim(api, m)
|
||||
}
|
||||
|
||||
//go:noinline
|
||||
func broadcastShim(api *httpapi.Server, m store.Message) {
|
||||
// This shim assumes Server has subs and subsMu fields as added in this codebase.
|
||||
// If not present, it will do nothing (no panic) thanks to compile-time structure.
|
||||
// Since we are in the same module, we can update together.
|
||||
// WARNING: keep in sync with httpapi.Server struct.
|
||||
// Using an internal copy of the logic to avoid import cycles.
|
||||
// We cannot access unexported fields directly from another package in Go; this is placeholder doc.
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,9 +1,10 @@
|
|||
package httpapi
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io/fs"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/url"
|
||||
|
|
@ -15,6 +16,7 @@ import (
|
|||
|
||||
"sojuboy/internal/store"
|
||||
"sojuboy/internal/summarizer"
|
||||
|
||||
xhtml "golang.org/x/net/html"
|
||||
)
|
||||
|
||||
|
|
@ -51,13 +53,20 @@ type Server struct {
|
|||
// Link card cache
|
||||
cardCache map[string]linkCard
|
||||
cardCacheExp map[string]time.Time
|
||||
// Link summary cache
|
||||
summaryCache map[string]string
|
||||
summaryCacheExp map[string]time.Time
|
||||
}
|
||||
|
||||
func (s *Server) Start(ctx context.Context) error {
|
||||
mux := http.NewServeMux()
|
||||
// Minimal web UI
|
||||
mux.HandleFunc("/", s.handleUI)
|
||||
mux.HandleFunc("/summarizer", s.handleSummarizerUI)
|
||||
// Minimal web UI (templated)
|
||||
mux.HandleFunc("/", s.handleUIDash)
|
||||
mux.HandleFunc("/summarizer", s.handleUISummarizer)
|
||||
// Serve embedded static files under /static/
|
||||
if sub, err := fs.Sub(staticFS, "static"); err == nil {
|
||||
mux.Handle("/static/", http.StripPrefix("/static/", http.FileServer(http.FS(sub))))
|
||||
}
|
||||
mux.HandleFunc("/login", s.handleLogin)
|
||||
mux.HandleFunc("/auth", s.handleAuth)
|
||||
mux.HandleFunc("/logout", s.handleLogout)
|
||||
|
|
@ -85,6 +94,7 @@ func (s *Server) Start(ctx context.Context) error {
|
|||
mux.HandleFunc("/api/history", s.handleHistory)
|
||||
mux.HandleFunc("/api/stream", s.handleStream)
|
||||
mux.HandleFunc("/api/linkcard", s.handleLinkCard)
|
||||
mux.HandleFunc("/api/linksummary", s.handleLinkSummary)
|
||||
|
||||
srv := &http.Server{
|
||||
Addr: s.ListenAddr,
|
||||
|
|
@ -198,12 +208,14 @@ func (s *Server) handleTail(w http.ResponseWriter, r *http.Request) {
|
|||
_, _ = w.Write([]byte(m.Time.UTC().Format(time.RFC3339) + " " + m.Author + " " + channel + " " + m.Body + "\n"))
|
||||
}
|
||||
}
|
||||
|
||||
// Simple link card structure
|
||||
type linkCard struct{
|
||||
type linkCard struct {
|
||||
URL string `json:"url"`
|
||||
Title string `json:"title"`
|
||||
Description string `json:"description"`
|
||||
Image string `json:"image"`
|
||||
HTML string `json:"html"`
|
||||
}
|
||||
|
||||
// handleLinkCard fetches basic OpenGraph/Twitter card metadata (best-effort) and returns a small card.
|
||||
|
|
@ -214,46 +226,180 @@ func (s *Server) handleLinkCard(w http.ResponseWriter, r *http.Request) {
|
|||
return
|
||||
}
|
||||
raw := r.URL.Query().Get("url")
|
||||
if raw == "" { w.WriteHeader(http.StatusBadRequest); _, _ = w.Write([]byte("missing url")); return }
|
||||
if raw == "" {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
_, _ = w.Write([]byte("missing url"))
|
||||
return
|
||||
}
|
||||
u, err := url.Parse(raw)
|
||||
if err != nil || (u.Scheme != "http" && u.Scheme != "https") { w.WriteHeader(http.StatusBadRequest); _, _ = w.Write([]byte("bad url")); return }
|
||||
if err != nil || (u.Scheme != "http" && u.Scheme != "https") {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
_, _ = w.Write([]byte("bad url"))
|
||||
return
|
||||
}
|
||||
// cache lookup
|
||||
if s.cardCache == nil { s.cardCache = make(map[string]linkCard); s.cardCacheExp = make(map[string]time.Time) }
|
||||
if s.cardCache == nil {
|
||||
s.cardCache = make(map[string]linkCard)
|
||||
s.cardCacheExp = make(map[string]time.Time)
|
||||
}
|
||||
if exp, ok := s.cardCacheExp[raw]; ok && time.Now().Before(exp) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_ = json.NewEncoder(w).Encode(s.cardCache[raw])
|
||||
return
|
||||
}
|
||||
|
||||
// Special handling for X/Twitter posts via oEmbed
|
||||
host := strings.ToLower(u.Host)
|
||||
if (host == "x.com" || host == "twitter.com" || strings.HasSuffix(host, ".twitter.com")) && strings.Contains(strings.ToLower(u.Path), "/status/") {
|
||||
oembed := "https://publish.twitter.com/oembed?omit_script=1&hide_thread=1&dnt=1&align=center&url=" + url.QueryEscape(raw)
|
||||
client := &http.Client{Timeout: 8 * time.Second}
|
||||
reqTw, _ := http.NewRequestWithContext(r.Context(), http.MethodGet, oembed, nil)
|
||||
respTw, errTw := client.Do(reqTw)
|
||||
if errTw == nil && respTw.StatusCode >= 200 && respTw.StatusCode < 300 {
|
||||
defer respTw.Body.Close()
|
||||
var o struct {
|
||||
HTML string `json:"html"`
|
||||
}
|
||||
if err := json.NewDecoder(respTw.Body).Decode(&o); err == nil && o.HTML != "" {
|
||||
card := linkCard{URL: raw, HTML: o.HTML}
|
||||
s.cardCache[raw] = card
|
||||
s.cardCacheExp[raw] = time.Now().Add(24 * time.Hour)
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_ = json.NewEncoder(w).Encode(card)
|
||||
return
|
||||
}
|
||||
}
|
||||
// fallthrough to generic fetch if oEmbed fails
|
||||
}
|
||||
|
||||
// Special handling for YouTube via oEmbed
|
||||
if host == "www.youtube.com" || host == "youtube.com" || host == "m.youtube.com" || host == "youtu.be" {
|
||||
watchURL := raw
|
||||
if host == "youtu.be" {
|
||||
// Convert youtu.be/ID to watch?v=ID
|
||||
id := strings.TrimPrefix(u.Path, "/")
|
||||
watchURL = "https://www.youtube.com/watch?v=" + id
|
||||
}
|
||||
oembed := "https://www.youtube.com/oembed?format=json&url=" + url.QueryEscape(watchURL)
|
||||
client := &http.Client{Timeout: 8 * time.Second}
|
||||
reqY, _ := http.NewRequestWithContext(r.Context(), http.MethodGet, oembed, nil)
|
||||
respY, errY := client.Do(reqY)
|
||||
if errY == nil && respY.StatusCode >= 200 && respY.StatusCode < 300 {
|
||||
defer respY.Body.Close()
|
||||
var o struct{
|
||||
Title string `json:"title"`
|
||||
Author string `json:"author_name"`
|
||||
Thumb string `json:"thumbnail_url"`
|
||||
HTML string `json:"html"`
|
||||
}
|
||||
if err := json.NewDecoder(respY.Body).Decode(&o); err == nil {
|
||||
card := linkCard{URL: raw, Title: o.Title, HTML: o.HTML}
|
||||
// Note: do not set Image when HTML is provided to avoid duplicate thumbnails (embed already includes preview)
|
||||
s.cardCache[raw] = card
|
||||
s.cardCacheExp[raw] = time.Now().Add(24 * time.Hour)
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_ = json.NewEncoder(w).Encode(card)
|
||||
return
|
||||
}
|
||||
}
|
||||
// fallthrough if oEmbed fails
|
||||
}
|
||||
|
||||
// fetch minimal HTML and extract tags using a tolerant HTML parser
|
||||
client := &http.Client{ Timeout: 10 * time.Second }
|
||||
client := &http.Client{Timeout: 10 * time.Second}
|
||||
req, _ := http.NewRequestWithContext(r.Context(), http.MethodGet, raw, nil)
|
||||
req.Header.Set("User-Agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0 Safari/537.36")
|
||||
req.Header.Set("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8")
|
||||
req.Header.Set("Accept-Language", "en-US,en;q=0.9")
|
||||
resp, err := client.Do(req)
|
||||
if err != nil { w.WriteHeader(http.StatusBadGateway); _, _ = w.Write([]byte("fetch error")); return }
|
||||
if err != nil {
|
||||
w.WriteHeader(http.StatusBadGateway)
|
||||
_, _ = w.Write([]byte("fetch error"))
|
||||
return
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 { w.WriteHeader(http.StatusBadGateway); _, _ = w.Write([]byte("bad status")); return }
|
||||
// limit to 256KB and parse tokens
|
||||
limited := http.MaxBytesReader(w, resp.Body, 262144)
|
||||
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
|
||||
w.WriteHeader(http.StatusBadGateway)
|
||||
_, _ = w.Write([]byte("bad status"))
|
||||
return
|
||||
}
|
||||
// If image content-type, return as image card
|
||||
if ct := strings.ToLower(resp.Header.Get("Content-Type")); strings.HasPrefix(ct, "image/") {
|
||||
card := linkCard{URL: raw, Image: raw}
|
||||
s.cardCache[raw] = card
|
||||
s.cardCacheExp[raw] = time.Now().Add(24 * time.Hour)
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_ = json.NewEncoder(w).Encode(card)
|
||||
return
|
||||
}
|
||||
|
||||
// limit to 768KB and parse tokens
|
||||
limited := http.MaxBytesReader(w, resp.Body, 786432)
|
||||
doc, err := xhtml.Parse(limited)
|
||||
if err != nil { w.WriteHeader(http.StatusBadGateway); _, _ = w.Write([]byte("parse error")); return }
|
||||
if err != nil {
|
||||
w.WriteHeader(http.StatusBadGateway)
|
||||
_, _ = w.Write([]byte("parse error"))
|
||||
return
|
||||
}
|
||||
var title, desc, img string
|
||||
var walker func(*xhtml.Node)
|
||||
walker = func(n *xhtml.Node) {
|
||||
if n.Type == xhtml.ElementNode && strings.EqualFold(n.Data, "meta") {
|
||||
if n.Type == xhtml.ElementNode {
|
||||
if strings.EqualFold(n.Data, "meta") {
|
||||
// property or name + content
|
||||
var pn = ""; var nm = ""; var content = ""
|
||||
for _, a := range n.Attr { if strings.EqualFold(a.Key, "property") { pn = a.Val } else if strings.EqualFold(a.Key, "name") { nm = a.Val } else if strings.EqualFold(a.Key, "content") { content = a.Val } }
|
||||
var pn = ""
|
||||
var nm = ""
|
||||
var content = ""
|
||||
for _, a := range n.Attr {
|
||||
if strings.EqualFold(a.Key, "property") {
|
||||
pn = a.Val
|
||||
} else if strings.EqualFold(a.Key, "name") {
|
||||
nm = a.Val
|
||||
} else if strings.EqualFold(a.Key, "content") {
|
||||
content = a.Val
|
||||
}
|
||||
}
|
||||
key := strings.ToLower(pn)
|
||||
if key == "" { key = strings.ToLower(nm) }
|
||||
if key == "" {
|
||||
key = strings.ToLower(nm)
|
||||
}
|
||||
switch key {
|
||||
case "og:title", "twitter:title": if title == "" { title = content }
|
||||
case "og:description", "twitter:description": if desc == "" { desc = content }
|
||||
case "og:image", "twitter:image": if img == "" { img = content }
|
||||
case "og:title", "twitter:title":
|
||||
if title == "" {
|
||||
title = content
|
||||
}
|
||||
case "og:description", "twitter:description", "description":
|
||||
if desc == "" {
|
||||
desc = content
|
||||
}
|
||||
case "og:image", "og:image:url", "og:image:secure_url", "twitter:image", "twitter:image:src":
|
||||
if img == "" {
|
||||
img = content
|
||||
}
|
||||
}
|
||||
for c := n.FirstChild; c != nil; c = c.NextSibling { walker(c) }
|
||||
}
|
||||
}
|
||||
for c := n.FirstChild; c != nil; c = c.NextSibling {
|
||||
walker(c)
|
||||
}
|
||||
}
|
||||
walker(doc)
|
||||
card := linkCard{ URL: raw, Title: strings.TrimSpace(title), Description: strings.TrimSpace(desc), Image: strings.TrimSpace(img) }
|
||||
|
||||
// normalize image URL
|
||||
if img != "" {
|
||||
if strings.HasPrefix(img, "//") {
|
||||
if u.Scheme == "" {
|
||||
u.Scheme = "https"
|
||||
}
|
||||
img = u.Scheme + ":" + img
|
||||
} else if !strings.HasPrefix(img, "http://") && !strings.HasPrefix(img, "https://") {
|
||||
if ref, err := url.Parse(img); err == nil {
|
||||
img = u.ResolveReference(ref).String()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
card := linkCard{URL: raw, Title: strings.TrimSpace(title), Description: strings.TrimSpace(desc), Image: strings.TrimSpace(img)}
|
||||
// cache for 24h
|
||||
s.cardCache[raw] = card
|
||||
s.cardCacheExp[raw] = time.Now().Add(24 * time.Hour)
|
||||
|
|
@ -261,6 +407,71 @@ func (s *Server) handleLinkCard(w http.ResponseWriter, r *http.Request) {
|
|||
_ = json.NewEncoder(w).Encode(card)
|
||||
}
|
||||
|
||||
// handleLinkSummary returns a brief AI summary for a given URL, cached for 24h.
|
||||
func (s *Server) handleLinkSummary(w http.ResponseWriter, r *http.Request) {
|
||||
if s.AuthToken != "" && !checkAuth(r, s.AuthToken) {
|
||||
w.WriteHeader(http.StatusUnauthorized)
|
||||
_, _ = w.Write([]byte("unauthorized"))
|
||||
return
|
||||
}
|
||||
raw := r.URL.Query().Get("url")
|
||||
if raw == "" {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
_, _ = w.Write([]byte("missing url"))
|
||||
return
|
||||
}
|
||||
if s.Summarizer == nil {
|
||||
w.WriteHeader(http.StatusServiceUnavailable)
|
||||
_, _ = w.Write([]byte("summarizer not configured"))
|
||||
return
|
||||
}
|
||||
if s.summaryCache == nil { s.summaryCache = make(map[string]string) }
|
||||
if s.summaryCacheExp == nil { s.summaryCacheExp = make(map[string]time.Time) }
|
||||
if exp, ok := s.summaryCacheExp[raw]; ok && time.Now().Before(exp) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_ = json.NewEncoder(w).Encode(map[string]any{"summary": s.summaryCache[raw]})
|
||||
return
|
||||
}
|
||||
// Prefer link-specific summarization when available
|
||||
if sl, ok := s.Summarizer.(interface{ SummarizeLink(context.Context, string) (string, error) }); ok {
|
||||
tout := s.SummarizerTimeout
|
||||
if tout <= 0 { tout = 5 * time.Minute }
|
||||
if tout > 2*time.Minute { tout = 2 * time.Minute }
|
||||
ctx, cancel := context.WithTimeout(r.Context(), tout)
|
||||
defer cancel()
|
||||
sum, err := sl.SummarizeLink(ctx, raw)
|
||||
if err != nil {
|
||||
w.WriteHeader(http.StatusBadGateway)
|
||||
_, _ = w.Write([]byte("summarizer error"))
|
||||
return
|
||||
}
|
||||
if sum == "" { sum = "(no summary)" }
|
||||
s.summaryCache[raw] = sum
|
||||
s.summaryCacheExp[raw] = time.Now().Add(24 * time.Hour)
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_ = json.NewEncoder(w).Encode(map[string]any{"summary": sum})
|
||||
return
|
||||
}
|
||||
// Fallback to generic path
|
||||
msgs := []store.Message{{Channel: "#links", Author: "link", Body: raw, Time: time.Now().UTC()}}
|
||||
tout := s.SummarizerTimeout
|
||||
if tout <= 0 { tout = 5 * time.Minute }
|
||||
if tout > 2*time.Minute { tout = 2 * time.Minute }
|
||||
ctx, cancel := context.WithTimeout(r.Context(), tout)
|
||||
defer cancel()
|
||||
sum, err := s.Summarizer.Summarize(ctx, "#links", msgs, 0)
|
||||
if err != nil {
|
||||
w.WriteHeader(http.StatusBadGateway)
|
||||
_, _ = w.Write([]byte("summarizer error"))
|
||||
return
|
||||
}
|
||||
if sum == "" { sum = "(no summary)" }
|
||||
s.summaryCache[raw] = sum
|
||||
s.summaryCacheExp[raw] = time.Now().Add(24 * time.Hour)
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_ = json.NewEncoder(w).Encode(map[string]any{"summary": sum})
|
||||
}
|
||||
|
||||
func (s *Server) handleMetrics(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("Content-Type", "text/plain; version=0.0.4")
|
||||
msgs := int64(0)
|
||||
|
|
@ -281,7 +492,7 @@ func (s *Server) handleMetrics(w http.ResponseWriter, r *http.Request) {
|
|||
|
||||
// --- Web UI handlers ---
|
||||
|
||||
func (s *Server) handleUI(w http.ResponseWriter, r *http.Request) {
|
||||
func (s *Server) handleUIDash(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path != "/" {
|
||||
w.WriteHeader(http.StatusNotFound)
|
||||
return
|
||||
|
|
@ -293,125 +504,17 @@ func (s *Server) handleUI(w http.ResponseWriter, r *http.Request) {
|
|||
return
|
||||
}
|
||||
}
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
// Pico.css from CDN and a tiny app
|
||||
page := `<!doctype html>
|
||||
<html>
|
||||
<head>
|
||||
<meta charset="utf-8"/>
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1"/>
|
||||
<title>sojuboy</title>
|
||||
<link rel="stylesheet" href="https://unpkg.com/@picocss/pico@2/css/pico.min.css">
|
||||
<style>
|
||||
body { padding: 0; }
|
||||
header.nav { position: sticky; top: 0; z-index: 10; padding: .6rem 1rem; border-bottom: 1px solid var(--muted-border-color); display: flex; justify-content: space-between; align-items: center; }
|
||||
header.nav a.brand { text-decoration: none; font-weight: 600; }
|
||||
main.container { display: grid; grid-template-columns: 220px 1fr; gap: 0; min-height: calc(100vh - 3rem); }
|
||||
aside.sidebar { border-right: 1px solid var(--muted-border-color); padding: .75rem; overflow-y: auto; }
|
||||
aside.sidebar a { display:block; padding:.25rem .5rem; border-radius:.25rem; text-decoration:none; }
|
||||
aside.sidebar a.active { background: var(--muted-color); color: var(--contrast); }
|
||||
section.chat { padding: .75rem 1rem; display:flex; flex-direction: column; height: calc(100vh - 3.5rem); }
|
||||
#tail { flex: 1; overflow: auto; white-space: pre-wrap; word-break: break-word; overflow-wrap: anywhere; font-family: ui-monospace, SFMono-Regular, Menlo, monospace; }
|
||||
.ts { opacity: .66; }
|
||||
.msg { margin-bottom: .25rem; }
|
||||
footer { text-align: center; font-size: .85rem; padding: .5rem 0; opacity: .7; }
|
||||
@media (max-width: 900px) { main.container { grid-template-columns: 1fr; } aside.sidebar { display:none; } }
|
||||
</style>
|
||||
<script>
|
||||
const st={ tailLoading:false, atBottom:true, current:'#', earliest:null, sse:null, channels:[] };
|
||||
function setToken(v){ st.token=v; localStorage.setItem('token', v); }
|
||||
async function api(path, params){
|
||||
const url = new URL(path, window.location.origin);
|
||||
if(params && params.query){ Object.entries(params.query).forEach(([k,v])=>url.searchParams.set(k,v)); }
|
||||
const opts = { headers: {} };
|
||||
// use cookie for auth; header optional if present
|
||||
if(st.token){ opts.headers['Authorization'] = 'Bearer '+st.token; }
|
||||
const res = await fetch(url, opts);
|
||||
if(!res.ok){ throw new Error('HTTP '+res.status); }
|
||||
return res.json();
|
||||
}
|
||||
async function loadInfo(){ /* no-op for now; footer shows version */ }
|
||||
async function loadChannels(){
|
||||
try{ const data = await api('/api/channels'); st.channels = data; renderChannels(); if(data.length>0){ selectChannel(data[0]); } }
|
||||
catch(e){ console.error(e); }
|
||||
}
|
||||
function renderChannels(){ const list=document.getElementById('chanlist'); list.innerHTML=''; st.channels.forEach(c=>{ const a=document.createElement('a'); a.href='#'; a.textContent=c; a.onclick=(ev)=>{ev.preventDefault(); selectChannel(c)}; if(c===st.current) a.className='active'; list.appendChild(a); }); }
|
||||
async function selectChannel(ch){ if(st.sse){ st.sse.close(); st.sse=null; } st.current=ch; renderChannels(); const el=document.getElementById('tail'); el.textContent=''; // bootstrap 50
|
||||
const data = await api('/api/tail',{query:{channel:ch,limit:50}}); appendBatch(data); el.scrollTop = el.scrollHeight; st.atBottom=true; st.earliest = data.length? data[0].time : null; startStream(); initScrollHandlers(); }
|
||||
function initScrollHandlers(){ const el=document.getElementById('tail'); el.onscroll=async ()=>{ st.atBottom = (el.scrollTop + el.clientHeight + 8) >= el.scrollHeight; if(el.scrollTop === 0 && st.earliest){ // load more
|
||||
try{ const older = await api('/api/history',{query:{channel:st.current,before:st.earliest,limit:50}}); if(older.length){ const hBefore = el.scrollHeight; prependBatch(older); st.earliest = older[0].time; const hAfter = el.scrollHeight; el.scrollTop = hAfter - hBefore; } }
|
||||
catch(e){}
|
||||
} } }
|
||||
function colorFor(nick){ let h=0; for(let i=0;i<nick.length;i++){ h=(h*31+nick.charCodeAt(i))>>>0 } return 'hsl('+(h%360)+',60%,'+(window.matchMedia('(prefers-color-scheme: dark)').matches? '70%':'35%')+')'; }
|
||||
function lineHTML(m){ const ts = '<span class=ts>[' + m.time + ']</span>'; const nick = '<b style="color:' + colorFor(m.author) + '\">' + m.author + '</b>'; const body = escapeHtml(m.body); return ts + ' ' + nick + ': ' + linkify(body); }
|
||||
function escapeHtml(s){ return s.replace(/[&<>"]/g, c=>({'&':'&','<':'<','>':'>','"':'"'}[c])); }
|
||||
function linkify(t){ return t.replace(/https?:\/\/\S+/g, function(u){ return '<a href="' + u + '" target="_blank" rel="noopener">' + u + '</a>'; }); }
|
||||
function appendBatch(arr){ const el=document.getElementById('tail'); const frag=document.createDocumentFragment(); arr.forEach(m=>{ const div=document.createElement('div'); div.className='msg'; div.innerHTML=lineHTML(m); frag.appendChild(div); processLinks(div); }); el.appendChild(frag); if(st.atBottom){ el.scrollTop = el.scrollHeight; } }
|
||||
function prependBatch(arr){ const el=document.getElementById('tail'); const oldTop=el.firstChild; const frag=document.createDocumentFragment(); arr.forEach(m=>{ const div=document.createElement('div'); div.className='msg'; div.innerHTML=lineHTML(m); frag.appendChild(div); processLinks(div); }); el.insertBefore(frag, el.firstChild); if(oldTop){ oldTop.scrollIntoView(); } }
|
||||
function processLinks(scope){ const links = scope.querySelectorAll('a[href]:not([data-card])'); links.forEach(a=>{ a.setAttribute('data-card','1'); fetch('/api/linkcard?url='+encodeURIComponent(a.href)).then(r=>r.json()).then(card=>{ if(!card) return; if(card.title||card.description||card.image){ const c=document.createElement('div'); c.className='card'; var html=''; if(card.image){ html += '<div><img src="'+card.image+'" alt="" style="max-width:160px;max-height:120px;object-fit:cover;border-radius:.25rem"></div>'; } html += '<div style="flex:1;margin-left:.5rem">'; if(card.title){ html += '<div style="font-weight:600">'+escapeHtml(card.title)+'</div>'; } if(card.description){ html += '<div style="opacity:.8">'+escapeHtml(card.description)+'</div>'; } html += '</div>'; c.innerHTML = '<div style="display:flex;align-items:flex-start;gap:.5rem">'+html+'</div>'; a.parentNode.insertBefore(c, a.nextSibling); } }).catch(()=>{}); }); }
|
||||
function startStream(){ const url=new URL('/api/stream', window.location.origin); url.searchParams.set('channel', st.current); const es=new EventSource(url); st.sse=es; es.onmessage=(ev)=>{ try{ const m=JSON.parse(ev.data); appendBatch([m]); }catch(e){} }; es.onerror=()=>{ es.close(); st.sse=null; setTimeout(startStream, 3000); } }
|
||||
async function doSumm(){
|
||||
const ch = document.getElementById('channel').value;
|
||||
const win = document.getElementById('window').value || '6h';
|
||||
const push = document.getElementById('push').checked ? '1' : '0';
|
||||
const btn = document.getElementById('summBtn');
|
||||
const prog = document.getElementById('summProg');
|
||||
btn.disabled = true; prog.style.display = 'inline-block';
|
||||
try{ const data = await api('/api/trigger',{query:{channel:ch,window:win,push:push}});
|
||||
const el = document.getElementById('summary');
|
||||
el.textContent = data.summary || '(empty)';
|
||||
el.scrollTop = el.scrollHeight;
|
||||
}catch(e){ document.getElementById('summary').textContent = 'error: '+e; }
|
||||
btn.disabled = false; prog.style.display = 'none';
|
||||
}
|
||||
window.addEventListener('DOMContentLoaded', ()=>{ loadChannels(); });
|
||||
function onFollowToggle(cb){
|
||||
if(cb.checked){
|
||||
if(st.tailTimer) clearInterval(st.tailTimer);
|
||||
st.tailTimer = setInterval(doTail, 3000);
|
||||
}else{
|
||||
if(st.tailTimer) { clearInterval(st.tailTimer); st.tailTimer=null; }
|
||||
}
|
||||
}
|
||||
function onChannelChange(){ doTail(); }
|
||||
</script>
|
||||
</head>
|
||||
<body>
|
||||
<header class="nav">
|
||||
<div><a class="brand" href="/">sojuboy</a></div>
|
||||
<nav>
|
||||
<ul>
|
||||
<li><a href="/summarizer">Summarizer</a></li>
|
||||
<li><a href="/logout">Logout</a></li>
|
||||
</ul>
|
||||
</nav>
|
||||
</header>
|
||||
<main class="container">
|
||||
<aside class="sidebar">
|
||||
<nav id="chanlist"></nav>
|
||||
</aside>
|
||||
<section class="chat">
|
||||
<div id="tail" style="height: calc(100vh - 4.5rem); overflow:auto"></div>
|
||||
</section>
|
||||
</main>
|
||||
<footer>
|
||||
<small>` + s.Version + ` (` + s.Commit + `)</small>
|
||||
</footer>
|
||||
</body>
|
||||
</html>`
|
||||
_, _ = w.Write([]byte(page))
|
||||
s.render(w, "dashboard.tmpl", map[string]any{})
|
||||
}
|
||||
|
||||
func (s *Server) handleSummarizerUI(w http.ResponseWriter, r *http.Request) {
|
||||
func (s *Server) handleUISummarizer(w http.ResponseWriter, r *http.Request) {
|
||||
if s.AuthToken != "" {
|
||||
if c, err := r.Cookie("auth_token"); err != nil || c.Value != s.AuthToken {
|
||||
http.Redirect(w, r, "/login", http.StatusFound)
|
||||
return
|
||||
}
|
||||
}
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
page := `<!doctype html><html><head><meta charset="utf-8"/><meta name="viewport" content="width=device-width, initial-scale=1"/><title>Summarizer · sojuboy</title><link rel="stylesheet" href="https://unpkg.com/@picocss/pico@2/css/pico.min.css"><style>body{padding:0} header.nav{position:sticky;top:0;z-index:10;padding:.6rem 1rem;border-bottom:1px solid var(--muted-border-color);display:flex;justify-content:space-between;align-items:center} header.nav a.brand{text-decoration:none;font-weight:600} main{max-width:900px;margin:0 auto;padding:1rem} #out{white-space:pre-wrap;word-break:break-word;overflow-wrap:anywhere} footer{text-align:center;font-size:.85rem;padding:.5rem 0;opacity:.7}</style><script>async function api(path,params){const url=new URL(path,window.location.origin);if(params&¶ms.query){Object.entries(params.query).forEach(([k,v])=>url.searchParams.set(k,v));}const res=await fetch(url);if(!res.ok) throw new Error('HTTP '+res.status);return res.text();}async function summarize(){const ch=document.getElementById('channel').value;const win=document.getElementById('window').value||'6h';const push=document.getElementById('push').checked?'1':'0';const btn=document.getElementById('btn');const out=document.getElementById('out');btn.disabled=true;out.textContent='';try{const txt=await api('/trigger',{query:{channel:ch,window:win,push:push}});out.textContent=txt;}catch(e){out.textContent='error: '+e;}btn.disabled=false;}</script></head><body><header class="nav"><div><a class="brand" href="/">sojuboy</a></div><nav><ul><li><a href="/summarizer" aria-current="page">Summarizer</a></li><li><a href="/logout">Logout</a></li></ul></nav></header><main><article><h3>On-demand summarization</h3><label>Channel<select id="channel"></select></label><label>Window<input id="window" value="6h"></label><label><input type="checkbox" id="push"> Send via Pushover</label><button id="btn" onclick="summarize()">Summarize</button><pre id="out"></pre></article></main><footer><small>` + s.Version + ` (` + s.Commit + `)</small></footer><script>(async()=>{try{const res=await fetch('/api/channels');const arr=await res.json();const sel=document.getElementById('channel');arr.forEach(c=>{const o=document.createElement('option');o.value=c;o.textContent=c;sel.appendChild(o);});}catch(e){}})();</script></body></html>`
|
||||
_, _ = w.Write([]byte(page))
|
||||
s.render(w, "summarizer.tmpl", map[string]any{})
|
||||
}
|
||||
|
||||
func (s *Server) handleInfo(w http.ResponseWriter, r *http.Request) {
|
||||
|
|
@ -422,17 +525,34 @@ func (s *Server) handleInfo(w http.ResponseWriter, r *http.Request) {
|
|||
"builtAt": s.BuiltAt,
|
||||
"startedAt": s.StartedAt.Format(time.RFC3339),
|
||||
"uptime": time.Since(s.StartedAt).Round(time.Second).String(),
|
||||
"messagesIngested": func() int64 { if s.Metrics==nil {return 0}; return atomic.LoadInt64(&s.Metrics.MessagesIngested) }(),
|
||||
"notificationsSent": func() int64 { if s.Metrics==nil {return 0}; return atomic.LoadInt64(&s.Metrics.NotificationsSent) }(),
|
||||
"messagesPruned": func() int64 { if s.Metrics==nil {return 0}; return atomic.LoadInt64(&s.Metrics.MessagesPruned) }(),
|
||||
"connected": func() bool { if s.Metrics==nil {return false}; return atomic.LoadInt64(&s.Metrics.ConnectedGauge)==1 }(),
|
||||
"messagesIngested": func() int64 {
|
||||
if s.Metrics == nil {
|
||||
return 0
|
||||
}
|
||||
return atomic.LoadInt64(&s.Metrics.MessagesIngested)
|
||||
}(),
|
||||
"notificationsSent": func() int64 {
|
||||
if s.Metrics == nil {
|
||||
return 0
|
||||
}
|
||||
return atomic.LoadInt64(&s.Metrics.NotificationsSent)
|
||||
}(),
|
||||
"messagesPruned": func() int64 {
|
||||
if s.Metrics == nil {
|
||||
return 0
|
||||
}
|
||||
return atomic.LoadInt64(&s.Metrics.MessagesPruned)
|
||||
}(),
|
||||
"connected": func() bool {
|
||||
if s.Metrics == nil {
|
||||
return false
|
||||
}
|
||||
return atomic.LoadInt64(&s.Metrics.ConnectedGauge) == 1
|
||||
}(),
|
||||
}
|
||||
_ = json.NewEncoder(w).Encode(resp)
|
||||
}
|
||||
|
||||
// Summarizer simple page placeholder (will reuse existing summarizer flow)
|
||||
// removed duplicate handleSummarizerUI definition (consolidated earlier)
|
||||
|
||||
// SSE stream of new messages for a channel
|
||||
func (s *Server) handleStream(w http.ResponseWriter, r *http.Request) {
|
||||
if s.AuthToken != "" && !checkAuth(r, s.AuthToken) {
|
||||
|
|
@ -441,21 +561,31 @@ func (s *Server) handleStream(w http.ResponseWriter, r *http.Request) {
|
|||
return
|
||||
}
|
||||
ch := strings.TrimSpace(r.URL.Query().Get("channel"))
|
||||
if ch == "" { w.WriteHeader(http.StatusBadRequest); _, _ = w.Write([]byte("missing channel")); return }
|
||||
key := strings.ToLower(ch)
|
||||
if ch == "" {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
_, _ = w.Write([]byte("missing channel"))
|
||||
return
|
||||
}
|
||||
key := strings.ToLower(ch) // Normalize channel name to lowercase
|
||||
|
||||
w.Header().Set("Content-Type", "text/event-stream")
|
||||
w.Header().Set("Cache-Control", "no-cache")
|
||||
w.Header().Set("Connection", "keep-alive")
|
||||
flusher, ok := w.(http.Flusher)
|
||||
if !ok { w.WriteHeader(http.StatusInternalServerError); return }
|
||||
if !ok {
|
||||
w.WriteHeader(http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
// register subscriber
|
||||
if s.subs == nil { s.subs = make(map[string][]chan store.Message) }
|
||||
if s.subs == nil {
|
||||
s.subs = make(map[string][]chan store.Message)
|
||||
}
|
||||
sub := make(chan store.Message, 64)
|
||||
s.subsMu.Lock()
|
||||
s.subs[key] = append(s.subs[key], sub)
|
||||
s.subsMu.Unlock()
|
||||
defer func(){
|
||||
defer func() {
|
||||
s.subsMu.Lock()
|
||||
subs := s.subs[key]
|
||||
for i := range subs {
|
||||
|
|
@ -468,7 +598,7 @@ func (s *Server) handleStream(w http.ResponseWriter, r *http.Request) {
|
|||
close(sub)
|
||||
}()
|
||||
|
||||
hb := time.NewTicker(15 * time.Second)
|
||||
hb := time.NewTicker(15 * time.Second) // SSE heartbeat
|
||||
defer hb.Stop()
|
||||
ctx := r.Context()
|
||||
for {
|
||||
|
|
@ -517,14 +647,30 @@ func (s *Server) handleTailJSON(w http.ResponseWriter, r *http.Request) {
|
|||
return
|
||||
}
|
||||
channel := r.URL.Query().Get("channel")
|
||||
if channel == "" { w.WriteHeader(http.StatusBadRequest); _, _ = w.Write([]byte("missing channel")); return }
|
||||
if channel == "" {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
_, _ = w.Write([]byte("missing channel"))
|
||||
return
|
||||
}
|
||||
limit := getIntQuery(r, "limit", 100)
|
||||
msgs, err := s.Store.ListRecentMessages(r.Context(), channel, limit)
|
||||
if err != nil { w.WriteHeader(http.StatusInternalServerError); _, _ = w.Write([]byte("store error")); return }
|
||||
if err != nil {
|
||||
w.WriteHeader(http.StatusInternalServerError)
|
||||
_, _ = w.Write([]byte("store error"))
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
type outMsg struct{ Time string `json:"time"`; Author string `json:"author"`; Body string `json:"body"`; Channel string `json:"channel"` }
|
||||
type outMsg struct {
|
||||
Time string `json:"time"`
|
||||
Author string `json:"author"`
|
||||
Body string `json:"body"`
|
||||
Channel string `json:"channel"`
|
||||
}
|
||||
arr := make([]outMsg, 0, len(msgs))
|
||||
for i := len(msgs)-1; i>=0; i-- { m := msgs[i]; arr = append(arr, outMsg{Time: m.Time.UTC().Format(time.RFC3339), Author: m.Author, Body: m.Body, Channel: channel}) }
|
||||
for i := len(msgs) - 1; i >= 0; i-- {
|
||||
m := msgs[i]
|
||||
arr = append(arr, outMsg{Time: m.Time.UTC().Format(time.RFC3339), Author: m.Author, Body: m.Body, Channel: channel})
|
||||
}
|
||||
_ = json.NewEncoder(w).Encode(arr)
|
||||
}
|
||||
|
||||
|
|
@ -535,19 +681,52 @@ func (s *Server) handleTriggerJSON(w http.ResponseWriter, r *http.Request) {
|
|||
return
|
||||
}
|
||||
channel := r.URL.Query().Get("channel")
|
||||
if channel == "" { w.WriteHeader(http.StatusBadRequest); _, _ = w.Write([]byte("missing channel")); return }
|
||||
win := r.URL.Query().Get("window"); if win=="" { win = "6h" }
|
||||
if channel == "" {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
_, _ = w.Write([]byte("missing channel"))
|
||||
return
|
||||
}
|
||||
win := r.URL.Query().Get("window")
|
||||
if win == "" {
|
||||
win = "6h"
|
||||
}
|
||||
push := r.URL.Query().Get("push") == "1"
|
||||
dur, err := time.ParseDuration(win)
|
||||
if err != nil { w.WriteHeader(http.StatusBadRequest); _, _ = w.Write([]byte("bad window")); return }
|
||||
if err != nil {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
_, _ = w.Write([]byte("bad window"))
|
||||
return
|
||||
}
|
||||
msgs, err := s.Store.ListMessagesSince(r.Context(), channel, time.Now().Add(-dur))
|
||||
if err != nil { w.WriteHeader(http.StatusInternalServerError); _, _ = w.Write([]byte("store error")); return }
|
||||
if s.Summarizer == nil { w.WriteHeader(http.StatusServiceUnavailable); _, _ = w.Write([]byte("summarizer not configured")); return }
|
||||
tout := s.SummarizerTimeout; if tout<=0 { tout = 5*time.Minute }
|
||||
ctx, cancel := context.WithTimeout(r.Context(), tout); defer cancel()
|
||||
if err != nil {
|
||||
w.WriteHeader(http.StatusInternalServerError)
|
||||
_, _ = w.Write([]byte("store error"))
|
||||
return
|
||||
}
|
||||
if s.Summarizer == nil {
|
||||
w.WriteHeader(http.StatusServiceUnavailable)
|
||||
_, _ = w.Write([]byte("summarizer not configured"))
|
||||
return
|
||||
}
|
||||
tout := s.SummarizerTimeout
|
||||
if tout <= 0 {
|
||||
tout = 5 * time.Minute
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(r.Context(), tout)
|
||||
defer cancel()
|
||||
sum, err := s.Summarizer.Summarize(ctx, channel, msgs, dur)
|
||||
if err != nil { w.WriteHeader(http.StatusBadGateway); _, _ = w.Write([]byte("summarizer error")); return }
|
||||
if push && s.Notifier != nil { title := fmt.Sprintf("IRC digest %s (%s)", channel, dur); _ = s.Notifier.Notify(r.Context(), title, sum); if s.Metrics != nil { atomic.AddInt64(&s.Metrics.NotificationsSent, 1) } }
|
||||
if err != nil {
|
||||
w.WriteHeader(http.StatusBadGateway)
|
||||
_, _ = w.Write([]byte("summarizer error"))
|
||||
return
|
||||
}
|
||||
if push && s.Notifier != nil {
|
||||
title := fmt.Sprintf("IRC digest %s (%s)", channel, dur)
|
||||
_ = s.Notifier.Notify(r.Context(), title, sum)
|
||||
if s.Metrics != nil {
|
||||
atomic.AddInt64(&s.Metrics.NotificationsSent, 1)
|
||||
}
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_ = json.NewEncoder(w).Encode(map[string]any{"summary": sum})
|
||||
}
|
||||
|
|
@ -561,16 +740,35 @@ func (s *Server) handleHistory(w http.ResponseWriter, r *http.Request) {
|
|||
}
|
||||
channel := r.URL.Query().Get("channel")
|
||||
beforeStr := r.URL.Query().Get("before")
|
||||
if channel == "" || beforeStr == "" { w.WriteHeader(http.StatusBadRequest); _, _ = w.Write([]byte("missing params")); return }
|
||||
if channel == "" || beforeStr == "" {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
_, _ = w.Write([]byte("missing params"))
|
||||
return
|
||||
}
|
||||
t, err := time.Parse(time.RFC3339, beforeStr)
|
||||
if err != nil { w.WriteHeader(http.StatusBadRequest); _, _ = w.Write([]byte("bad before")); return }
|
||||
if err != nil {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
_, _ = w.Write([]byte("bad before"))
|
||||
return
|
||||
}
|
||||
limit := getIntQuery(r, "limit", 50)
|
||||
msgs, err := s.Store.ListMessagesBefore(r.Context(), channel, t, limit)
|
||||
if err != nil { w.WriteHeader(http.StatusInternalServerError); _, _ = w.Write([]byte("store error")); return }
|
||||
if err != nil {
|
||||
w.WriteHeader(http.StatusInternalServerError)
|
||||
_, _ = w.Write([]byte("store error"))
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
type outMsg struct{ Time string `json:"time"`; Author string `json:"author"`; Body string `json:"body"`; Channel string `json:"channel"` }
|
||||
type outMsg struct {
|
||||
Time string `json:"time"`
|
||||
Author string `json:"author"`
|
||||
Body string `json:"body"`
|
||||
Channel string `json:"channel"`
|
||||
}
|
||||
arr := make([]outMsg, 0, len(msgs))
|
||||
for _, m := range msgs { arr = append(arr, outMsg{Time: m.Time.UTC().Format(time.RFC3339), Author: m.Author, Body: m.Body, Channel: m.Channel}) }
|
||||
for _, m := range msgs {
|
||||
arr = append(arr, outMsg{Time: m.Time.UTC().Format(time.RFC3339), Author: m.Author, Body: m.Body, Channel: m.Channel})
|
||||
}
|
||||
_ = json.NewEncoder(w).Encode(arr)
|
||||
}
|
||||
|
||||
|
|
@ -647,7 +845,11 @@ func (s *Server) handleLogin(w http.ResponseWriter, r *http.Request) {
|
|||
}
|
||||
|
||||
func (s *Server) handleAuth(w http.ResponseWriter, r *http.Request) {
|
||||
if err := r.ParseForm(); err != nil { w.WriteHeader(http.StatusBadRequest); _, _ = w.Write([]byte("bad request")); return }
|
||||
if err := r.ParseForm(); err != nil {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
_, _ = w.Write([]byte("bad request"))
|
||||
return
|
||||
}
|
||||
tok := r.Form.Get("token")
|
||||
if tok == "" || s.AuthToken == "" || tok != s.AuthToken {
|
||||
w.WriteHeader(http.StatusUnauthorized)
|
||||
|
|
@ -657,12 +859,33 @@ func (s *Server) handleAuth(w http.ResponseWriter, r *http.Request) {
|
|||
// set cookie for 7 days
|
||||
maxAge := 7 * 24 * 60 * 60
|
||||
secure := r.TLS != nil || strings.EqualFold(r.Header.Get("X-Forwarded-Proto"), "https")
|
||||
http.SetCookie(w, &http.Cookie{Name:"auth_token", Value:tok, Path:"/", MaxAge:maxAge, HttpOnly:true, Secure:secure, SameSite:http.SameSiteLaxMode})
|
||||
http.SetCookie(w, &http.Cookie{Name: "auth_token", Value: tok, Path: "/", MaxAge: maxAge, HttpOnly: true, Secure: secure, SameSite: http.SameSiteLaxMode})
|
||||
w.Header().Set("Location", "/")
|
||||
w.WriteHeader(http.StatusFound)
|
||||
}
|
||||
|
||||
func (s *Server) handleLogout(w http.ResponseWriter, r *http.Request) {
|
||||
http.SetCookie(w, &http.Cookie{Name:"auth_token", Value:"", Path:"/", MaxAge:-1})
|
||||
http.SetCookie(w, &http.Cookie{Name: "auth_token", Value: "", Path: "/", MaxAge: -1})
|
||||
http.Redirect(w, r, "/login", http.StatusFound)
|
||||
}
|
||||
|
||||
// Broadcast sends a message to all SSE subscribers for a given channel.
|
||||
func (s *Server) Broadcast(channel string, m store.Message) {
|
||||
key := strings.ToLower(strings.TrimSpace(channel))
|
||||
if key == "" {
|
||||
return
|
||||
}
|
||||
s.subsMu.RLock()
|
||||
subs := s.subs[key]
|
||||
s.subsMu.RUnlock()
|
||||
for _, sub := range subs {
|
||||
select {
|
||||
case sub <- m:
|
||||
// ok
|
||||
default:
|
||||
if s.Logger != nil {
|
||||
s.Logger.Warn("sse subscriber buffer full, dropping message", "channel", key)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
|
|||
27
internal/httpapi/static/app.css
Normal file
27
internal/httpapi/static/app.css
Normal file
|
|
@ -0,0 +1,27 @@
|
|||
html, body { height: 100%; }
|
||||
body { padding: 0; min-height: 100vh; }
|
||||
header.nav { position: fixed; top: 0; left: 0; right: 0; z-index: 1000; padding: .6rem 1rem; border-bottom: 1px solid var(--muted-border-color); display: grid; grid-template-columns: 1fr auto 1fr; align-items: center; background-color: var(--pico-background-color, #fff); }
|
||||
header.nav .brand { justify-self: start; cursor: pointer; }
|
||||
header.nav .commit { justify-self: center; font-size: .7rem; opacity: .55; }
|
||||
header.nav nav { justify-self: end; }
|
||||
main, article, .dash { margin-top: var(--headerH, 56px); margin-bottom: var(--footerH, 44px); }
|
||||
header.nav a.brand { text-decoration: none; font-weight: 600; }
|
||||
/* Dashboard-only grid layout */
|
||||
.dash { display: grid; grid-template-columns: 1fr; gap: 0; }
|
||||
.dash aside.sidebar { border-right: 1px solid var(--muted-border-color); padding: .75rem; overflow-y: auto; }
|
||||
.dash aside.sidebar a { display:block; padding:.25rem .5rem; border-radius:.25rem; text-decoration:none; }
|
||||
.dash aside.sidebar a.active { background: var(--muted-color); color: var(--contrast); }
|
||||
.dash section.chat { padding: .75rem 1rem; display:flex; flex-direction: column; }
|
||||
#tail { flex: 1; white-space: pre-wrap; word-break: break-word; overflow-wrap: anywhere; font-family: ui-monospace, SFMono-Regular, Menlo, monospace; }
|
||||
.ts { opacity: .66; }
|
||||
.msg { margin-bottom: .25rem; }
|
||||
footer { position: fixed; left: 0; right: 0; bottom: 0; z-index: 1000; text-align: center; font-size: .85rem; padding: .5rem 0; background-color: var(--pico-background-color, #fff); border-top: 1px solid var(--muted-border-color); }
|
||||
@media (max-width: 900px) { .dash { grid-template-columns: 1fr; } .dash aside.sidebar { display:none; } }
|
||||
/* Summarizer output wrapping */
|
||||
#out { white-space: pre-wrap; word-break: break-word; overflow-wrap: anywhere; }
|
||||
|
||||
/* Cards */
|
||||
.card { margin:.35rem 0; padding:.5rem; border:1px solid var(--muted-border-color); border-radius:.5rem; background: var(--pico-card-background-color, rgba(0,0,0,0.02)); }
|
||||
.card .card-head { display:flex; align-items:center; justify-content:space-between; gap:.5rem; cursor:pointer; }
|
||||
.card .card-actions { display:flex; align-items:center; gap:.4rem; }
|
||||
.card .summary-title { font-weight:600; margin-top:.25rem; opacity:.85; }
|
||||
114
internal/httpapi/static/app.js
Normal file
114
internal/httpapi/static/app.js
Normal file
|
|
@ -0,0 +1,114 @@
|
|||
// Shared state
|
||||
const st = { tailLoading: false, atBottom: true, current: '#', earliest: null, sse: null, channels: [], twLoaded: false, seen: new Set(), loadingHistory: false };
|
||||
|
||||
function measureBars(){
|
||||
const hdr = document.querySelector('header.nav');
|
||||
const ftr = document.querySelector('footer');
|
||||
if(hdr){ document.documentElement.style.setProperty('--headerH', hdr.getBoundingClientRect().height+'px'); }
|
||||
if(ftr){ document.documentElement.style.setProperty('--footerH', ftr.getBoundingClientRect().height+'px'); }
|
||||
}
|
||||
|
||||
function colorFor(nick){ let h=0; for(let i=0;i<nick.length;i++){ h=(h*31+nick.charCodeAt(i))>>>0 } return 'hsl('+(h%360)+',60%,'+(window.matchMedia('(prefers-color-scheme: dark)').matches? '70%':'35%')+')'; }
|
||||
function escapeHtml(s){ return s.replace(/[&<>"]/g, c=>({'&':'&','<':'<','>':'>','"':'"'}[c])); }
|
||||
function linkify(t){ return t.replace(/https?:\/\/\S+/g, function(u){ return '<a href="' + u + '" target="_blank" rel="noopener">' + u + '</a>'; }); }
|
||||
function lineHTML(m){ const ts = '<span class=ts>[' + m.time + ']</span>'; const nick = '<b style="color:' + colorFor(m.author) + '">' + m.author + '</b>'; const body = escapeHtml(m.body); return ts + ' ' + nick + ': ' + linkify(body); }
|
||||
|
||||
function msgKey(m){ return m.time + '|' + m.author + '|' + m.body; }
|
||||
|
||||
function snapBottom(){ window.scrollTo(0, document.documentElement.scrollHeight); }
|
||||
function pinBottomMulti(){ if(!st.atBottom) return; [0,16,64,200].forEach(d=> setTimeout(()=>requestAnimationFrame(snapBottom), d)); }
|
||||
|
||||
async function api(path, params){
|
||||
const url = new URL(path, window.location.origin);
|
||||
if(params && params.query){ Object.entries(params.query).forEach(([k,v])=>url.searchParams.set(k,v)); }
|
||||
const res = await fetch(url);
|
||||
if(!res.ok) throw new Error('HTTP '+res.status);
|
||||
const ct = res.headers.get('content-type')||'';
|
||||
if(ct.includes('application/json')) return res.json();
|
||||
return res.text();
|
||||
}
|
||||
|
||||
function ensureTwitterWidgets(){ if(st.twLoaded) { if(window.twttr && twttr.widgets && typeof twttr.widgets.load==='function'){ /* keep for later calls */ } return; } st.twLoaded = true; const s=document.createElement('script'); s.async=true; s.src='https://platform.twitter.com/widgets.js'; document.head.appendChild(s); }
|
||||
|
||||
function appendBatch(arr){ const el=document.getElementById('tail'); const frag=document.createDocumentFragment(); arr.forEach(m=>{ const key=msgKey(m); if(st.seen.has(key)) return; st.seen.add(key); const div=document.createElement('div'); div.className='msg'; div.dataset.key=key; div.innerHTML=lineHTML(m); frag.appendChild(div); processLinks(div); }); el.appendChild(frag); pinBottomMulti(); }
|
||||
function prependBatch(arr){ const el=document.getElementById('tail'); const oldTop=el.firstChild; const beforeTop = oldTop ? oldTop.getBoundingClientRect().top : 0; const frag=document.createDocumentFragment(); arr.forEach(m=>{ const key=msgKey(m); if(st.seen.has(key)) return; st.seen.add(key); const div=document.createElement('div'); div.className='msg'; div.dataset.key=key; div.innerHTML=lineHTML(m); frag.appendChild(div); processLinks(div); }); el.insertBefore(frag, el.firstChild); if(oldTop){ const afterTop = oldTop.getBoundingClientRect().top; const delta = afterTop - beforeTop; window.scrollBy(0, delta); } }
|
||||
|
||||
function processLinks(scope){ const links = scope.querySelectorAll('a[href]:not([data-card])'); links.forEach(a=>{ a.setAttribute('data-card','1');
|
||||
// Fetch and render card
|
||||
fetch('/api/linkcard?url='+encodeURIComponent(a.href)).then(r=>r.json()).then(card=>{
|
||||
if(!card) return;
|
||||
if(card.title||card.description||card.image||card.html){
|
||||
const c=document.createElement('div'); c.className='card';
|
||||
const details=document.createElement('div'); details.className='card-details';
|
||||
var html='';
|
||||
if(card.image){ html += '<div><img src="'+card.image+'" alt="" style="max-width:160px;max-height:120px;object-fit:cover;border-radius:.25rem"></div>'; }
|
||||
html += '<div style="flex:1;margin-left:.5rem">';
|
||||
if(card.title){ html += '<div style="font-weight:600">'+escapeHtml(card.title)+'</div>'; }
|
||||
if(card.description){ html += '<div style="opacity:.8">'+escapeHtml(card.description)+'</div>'; }
|
||||
html += '</div>';
|
||||
const row=document.createElement('div'); row.style.display='flex'; row.style.alignItems='flex-start'; row.style.gap='.5rem'; row.innerHTML=html;
|
||||
details.appendChild(row);
|
||||
if(card.html){
|
||||
const wrap=document.createElement('div');
|
||||
wrap.innerHTML=card.html;
|
||||
// Tweak YouTube iframe sizing
|
||||
const ifr=wrap.querySelector('iframe');
|
||||
if(ifr){ ifr.removeAttribute('width'); ifr.removeAttribute('height'); ifr.style.width='100%'; ifr.style.maxWidth='640px'; ifr.style.aspectRatio='16/9'; ifr.style.height='auto'; ifr.style.borderRadius='.5rem'; }
|
||||
details.appendChild(wrap);
|
||||
// Twitter embed render pass
|
||||
ensureTwitterWidgets();
|
||||
if(window.twttr && twttr.widgets && typeof twttr.widgets.load==='function'){ try{ twttr.widgets.load(wrap); }catch(e){} }
|
||||
}
|
||||
details.querySelectorAll('img').forEach(img=> img.addEventListener('load', ()=> pinBottomMulti()));
|
||||
c.appendChild(details);
|
||||
const sum=document.createElement('div'); sum.className='link-summary'; sum.style.whiteSpace='pre-wrap'; sum.style.marginTop='.25rem'; sum.style.display='none'; c.appendChild(sum);
|
||||
// Place actions inline next to the original link
|
||||
const act=document.createElement('span'); act.style.marginLeft='.35rem';
|
||||
const sumBtn=document.createElement('button'); sumBtn.type='button'; sumBtn.title='Summarize'; sumBtn.textContent='🌚'; sumBtn.style.padding='0 .35rem'; sumBtn.style.fontSize='.9rem';
|
||||
const chevron=document.createElement('button'); chevron.type='button'; chevron.title='Expand/collapse'; chevron.textContent='▾'; chevron.style.padding='0 .35rem'; chevron.style.fontSize='.9rem';
|
||||
const spinner=document.createElement('span'); spinner.textContent=''; spinner.style.marginLeft='.25rem';
|
||||
act.appendChild(sumBtn); act.appendChild(chevron); act.appendChild(spinner);
|
||||
a.insertAdjacentElement('afterend', act);
|
||||
// Insert card after the link line
|
||||
a.parentNode.insertBefore(c, act.nextSibling);
|
||||
const toggle = ()=>{ const hidden = c.style.display==='none'; c.style.display = hidden? '' : 'none'; chevron.textContent = hidden? '▾':'▸'; pinBottomMulti(); };
|
||||
chevron.onclick = (ev)=>{ ev.stopPropagation(); toggle(); };
|
||||
sumBtn.onclick = async ()=>{
|
||||
if(sum.style.display!== 'none' && sum.textContent){ sum.style.display='none'; sumBtn.textContent='🌚'; pinBottomMulti(); return; }
|
||||
// Ensure card is visible when showing summary
|
||||
if(c.style.display==='none'){ toggle(); }
|
||||
sumBtn.disabled=true; spinner.textContent='…'; sum.textContent=''; sum.style.display='';
|
||||
try{ const data = await api('/api/linksummary',{query:{url:a.href}}); sum.textContent = (data && data.summary) ? data.summary : '(no summary)'; sumBtn.textContent='🌝'; }
|
||||
catch(e){ sum.textContent = 'error: '+e; }
|
||||
spinner.textContent=''; sumBtn.disabled=false; pinBottomMulti();
|
||||
};
|
||||
}
|
||||
}).catch(()=>{});
|
||||
}); }
|
||||
|
||||
async function loadChannels(){ try{ const data = await api('/api/channels'); st.channels = data; renderChannels(); if(data.length>0){ selectChannel(data[0]); } } catch(e){} }
|
||||
function renderChannels(){ const list=document.getElementById('nav-chans') || document.getElementById('brand-chans')?.querySelector('ul'); if(!list) return; list.innerHTML=''; st.channels.forEach(c=>{ const li=document.createElement('li'); const a=document.createElement('a'); a.href='#'; a.textContent=c + (c===st.current? ' ✓':''); a.onclick=(ev)=>{ev.preventDefault(); selectChannel(c);}; li.appendChild(a); list.appendChild(li); }); }
|
||||
|
||||
async function selectChannel(ch){ if(st.sse){ st.sse.close(); st.sse=null; } st.current=ch; st.seen = new Set(); st.loadingHistory = false; renderChannels(); const el=document.getElementById('tail'); if(!el) return; el.textContent=''; const data = await api('/api/tail',{query:{channel:ch,limit:50}}); appendBatch(data); requestAnimationFrame(()=>{ snapBottom(); st.atBottom=true; }); st.earliest = data.length? data[0].time : null; startStream(); initScrollHandlers(); }
|
||||
function initScrollHandlers(){ const el=document.getElementById('tail'); if(!el) return; const onScroll = async ()=>{ const nearBottom = (window.innerHeight + window.pageYOffset + 16) >= document.documentElement.scrollHeight; st.atBottom = nearBottom; const nearTop = window.pageYOffset <= 4; if(nearTop && st.earliest && !st.loadingHistory){ try{ st.loadingHistory = true; const before = st.earliest; const older = await api('/api/history',{query:{channel:st.current,before:before,limit:50}}); if(older.length){ prependBatch(older); st.earliest = older[0].time; } } catch(e){} finally { st.loadingHistory = false; } } }; window.removeEventListener('scroll', st._scrollHandler || (()=>{})); st._scrollHandler = onScroll; window.addEventListener('scroll', onScroll, {passive:true}); }
|
||||
|
||||
function startStream(){ const el=document.getElementById('tail'); if(!el) return; const url=new URL('/api/stream', window.location.origin); url.searchParams.set('channel', st.current); const es=new EventSource(url); st.sse=es; es.onmessage=(ev)=>{ try{ const m=JSON.parse(ev.data); appendBatch([m]); }catch(e){} }; es.onerror=()=>{ es.close(); st.sse=null; setTimeout(startStream, 3000); } }
|
||||
|
||||
async function summarize(){ const ch=document.getElementById('channel'); const win=document.getElementById('window'); const push=document.getElementById('push'); const btn=document.getElementById('btn'); const out=document.getElementById('out'); if(!ch||!win||!btn||!out) return; btn.disabled=true; out.textContent=''; try{ const data = await api('/api/trigger',{query:{channel:ch.value,window:win.value||'6h',push:push && push.checked? '1':'0'}}); if(typeof data === 'string'){ out.textContent = data; } else { out.textContent = (data.summary||''); } } catch(e){ out.textContent = 'error: '+e; } btn.disabled=false; }
|
||||
|
||||
window.addEventListener('DOMContentLoaded', ()=>{
|
||||
if('scrollRestoration' in history){ history.scrollRestoration = 'manual'; }
|
||||
measureBars();
|
||||
loadChannels();
|
||||
// Open brand dropdown on hover (desktop) for the dashboard
|
||||
const brandDetails = document.getElementById('brand-chans');
|
||||
if(brandDetails){
|
||||
brandDetails.addEventListener('mouseenter', ()=>{ brandDetails.setAttribute('open',''); });
|
||||
brandDetails.addEventListener('mouseleave', ()=>{ brandDetails.removeAttribute('open'); });
|
||||
}
|
||||
if(document.getElementById('channel')){
|
||||
fetch('/api/channels').then(r=>r.json()).then(arr=>{ const sel=document.getElementById('channel'); arr.forEach(c=>{ const o=document.createElement('option'); o.value=c; o.textContent=c; sel.appendChild(o); }); }).catch(()=>{});
|
||||
}
|
||||
});
|
||||
window.addEventListener('load', measureBars);
|
||||
window.addEventListener('resize', ()=>{ measureBars(); if(st.atBottom) pinBottomMulti(); });
|
||||
54
internal/httpapi/templates.go
Normal file
54
internal/httpapi/templates.go
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
package httpapi
|
||||
|
||||
import (
|
||||
"embed"
|
||||
"html/template"
|
||||
"net/http"
|
||||
"path"
|
||||
"strings"
|
||||
"sync"
|
||||
)
|
||||
|
||||
//go:embed templates/*.tmpl
|
||||
var uiFS embed.FS
|
||||
|
||||
//go:embed static/*
|
||||
var staticFS embed.FS
|
||||
|
||||
var (
|
||||
tplOnce sync.Once
|
||||
tpl *template.Template
|
||||
)
|
||||
|
||||
func (s *Server) parseTemplatesOnce() {
|
||||
tplOnce.Do(func() {
|
||||
// Parse all templates under templates/
|
||||
t := template.New("base").Funcs(template.FuncMap{})
|
||||
t = template.Must(t.ParseFS(uiFS, "templates/*.tmpl"))
|
||||
tpl = t
|
||||
})
|
||||
}
|
||||
|
||||
func (s *Server) render(w http.ResponseWriter, name string, data map[string]any) {
|
||||
s.parseTemplatesOnce()
|
||||
if data == nil {
|
||||
data = map[string]any{}
|
||||
}
|
||||
data["Version"] = s.Version
|
||||
data["Commit"] = s.Commit
|
||||
if s.Commit != "" {
|
||||
c := s.Commit
|
||||
if len(c) > 12 { c = c[:12] }
|
||||
data["CommitShort"] = c
|
||||
}
|
||||
base := strings.TrimSuffix(path.Base(name), path.Ext(name))
|
||||
if base == "dashboard" {
|
||||
data["Title"] = "sojuboy"
|
||||
} else {
|
||||
data["Title"] = strings.Title(base)
|
||||
}
|
||||
// Tell layout which content template to include
|
||||
data["Content"] = base
|
||||
w.Header().Set("Content-Type", "text/html; charset=utf-8")
|
||||
_ = tpl.ExecuteTemplate(w, "layout.tmpl", data)
|
||||
}
|
||||
9
internal/httpapi/templates/dashboard.tmpl
Normal file
9
internal/httpapi/templates/dashboard.tmpl
Normal file
|
|
@ -0,0 +1,9 @@
|
|||
{{ define "dashboard" }}
|
||||
<main class="dash single">
|
||||
<section class="chat" style="grid-column: 1 / -1">
|
||||
<div id="tail"></div>
|
||||
</section>
|
||||
</main>
|
||||
{{ end }}
|
||||
{{/* layout executes us via .Content */}}
|
||||
|
||||
41
internal/httpapi/templates/layout.tmpl
Normal file
41
internal/httpapi/templates/layout.tmpl
Normal file
|
|
@ -0,0 +1,41 @@
|
|||
{{ define "layout.tmpl" }}
|
||||
<!doctype html>
|
||||
<html>
|
||||
<head>
|
||||
<meta charset="utf-8"/>
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1"/>
|
||||
<title>{{ .Title }} · sojuboy</title>
|
||||
<link rel="stylesheet" href="https://unpkg.com/@picocss/pico@2/css/pico.min.css">
|
||||
<link rel="stylesheet" href="/static/app.css">
|
||||
<script defer src="/static/app.js"></script>
|
||||
</head>
|
||||
<body>
|
||||
<header class="nav">
|
||||
<div>
|
||||
{{ if eq .Content "dashboard" }}
|
||||
<details id="brand-chans" role="list">
|
||||
<summary class="brand" aria-haspopup="listbox">sojuboy</summary>
|
||||
<ul id="nav-chans" role="listbox"></ul>
|
||||
</details>
|
||||
{{ else }}
|
||||
<a class="brand" href="/">sojuboy</a>
|
||||
{{ end }}
|
||||
</div>
|
||||
<div class="commit">{{ .CommitShort }}</div>
|
||||
<nav>
|
||||
<ul>
|
||||
<li><a href="/summarizer">Summarizer</a></li>
|
||||
<li><a href="/logout">Logout</a></li>
|
||||
</ul>
|
||||
</nav>
|
||||
</header>
|
||||
{{ if eq .Content "dashboard" }}
|
||||
{{ template "dashboard" . }}
|
||||
{{ else if eq .Content "summarizer" }}
|
||||
{{ template "summarizer" . }}
|
||||
{{ end }}
|
||||
|
||||
</body>
|
||||
</html>
|
||||
{{ end }}
|
||||
|
||||
14
internal/httpapi/templates/summarizer.tmpl
Normal file
14
internal/httpapi/templates/summarizer.tmpl
Normal file
|
|
@ -0,0 +1,14 @@
|
|||
{{ define "summarizer" }}
|
||||
<main class="container">
|
||||
<article>
|
||||
<h3>On-demand summarization</h3>
|
||||
<label>Channel<select id="channel"></select></label>
|
||||
<label>Window<input id="window" value="6h"></label>
|
||||
<label><input type="checkbox" id="push"> Send via Pushover</label>
|
||||
<button id="btn" onclick="summarize()">Summarize</button>
|
||||
<pre id="out"></pre>
|
||||
</article>
|
||||
</main>
|
||||
{{ end }}
|
||||
{{/* layout executes us via .Content */}}
|
||||
|
||||
|
|
@ -2,6 +2,7 @@ package summarizer
|
|||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
|
|
@ -161,6 +162,152 @@ func (o *OpenAI) Summarize(ctx context.Context, channel string, msgs []store.Mes
|
|||
return out, nil
|
||||
}
|
||||
|
||||
func (o *OpenAI) SummarizeLink(ctx context.Context, rawURL string) (string, error) {
|
||||
if o == nil || o.apiKey == "" {
|
||||
return "", nil
|
||||
}
|
||||
cfg := openai.DefaultConfig(o.apiKey)
|
||||
if strings.TrimSpace(o.baseURL) != "" {
|
||||
cfg.BaseURL = o.baseURL
|
||||
}
|
||||
client := openai.NewClientWithConfig(cfg)
|
||||
|
||||
content := ""
|
||||
title := ""
|
||||
img := ""
|
||||
|
||||
lu, _ := url.Parse(rawURL)
|
||||
host := strings.ToLower(lu.Host)
|
||||
isYouTube := host == "www.youtube.com" || host == "youtube.com" || host == "m.youtube.com" || host == "youtu.be"
|
||||
|
||||
ua := "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0 Safari/537.36"
|
||||
accept := "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
|
||||
|
||||
if isImageURL(rawURL) {
|
||||
img = rawURL
|
||||
} else if isYouTube {
|
||||
// YouTube: try oEmbed for title + thumbnail
|
||||
watchURL := rawURL
|
||||
if host == "youtu.be" {
|
||||
id := strings.TrimPrefix(lu.Path, "/")
|
||||
watchURL = "https://www.youtube.com/watch?v=" + id
|
||||
}
|
||||
ctx2, cancel := context.WithTimeout(ctx, o.linkTimeout)
|
||||
defer cancel()
|
||||
oembed := "https://www.youtube.com/oembed?format=json&url=" + url.QueryEscape(watchURL)
|
||||
req, _ := http.NewRequestWithContext(ctx2, http.MethodGet, oembed, nil)
|
||||
req.Header.Set("User-Agent", ua)
|
||||
req.Header.Set("Accept", accept)
|
||||
if resp, err := http.DefaultClient.Do(req); err == nil {
|
||||
func() {
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
|
||||
var oem struct {
|
||||
Title string `json:"title"`
|
||||
Thumb string `json:"thumbnail_url"`
|
||||
}
|
||||
if err := json.NewDecoder(resp.Body).Decode(&oem); err == nil {
|
||||
if oem.Title != "" {
|
||||
title = oem.Title
|
||||
}
|
||||
if oem.Thumb != "" {
|
||||
img = oem.Thumb
|
||||
}
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
// No robust transcript grab here; rely on model generalization + title
|
||||
} else if o.followLinks {
|
||||
ctx2, cancel := context.WithTimeout(ctx, o.linkTimeout)
|
||||
defer cancel()
|
||||
req, err := http.NewRequestWithContext(ctx2, http.MethodGet, rawURL, nil)
|
||||
if err == nil {
|
||||
req.Header.Set("User-Agent", ua)
|
||||
req.Header.Set("Accept", accept)
|
||||
req.Header.Set("Accept-Language", "en-US,en;q=0.9")
|
||||
resp, err := http.DefaultClient.Do(req)
|
||||
if err == nil {
|
||||
func() {
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
|
||||
lr := &io.LimitedReader{R: resp.Body, N: int64(o.linkMaxBytes)}
|
||||
b, _ := io.ReadAll(lr)
|
||||
text := string(b)
|
||||
if base, perr := url.Parse(rawURL); perr == nil {
|
||||
if art, err := readability.FromReader(strings.NewReader(text), base); err == nil {
|
||||
if at := strings.TrimSpace(art.TextContent); at != "" {
|
||||
text = at
|
||||
if title == "" && strings.TrimSpace(art.Title) != "" {
|
||||
title = strings.TrimSpace(art.Title)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
text = strings.ReplaceAll(text, "\r", "")
|
||||
text = strings.TrimSpace(text)
|
||||
if len(text) > 6000 {
|
||||
text = text[:6000]
|
||||
}
|
||||
content = text
|
||||
}
|
||||
}()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Build link-specific prompt
|
||||
sys := "You summarize the content at a single URL. You are given extracted text, title, or image/thumbnail. If the extract is limited, infer the best short summary from what’s available. Do not say you can’t open links or ask for more text; if there’s truly nothing usable, return '(no summary)'. Be concise and natural."
|
||||
var userParts []openai.ChatMessagePart
|
||||
b := strings.Builder{}
|
||||
b.WriteString("URL: ")
|
||||
b.WriteString(rawURL)
|
||||
b.WriteString("\n")
|
||||
if title != "" {
|
||||
b.WriteString("Title: ")
|
||||
b.WriteString(title)
|
||||
b.WriteString("\n")
|
||||
}
|
||||
b.WriteString("\n")
|
||||
if content != "" {
|
||||
b.WriteString("Extracted content (may be truncated):\n")
|
||||
b.WriteString(content)
|
||||
b.WriteString("\n\n")
|
||||
}
|
||||
b.WriteString("Write a short, skimmable summary of the page/video/image above. If relevant, include key takeaways and any notable cautions. Keep it under a few short paragraphs.")
|
||||
userParts = append(userParts, openai.ChatMessagePart{Type: openai.ChatMessagePartTypeText, Text: b.String()})
|
||||
if img != "" {
|
||||
userParts = append(userParts, openai.ChatMessagePart{Type: openai.ChatMessagePartTypeImageURL, ImageURL: &openai.ChatMessageImageURL{URL: img}})
|
||||
}
|
||||
|
||||
model := o.model
|
||||
if strings.TrimSpace(model) == "" {
|
||||
model = "gpt-4o-mini"
|
||||
}
|
||||
reasoningLike := strings.HasPrefix(model, "gpt-5") || strings.HasPrefix(model, "o1") || strings.Contains(model, "reasoning")
|
||||
|
||||
req := openai.ChatCompletionRequest{
|
||||
Model: model,
|
||||
Messages: []openai.ChatCompletionMessage{
|
||||
{Role: openai.ChatMessageRoleSystem, Content: sys},
|
||||
{Role: openai.ChatMessageRoleUser, MultiContent: userParts},
|
||||
},
|
||||
MaxCompletionTokens: o.maxTokens,
|
||||
}
|
||||
if !reasoningLike {
|
||||
req.Temperature = 0.2
|
||||
}
|
||||
|
||||
resp, err := client.CreateChatCompletion(ctx, req)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if len(resp.Choices) == 0 {
|
||||
return "", nil
|
||||
}
|
||||
return strings.TrimSpace(resp.Choices[0].Message.Content), nil
|
||||
}
|
||||
|
||||
func linksFromImages(imgs []string) []linkSnippet {
|
||||
out := make([]linkSnippet, 0, len(imgs))
|
||||
for _, u := range imgs {
|
||||
|
|
|
|||
|
|
@ -9,6 +9,5 @@ import (
|
|||
|
||||
type Summarizer interface {
|
||||
Summarize(ctx context.Context, channel string, msgs []store.Message, window time.Duration) (string, error)
|
||||
SummarizeLink(ctx context.Context, rawURL string) (string, error)
|
||||
}
|
||||
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue